diff --git a/.agents/skills/dsh-code-review/SKILL.md b/.agents/skills/dsh-code-review/SKILL.md index 7d4dcd6cd0..54510133db 100644 --- a/.agents/skills/dsh-code-review/SKILL.md +++ b/.agents/skills/dsh-code-review/SKILL.md @@ -7,7 +7,7 @@ description: Use when reviewing a pull request in the deepseek-harness repo — **This skill is guidance, not a complete checklist.** It is a where-to-look map that lowers your startup cost on an unfamiliar PR — clearing every item here does not mean the PR is good. You are the reviewer: reason independently from the code in front of you, and think broadly across every dimension a change can fail on. The items below are the failure modes this repo has already paid for; a real review also catches the ones nobody has written down yet. -Independent judgment governs *what to look at* and *how to apply a rule to this case* — not whether the repo's documented requirements still hold. AGENTS.md, packages/AGENTS.md, and the [quality gates](../../../docs/rfc/implemented/2026-06-11-quality-gates.md) remain authoritative; a missing HMR-safety test or out-of-sync docs is a blocking gap regardless of your judgment, not a suggestion you can waive. Use your own reasoning to go *beyond* these checks and to weigh genuine edge cases against an RFC (raise it as a discussion, don't silently override) — never to demote a documented blocker to optional. +Independent judgment governs *what to look at* and *how to apply a rule to this case* — not whether the repo's documented requirements still hold. AGENTS.md, packages/AGENTS.md, and the [quality gates](../../../docs/rfc/implemented/process/2026-06-11-quality-gates.md) remain authoritative; a missing HMR-safety test or out-of-sync docs is a blocking gap regardless of your judgment, not a suggestion you can waive. Use your own reasoning to go *beyond* these checks and to weigh genuine edge cases against an RFC (raise it as a discussion, don't silently override) — never to demote a documented blocker to optional. ## How to think about a review @@ -25,15 +25,16 @@ These define the conventions and gates this repo is checked against, and they ar - **AGENTS.md § Defensive patterns (hard-won)** — each bullet is a bug class that bit us. Reviewing anything touching process lifecycle, async/await, disposal, or adapter error paths? Re-read this first — then look for the *adjacent* mistake it doesn't name. - **AGENTS.md § Type Safety and Documentation** — the doc-sync rule (code change ⇒ update README + JSDoc in the SAME commit) and the no-hard-wrap markdown convention. - **[packages/AGENTS.md](../../../packages/AGENTS.md)** — per-package conventions (file layout, the HMR-safety test requirement). -- **[RFC index](../../../docs/rfc/README.md)** — the *why* behind the architecture. Especially [quality gates](../../../docs/rfc/implemented/2026-06-11-quality-gates.md) (what a PR must pass) and [capability seams](../../../docs/rfc/implemented/2026-06-13-capability-seams.md) (the three-package split). If a change seems to fight an RFC, that's a discussion, not a silent override — and not an automatic veto either: an RFC can be wrong for this case, so reason about it. +- **[RFC index](../../../docs/rfc/README.md)** — the *why* behind the architecture. Especially [quality gates](../../../docs/rfc/implemented/process/2026-06-11-quality-gates.md) (what a PR must pass) and [capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md) (the three-package split). If a change seems to fight an RFC, that's a discussion, not a silent override — and not an automatic veto either: an RFC can be wrong for this case, so reason about it. ## Hard blockers (documented requirements — missing one blocks merge) These come straight from the source docs above. They are not discretionary; absence is a blocking gap. -1. **Docs in sync.** If the PR changes a config key, default, error code, wire field, or event name, it must update the package README + module/JSDoc in the same diff. The `doc-sync` gate (check #3) does not catch prose drift in config keys, defaults, error codes, or wire fields — that is on the reviewer, but it is still required, not optional. -2. **HMR-safety test.** Any new registry/registration needs a test that disposes the contributing fiber and asserts cleanup (packages/AGENTS.md). Its absence blocks merge. -3. **Quality gates pass.** typecheck, lint, test, test:coverage (100% per-file on `packages/*/src`), knip, build, publint, constraints, `doc-sync` (doc-typecheck + verify-event-taxonomy + verify-md-wrap), module-graph freshness (the quality-gates RFC). Don't re-review what a gate already enforces — trust the gate and spend attention on what it can't check. Note that the `doc-sync` gate only covers compilable `ts` blocks, the event-taxonomy table, and markdown wrapping; prose drift (check #1) is *additional* manual review on top of it, not covered by it. +1. **Docs in sync.** If the PR changes a config key, default, error code, wire field, or event name, it must update the package README + module/JSDoc in the same diff. The `doc-sync` gate (check #4) does not catch prose drift in config keys, defaults, error codes, or wire fields — that is on the reviewer, but it is still required, not optional. +2. **Core-data-structures catalog in sync.** If the PR adds, removes, or reshapes a type the [core-data-structures catalog](../../../docs/core-data-structures/core.md) documents — a new `…Map` variant, a new content-block/session-event type, a field on `GenerateOptions`/`Agent`/`ToolDefinition`/a bash type, or a whole new core/seam type — it must update that catalog in the same diff (prose + any verbatim ` ```ts type-equiv ` block + the 1:1 `scripts/type-equiv.manifest.json`). The `verify-type-equiv` gate (part of `doc-sync`) catches a *drifted paste* of an already-documented type, but it cannot tell you a brand-new core type went undocumented — that judgment is yours. Confirm a genuinely spine-level type landed in core.md and a new capability's vocabulary on a sub-page, per the spine-vs-seam line in [core.md § What counts as "core"](../../../docs/core-data-structures/core.md#what-counts-as-core). A pure internal type with no cross-package reach needs no catalog entry — say so if it's a judgment call. +3. **HMR-safety test.** Any new registry/registration needs a test that disposes the contributing fiber and asserts cleanup (packages/AGENTS.md). Its absence blocks merge. +4. **Quality gates pass.** typecheck, lint, test, test:coverage (100% per-file on `packages/*/src`), knip, build, publint, constraints, `doc-sync` (doc-typecheck + verify-cordis-catalog + verify-md-wrap + verify-md-links + verify-type-equiv), module-graph freshness (the quality-gates RFC). Don't re-review what a gate already enforces — trust the gate and spend attention on what it can't check. Note that the `doc-sync` gate only covers compilable `ts` blocks, the generated cordis events/services catalog, markdown wrapping/links, and verbatim type-equiv blocks; prose drift (checks #1 and #2) is *additional* manual review on top of it, not covered by it. ## Reviewer-only checks (gates can't catch these — judgment required) @@ -42,7 +43,8 @@ Where your independent reasoning earns its keep. Start here, then keep going acr - **e2e verifies the world, not the agent's self-report.** For real-API tests, confirm the assertion re-runs the command/checks the file externally — a keyword probe lets a cheating agent pass (see AGENTS.md e2e bullet). For a behavior change to the agent's real flows, a no-key/mock test alone is usually insufficient: a with-key e2e (especially a smoke test that boots the real example and checks the world) is cheap here and catches "green units, broken product" — encourage it rather than treating real-API tests as expensive (see AGENTS.md § Secrets / .env). - **Plugin export shape + real-loader coverage.** A new/changed `cordis.yml`-loaded plugin: is it a function/namespace plugin (`name`/`inject`/`Config`/`apply` named exports) with NO `export default`? A stray default export makes the Loader's `unwrapExports` drop `inject` and the plugin crashes at load with `cannot get property … without inject` — invisible to hand-built `ctx.plugin({...})` tests and to line coverage. Confirm there's a test driving it through the REAL loader path (the no-key subprocess e2e for ACP is the model). And any opportunistic read of a service NOT in `static inject` should use `ctx.get(name)`, not `ctx.` (the property proxy throws through a foreign shadow). See [packages/AGENTS.md](../../../packages/AGENTS.md) and [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md). - **Seam discipline.** New swappable capability? Check it's split per the capability-seams RFC (interface / impl / consumer), and that the consumer injects the interface key, never an implementation type. -- **Test quality.** A test that passes but asserts the wrong thing is worse than none. Check that new tests would actually fail if the behavior regressed, and that they exercise the contract (events fired, disposal reached) rather than restating the implementation. +- **Test quality — sufficiency, not just coverage.** 100% per-file coverage and a green suite are necessary, not sufficient: they prove the lines *ran*, not that the feature *works the way it ships*. Judge whether the tests are sufficient on two axes. (1) **Would they fail if the behavior regressed?** A test that passes but asserts the wrong thing — or restates the implementation instead of the contract (events fired, disposal reached, the world changed) — is worse than none. (2) **Do they exercise the REAL thing, the way it's actually used?** Prefer the genuine collaborator over a fake, drive the change through its real entry path (the cordis Loader, the ACP bridge, a booted subprocess — not a hand-built `ctx.plugin({...})` that bypasses `unwrapExports`), and verify the WORLD (re-read the file/log/registry externally), not the agent's self-report. A test that fakes the inputs just enough to cover every line will agree with whatever the author assumed; the real thing won't. When a test sets up a *clean/happy* path to reach a line, ask whether the line's PURPOSE is exercised — e.g. a durability/teardown path "tested" by a fully-completed turn never proves the mid-flight teardown it exists for; a torn-tail recovery branch covered by a well-formed log never proves recovery. Flag tests that hit the line but not the scenario. See AGENTS.md § Defensive patterns "Line coverage is not behavior coverage" and "Prefer the REAL implementation over a mock/stand-in in tests". +- **Snapshot coverage for transcript/UX changes.** If the PR changes the editor-facing transcript or end-to-end agent UX — the ACP bridge's event→update translation, the agent loop's observable output, tool presentation, or anything an editor renders — it must add or update a snapshot scenario (`examples/*/tests/**/*.snapshot.ts`, goldens under `examples/acp-agent/tests/snapshots/`) or note explicitly why none applies (AGENTS.md § Conventions). Review the golden diff itself: a changed `stdout.golden.txt` / `session.golden.txt` is a behavior change in disguise — confirm it's intended, not an accidental regression someone re-recorded away. A pure internal refactor with no observable-output change is exempt, but the PR should say so. See [docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md). - **Intent and contracts.** Does the change do what the PR says, and honor the documented contract on *both* sides of every seam it touches (see AGENTS.md "Honor cross-seam contracts on BOTH sides")? ## How to respond diff --git a/.agents/skills/dsh-find-simplifications/SKILL.md b/.agents/skills/dsh-find-simplifications/SKILL.md new file mode 100644 index 0000000000..2dcea37c66 --- /dev/null +++ b/.agents/skills/dsh-find-simplifications/SKILL.md @@ -0,0 +1,106 @@ +--- +name: dsh-find-simplifications +description: 'Use when working in the deepseek-harness repo to find non-obvious simplification candidates and write proposed RFCs or inline TODO/FIXME/XXX notes for dead, duplicated, speculative, or over-built code surfaces; especially for requests like "find simplification RFCs", "look for unnecessary complexity", "audit for removal-style cleanups", or "fold worthwhile simplification ideas from another PR".' +--- + +# Finding DeepSeek Harness Simplifications + +This skill helps turn a broad "find things to simplify" request into evidence-backed RFCs that remove or collapse existing harness surface area. It is guidance, not a checklist: follow the code, keep judgment active, and prefer a few well-proven candidates over a pile of thin guesses. + +## Start With Repo Context + +- Read `AGENTS.md`, especially the pre-release stance, tests-document-behavior section, conventions, defensive patterns, and Type Safety and Documentation section. +- Skim [docs/architecture.md](../../../docs/architecture.md) before judging anything under `packages/`; simplifications that fight the service map or event taxonomy need extra evidence. +- Use the RFC index ([docs/rfc/README.md](../../../docs/rfc/README.md)) to understand intentional architecture. The most relevant implemented examples are [drop mutable session summary](../../../docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md), [shared persistence write coordinator](../../../docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md), [capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md), and the twin adapter / dual persistence backend RFCs. +- Treat dual LLM adapters and dual persistence backends as intentional by default. Do not propose deleting either twin/backend as "low effort" unless the user explicitly overrides that constraint. Removing an unused method or hook inside a protected seam can still be valid if it does not collapse the protected design. + +## What Counts As A Strong Candidate + +A strong simplification removes, folds, or demotes something real and has clear evidence that the current shape costs more than it buys: + +- A public method, event, config knob, registry notification, helper, package, durable event, or test artifact has no production consumer. +- Tests or docs are the only consumers, and the behavior they pin is not load-bearing. +- Two representations mirror the same fact, especially across durable session events and transient `agent/*` events. +- A seam has methods every implementation must support but no consumer uses. +- A package boundary exists only for test/demo/support code and adds publish or dependency overhead. +- A feature implements speculative product generality: multi-session/session-load, background task rosters, live registry invalidation, mid-turn steering, tool-owned UI rendering, and similar shapes with no product owner. +- An invariant, rollback path, goldens set, or special-case test exists only to protect an unused surface. +- The simplified behavior may differ slightly, but the new behavior is still reasonable and easier to explain. + +Thin candidates are usually not enough for an RFC: deleting one typo, running `knip` once, removing an intentionally documented backend/adapter, or flagging "this looks complex" without call-site proof. + +## Survey Broadly + +Use parallel subagents when the user asks for breadth or many candidates. Give each agent a domain and require evidence, not guesses. Useful domains: + +- Agent loop and session log: turn/step boundaries, steering, abort/cancel, durable events, replay, load/resume. +- ACP and UI surfaces: `session/*` methods, terminal `_meta`, transcript rendering, single vs multi-session state. +- LLM/tools/system prompt: stream/generate surfaces, assemblers, registries, tool schema defaults, presentation hooks. +- Bash and tool execution: foreground/background split, task ownership, output spill files, executor methods. +- Packages/examples/scripts/tests: package boundaries, static inventories, redundant snapshot goldens, support packages. + +If subagents are unavailable, simulate the same breadth yourself. Do not let the first good candidate stop the survey. + +## Prove Or Reject Each Candidate + +For every symbol or behavior, classify consumers before writing: + +- Production corpus: `packages/*/src`, `examples/*/src`, `examples/**/*.yml`, runtime scripts, and loader/config paths. +- Non-production corpus: tests, README/docs, RFCs, snapshots, generated goldens, and comments. +- Ambiguous corpus: examples and scripts that may be product smoke paths. Inspect usage before classifying. + +Use `rg` first. Good searches include the exact symbol, event name, package name, config key, method name with both `.name(` and `name(`, and any wire strings. Then read the call sites. `knip` can help, but it is not a substitute for understanding public interfaces, dynamic event names, tests, docs, and Cordis loader paths. + +Reject or downgrade a candidate when: + +- A production caller exists and the simplification would be a feature decision rather than a cleanup. +- The surface is explicitly justified by an implemented RFC or a hard-won defensive pattern, and the new evidence does not beat that reason. +- The removal would force unrelated churn without actually making the contract smaller. +- The idea is correct but tiny. Add a targeted TODO/FIXME/XXX instead, using the urgency semantics in [docs/development.md](../../../docs/development.md). + +## Write The RFC + +Create one file per durable proposal under `docs/rfc/proposed/yyyy-mm-dd-topic.md` and add it to the Proposed table in `docs/rfc/README.md`. Keep prose paragraphs on one physical line and use relative Markdown links. + +Prefer this shape, adjusting when the idea needs it: + +- `# RFC: ` +- `Status: proposed` +- `## Problem`: name the current surface, cite the relevant files, and state the consumer evidence. Separate production callers from tests/docs. +- `## Proposal`: say exactly what to remove, fold, demote, or rehome. Include tests, docs, READMEs, JSDoc, event-taxonomy, snapshot, and generated-file cleanup when relevant. +- `## Why not keep it?` or `## What we give up`: make the strongest counterargument legible. +- `## Acceptance criteria`: observable end state and gates. +- `## Risks`: public API changes, behavior changes, future product wants, and why the tradeoff is still reasonable. + +Be concrete enough that an implementing PR can follow the trail. Avoid vague "simplify this package" RFCs. When a proposal overlaps an existing RFC, consolidate the useful details into the existing one rather than creating a duplicate. + +## Inline TODO Notes + +Use inline TODO/FIXME/XXX only for small, local cleanups that are clearly useful but not durable design decisions. Keep them short and actionable: + +- Name the smell with a stable tag, e.g. `TODO(double-default)` or `XXX(unused-default)`. +- Explain why it is safe to revisit and what action would simplify it. +- Do not add TODOs for speculative complaints or for behavior that needs an RFC-level decision. + +## When Folding Another PR Or Branch + +Diff the sibling branch against `origin/master`, not against the current PR branch, so you see its independent contribution. For each item: + +- Port non-overlapping RFCs or TODOs that meet the quality bar. +- Consolidate overlapping material into the existing RFC that owns the topic. +- Do not port duplicate or lower-confidence proposals just to preserve the count. +- Update the PR body so reviewers see the true candidate count and scope. +- Close the duplicate PR only when the user asked you to, or when you clearly own that housekeeping. + +## Validation And PR Hygiene + +For docs-only RFC work, run at least `pnpm run doc-sync`, `pnpm run lint`, and `git diff --check`. For code comments or skill changes, also run the relevant validator when one exists. Before pushing, expect the pre-push hook to run module graph freshness, unit tests, snapshots, doc-sync, and hygiene. + +When opening or updating a PR, summarize: + +- How many RFCs and inline notes were added. +- The main areas surveyed. +- What was intentionally excluded. +- Which checks passed. + +Use a draft PR while the survey is still expanding; mark ready only when the candidate set, review responses, and validation are settled. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f07f01b9ec..f164e3c4c4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,22 +33,21 @@ jobs: - name: Constraints run: pnpm run constraints - # Before lint: the type-aware ESLint config resolves vendor packages via - # their built declarations (tsconfig.typecheck.json -> vendor/*/lib), - # which `pnpm run typecheck` emits. Lint on a fresh checkout would otherwise - # see unresolved types and erupt with no-unsafe-* errors. + # Before lint: root typecheck validates the package/vendor reference graph + # and refreshes TSC intermediates so type-aware ESLint sees the same project + # boundaries as the build. - name: Typecheck (src + tests + examples) run: pnpm run typecheck - name: Lint run: pnpm run lint - # Doc-sync gates (doc-sync-enforcement RFC). doc-typecheck compiles the fenced ts blocks in - # the docs and resolves vendor packages via their built declarations, which - # the typecheck step above emits — so it runs after typecheck. The event - # taxonomy check and the markdown wrap check only read source. Same - # `doc-sync` script the pre-push hook runs (quality-gates RFC: one source of truth). - - name: Doc-sync gates (doc code blocks + event taxonomy + markdown wrap) + # Doc-sync gates (doc-sync-enforcement RFC). doc-typecheck compiles the + # fenced ts blocks against the root project-reference graph. The cordis + # catalog freshness check, type-equiv check, and markdown wrap/link checks + # only read source. Same `doc-sync` script the pre-push hook runs + # (quality-gates RFC: one source of truth). + - name: Doc-sync gates (doc code blocks + cordis catalog + type-equiv + markdown wrap/links) run: pnpm run doc-sync # Module-graph freshness: regenerate docs/module-graph.md from the @@ -60,18 +59,28 @@ jobs: - name: Tests with coverage gate (per-file 100%) run: pnpm run test:coverage + # ACP snapshot tests (acp-snapshot-tests RFC): boot the real acp-agent + # subprocess and replay recorded session-log fixtures, diffing the + # normalized stdout transcript + re-persisted log against committed + # goldens. KEYLESS by design — the same `test:snapshot` script the pre-push + # hook runs (one source of truth), so the full-transcript regression net + # is part of every PR gate, not just local pre-push. + - name: Snapshot tests (ACP transcript replay) + run: pnpm run test:snapshot + # Before hygiene: publint validates the packed artifacts (lib/index.js), - # which only the tsdown bundling step emits. + # which only the tsdown bundling step emits, and verify-node-next-types + # validates the built declarations. - name: Build (tsc -b + tsdown bundles) run: pnpm run build - - name: Hygiene (knip + publint) - run: pnpm run knip && pnpm run publint + - name: Hygiene (knip + publint + constraints + NodeNext types) + run: pnpm run hygiene - name: Demo smoke test run: | set -euo pipefail - out=$(printf 'echo ci smoke\n' | timeout 60 node --expose-internals --import tsx examples/echo-agent/start.ts 2>&1) + out=$(printf 'echo ci smoke\n' | timeout 60 pnpm run demo:echo 2>&1) echo "$out" echo "$out" | grep -q '\[tool call\] echo({"text":"ci smoke"})' echo "$out" | grep -q '\[tool result\] ECHO: CI SMOKE' @@ -79,3 +88,12 @@ jobs: # per-run session log named main-session-.jsonl. Assert one exists. ls .sessions/_no-cwd/main-session-*.jsonl >/dev/null rm -rf .sessions + + # The published `bin` is `lib/bin.js`, run under plain `node` by a real + # consumer — NOT the tsx dev path the demo smoke and demo:* scripts use. + # These keyless smokes boot the BUILT bins (this step runs AFTER the build) + # in a temp dir that mirrors a real install, catching a regression in the + # published artifact that tsx would mask. They self-skip if lib/ is absent, + # so the e2e job (which does not build) does not run them. + - name: Built-bin smoke test (published lib/bin.js under node) + run: pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml new file mode 100644 index 0000000000..7a8c9dc88c --- /dev/null +++ b/.github/workflows/e2e.yml @@ -0,0 +1,100 @@ +name: E2E (real DeepSeek API) + +# Real-API end-to-end suite (`pnpm run test:e2e`). Unlike ci.yml this job +# consumes the DEEPSEEK_API_KEY_EXTERNAL secret (mapped to the env var +# DEEPSEEK_API_KEY the tests read) and hits the external API at +# https://api.deepseek.com (DEEPSEEK_BASE_URL is pinned to it explicitly so a +# stray repo-root .env can't redirect the run). +# +# pull_request IS a trigger, but GitHub withholds repo secrets from BOTH forked +# PRs and Dependabot PRs (the latter are same-repo, fork==false, but still +# keyless). The job-level `if:` below skips the whole job for those untrusted +# PRs. The Dependabot test keys on the PR AUTHOR (pull_request.user.login), not +# github.actor (the run trigger) — a maintainer reopening a Dependabot PR would +# make github.actor human while the PR is still keyless. When the job runs, the +# secret is expected, so the preflight hard-fails if it's missing (a missing +# secret would otherwise make the self-skipping suite report a false green). A +# job skipped by a job-level `if:` reports as a SUCCESSFUL check, so this +# workflow is safe to use as a required status check if desired. +# +# SECURITY — NEVER change this trigger to `pull_request_target`. `pull_request` +# checks out and runs the PR's HEAD code WITHOUT secrets for forks, which is what +# keeps an untrusted fork from exfiltrating the key. `pull_request_target` runs +# in the BASE repo's context WITH secrets while still able to check out untrusted +# fork code — a textbook key-leak vector, especially once this repo is public. +# The fork/secret model and its public-repo implications are recorded in +# docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md. +# +# Note: scheduled triggers are auto-disabled after 60 days of repo inactivity; +# push/pull_request/workflow_dispatch act as backstops. +on: + workflow_dispatch: + push: + branches: [main, master] + pull_request: + schedule: + # 00:17 UTC nightly = 08:17 Asia/Shanghai — off the top-of-hour cron stampede. + - cron: '17 0 * * *' + +# Cancel a superseded PR run (it is on a stale commit); never cancel a +# push/schedule run — it is already producing the post-merge/nightly signal. +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +# Least privilege: this job only reads the repo to run tests. +permissions: + contents: read + +jobs: + e2e: + runs-on: ubuntu-latest + # Run on every trusted event. Skip untrusted PRs (forks + Dependabot) where + # the secret is withheld — they would otherwise hard-fail the preflight. + if: >- + github.event_name != 'pull_request' + || !(github.event.pull_request.head.repo.fork || github.event.pull_request.user.login == 'dependabot[bot]') + # Serial files (fileParallelism: false), 120s/test, retry 2. 45m bounds a + # wedged run while leaving headroom for retry storms against a slow API. + timeout-minutes: 45 + steps: + - uses: actions/checkout@v6 + + - uses: actions/setup-node@v6 + with: + node-version: 24 + + - name: Enable corepack (pnpm) + run: corepack enable + + - name: Install (immutable) + run: pnpm install --frozen-lockfile + + # Guard against a false green: the e2e suites self-skip when the key is + # absent, so a missing/misconfigured secret would otherwise pass as + # "all skipped". This job only runs on trusted events (the `if:` above + # excludes keyless PRs), so the secret MUST be present — fail loudly if not. + - name: Preflight (require DEEPSEEK_API_KEY) + env: + # Repo secret DEEPSEEK_API_KEY_EXTERNAL holds the external-API key; + # the tests read process.env.DEEPSEEK_API_KEY, so map it across here. + DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY_EXTERNAL }} + run: | + set -euo pipefail + if [ -z "${DEEPSEEK_API_KEY:-}" ]; then + echo "::error::DEEPSEEK_API_KEY is empty. The e2e suite would self-skip and" + echo "::error::report a false green. Configure the repo secret DEEPSEEK_API_KEY_EXTERNAL." + exit 1 + fi + echo "DEEPSEEK_API_KEY present." + + # Real-API end-to-end tests only. The keyless gates (lint/typecheck/ + # coverage/snapshot/build/etc.) already run in ci.yml on every push/PR; + # no need to repeat them or build first (tests run unbuilt via tsx). + # DEEPSEEK_BASE_URL is pinned to the external API; the secret is scoped to + # this step (and preflight) only — never exposed to checkout/setup/install. + - name: E2E tests (real DeepSeek API) + env: + DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY_EXTERNAL }} + DEEPSEEK_BASE_URL: https://api.deepseek.com + run: pnpm run test:e2e diff --git a/.gitignore b/.gitignore index af71278311..b52f86cd61 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,7 @@ examples/*/*.jsonl examples/*/.sessions/ coverage/ .doc-typecheck-*/ +.humanize/ .vscode/ .DS_Store .idea diff --git a/AGENTS.md b/AGENTS.md index d4b8474865..d7340c631d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,6 +2,38 @@ This is the monorepo for the DeepSeek Harness group. It currently hosts the code for **DeepSeek Code**, DeepSeek's coding agent product. +## Pre-release stance: foundation over blast radius + +**This applies only while the harness is unreleased — remove this section at the first tagged/published release.** There are no external consumers yet, so optimize for the *correct foundation*, not for a small diff. When the right structure means moving a file across package boundaries, renaming a public symbol, or repackaging a plugin, do it — and update every reference in the same change. Do **not** add backward-compat shims, deprecation aliases, re-export stubs, or "keep it where it is to avoid churn" hedges; those are debts you take on to protect callers you do not have. Churn now is cheap; a wrong foundation set in stone is not. (Once released, this inverts — backward compatibility becomes a real constraint and this section comes out.) + +This extends to **on-disk formats, schemas, and stored data**: while unreleased there is no persisted user data to preserve, so a format/schema/contract change needs **no migration path** — a backend REJECTS anything not at the current version rather than upgrading it. How the *version number itself* behaves pre-release is a per-format choice between two equally-valid stances, and the repo uses both deliberately. **Monotonic bump-and-reject**: each breaking change increments the version — e.g. the SQLite backend's `SCHEMA_VERSION` bump that drops columns rejects any non-current `user_version` on open, with no migration; use it when a stored artifact has a small enumerable set of revisions worth telling apart. **A pinned `0` "unstable / pre-release" version**: the format stays at `0` and absorbs ALL pre-release shape churn without bumping, while a backend still rejects any non-`0` log — the session event log uses this (`SESSION_FORMAT_VERSION = 0` in `dsh-session`), because its shape changes often while unreleased and bumping on every tweak would dress up an unstable format as a sequence of stable boundaries that mean nothing yet; pinning `0` and documenting it "no compatibility implied" makes the instability *explicit* instead of pretending each revision is a real version. Either way there is no migration code, and either way a real monotonic policy begins at the first tagged release. A migration written now is a shim for data that does not exist. + +## Tests document behavior, not golden truth + +A passing test pins the behavior the code **currently** has — not necessarily the behavior it **should** have. Existing tests faithfully document existing behavior, but existing behavior is not automatically golden: it can be the residue of a past compromise, a half-built feature, or a limitation that no longer applies. So when a refactor or review makes you ask "can I change this?", a green test is **not** the answer — the question is whether the behavior the test pins is actually correct. + +Before you preserve a behavior solely to keep a test green, ask: is this behavior load-bearing (a real consumer depends on it, a contract promises it, a user observes it), or is it an artifact? If it's an artifact, **change the behavior AND its test together, in the same change, and say why in the PR** — do not contort new code to keep an obsolete assertion passing, and do not treat "but the test expects X" as a reason X must stay. Conversely, do not delete a test just because it is inconvenient: the discipline cuts both ways — you must show the *behavior* is dead, not merely that the test is in your way. + +The worked example is [Drop the mutable session summary](docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md): an entire `SessionSummary` type, a `SessionPersistence.update()` method, a JSONL sidecar, and SQLite columns existed and were exercised by their own contract test — yet **nothing in production CONSUMED any of it, and `update()` had no production caller**. (The backends did *write* summary state — JSONL touched the sidecar after a durable append, SQLite bumped `updated_at` in the append transaction — but those writes fed only reads that nothing performed.) The tests documented the behavior perfectly; the behavior was dead. Deleting the behavior and its tests together removed ~400 lines and erased a durability divergence the next refactor would have had to model. (This is the test-tier echo of "verify the world, not a synthetic stand-in" in § Defensive patterns: a test agrees with whatever it was written to assert; only a real consumer proves the behavior matters.) + +## RFCs are proposals, not golden truth + +The same discipline applies one level up, to the RFCs in `docs/rfc/`. A **proposed** RFC records an *intended* change argued at a point in time; it is not a contract to implement verbatim. The author reasoned from the code as they understood it then — and they can be wrong, or the code can have moved. So before implementing an RFC, **validate its premise against the current code first**: confirm the thing it wants removed or changed is actually dead/safe, and that the migration it proposes is genuinely cleaner than what exists. + +When carrying out the change fights back — a removal forces an awkward migration, deletes machinery that turns out to be load-bearing, or pushes consumers onto a more brittle hand-rolled equivalent — treat that friction as **evidence the RFC over-reached**, not as work to push through. Keep, split, or amend the change to match what the code actually wants, and say so in the PR. An RFC that ships in amended form gets its text amended on the way to `implemented/`, so the landed RFC describes what actually shipped rather than the original guess. The discipline cuts both ways: an RFC is also not a reason to *avoid* a change a maintainer would otherwise make — it is one input, weighed against the code in front of you. + +The worked example is [Keep one public stop primitive](docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md): it proposed removing BOTH `Agent.abort()` and `Agent.whenIdle()` as redundant stop/quiescence surface. Validating against the code, `abort()` was genuinely dead — no production caller, the loop aborts its own `AbortController` directly — so it was removed as proposed. But `whenIdle()` was load-bearing: a deliberate quiescence primitive with live ACP consumers, and the RFC's suggested migration (observe the `running`→`idle` transition by hand) is exactly the brittle path § Defensive patterns warns against ("Async state is not synchronous state"). So only `abort()` shipped, `whenIdle()` stayed, and the RFC's text was amended on the way to `implemented/` to record the narrowed scope — the landed RFC is not a lie about what was built. + +## Orchestrating review feedback across a stacked PR chain + +A wave of review comments lands across several PRs in a dependent stack (`A ← B ← C …`) at once. Resolving it well is a discipline of its own, learned the hard way: + +- **One worktree per PR branch; never rewrite a pushed branch.** Each PR's fixes happen in that PR's own worktree. To bring a child up to date with a parent's new commits, **merge the parent down** — never rebase/amend/force-push a branch that is already pushed (see [§ Conventions](#conventions) "Never rewrite a pushed branch"). The stacked-merge graph and the per-round review-fix history depend on it. +- **A fix belongs on the PR that INTRODUCED the issue, then flows DOWN.** When a comment on PR `B` points at code `B` introduced, fix it on `B` and merge `B` into `C` — even if `C` already carries the same file through the chain. Originating the fix on the downstream `C` leaves `B` shipping the unfixed code and the fix invisible to a reviewer of `B`. (This bit us: a snapshot-test guard flagged on the lower PR got fixed only on the top PR, so the lower PR still read as unaddressed until the fix was relocated to its true origin and merged down.) +- **Each review fix is a SEPARATE commit, never an amend.** The "fix review findings" commit is part of the record — it shows what the review caught and how. Amending erases that. (Amend is fine only for your own not-yet-pushed work.) +- **Delegated work is trust-but-verify.** When sub-agents implement fixes in parallel, their report describes what they INTENDED, not necessarily what landed. Re-run the gates yourself on the actual tree, and for a regression guard, **prove it FAILS on the unfixed code** (introduce the regression, watch the test go red, revert) — a guard that passes both ways guards nothing. A sub-agent that "reframes the problem as already-handled" instead of fixing it is a signal to dig in personally, not to accept the reframing. +- **Triage on the merits, then reply in-thread.** Verify each comment against the code before acting (a reviewer flagging the right symptom can still mis-diagnose the cause — confirm both). Reply in the GitHub review thread (`gh api …/pulls/{pr}/comments/{id}/replies`), not as a top-level comment, stating the fix and the commit that carries it. + ## Architecture This codebase is based on the **Cordis** framework, built microkernel-style: **everything is a plugin**. All necessary Cordis dependencies are copied into this monorepo as vendored source (under `vendor/`) instead of being depended on via npm. @@ -20,33 +52,66 @@ vendor/ Vendored Cordis framework source (original npm names, private). See vendor/README.md for the manifest, local-modification log, and the upstream sync procedure. Do NOT edit casually — every divergence must be logged there. -packages/ Harness packages, all named @deepseek-ai/dsh-: - llm/ abstract LLM service + content-block vocabulary - llm-deepseek/ DeepSeek API adapter (hand-rolled fetch/SSE) - llm-pi-ai/ DeepSeek adapter via @earendil-works/pi-ai (design twin) - session/ event-sourced session log + in-memory store - system-prompt/ prompt-section + tool-schema assembly registry - tools/ tool registry + tools/execute waterfall - agent/ Agent interface, registry, agent/* event vocabulary - agent-loop/ THE concrete plugin: LoopAgent + the loop driver - invariants/ dev-mode event-contract invariants + session-log freeze - bash/ abstract bash executor seam (ctx.bash) — interface only - bash-local/ local-subprocess BashExecutor implementation - tool-bash/ model-facing bash/bash_output/bash_kill tool schemas - acp/ Agent Client Protocol bridge: drive the agent from an ACP - editor (Zed) over JSON-RPC stdio -examples/ Runnable demos (not workspaces). echo-agent = mock model + echo - tool + stdio UI + JSONL persistence, wired via cordis.yml. - coding-agent = the real thing: DeepSeek V4 + bash tools - (pnpm run demo:coding, needs DEEPSEEK_API_KEY). - acp-agent = the coding agent exposed as an ACP server over - JSON-RPC stdio (pnpm run demo:acp, needs DEEPSEEK_API_KEY). - base.yml = shared provider/tool core both real demos include. +packages/ Harness packages, grouped by role at packages///. + Every package is named @deepseek-ai/dsh-; the group dir is a + pure container (no package.json). See packages/README.md and each + group's README.md for the product-vs-support split. + core/ product API spine + session/ event-sourced session log + in-memory store + system-prompt/ prompt-section + tool-schema assembly registry + tools/ tool registry + tools/execute waterfall + agent/ Agent interface, registry, agent/* event vocabulary + agent-loop/ THE concrete plugin: ReactLoopAgent + the loop driver + agent-core/ bundle plugin: the providerless/executor-less/UI-less spine + (timer+llm+sessions+system-prompt+tools+agents+invariants+ + tool-bash+agent-loop) as code; forwards agent-loop's `agents` + llm/ LLM capability family + llm/ abstract LLM service + content-block vocabulary + llm-deepseek/ DeepSeek API adapter (hand-rolled fetch/SSE) + llm-pi-ai/ DeepSeek adapter via @earendil-works/pi-ai (design twin) + bash/ bash capability family + bash/ abstract bash executor seam (ctx.bash) — interface only + bash-local/ local-subprocess BashExecutor implementation + tool-bash/ model-facing bash/bash_output/bash_kill tool schemas + session-persistence/ persistence capability family + session-persistence/ durable persistence seam + write coordinator + session-persistence-jsonl/ JSONL-sidecar backend + session-persistence-sqlite/ SQLite backend + ui/ product integration surfaces + acp/ Agent Client Protocol bridge: drive the agent from an ACP + editor (Zed) over JSON-RPC stdio + stdio-agent/ stdio chat APP: agent-core spine + console logger + readline + UI + a pre-created main agent + a bin (the demo:echo/coding + front door) + acp-agent/ ACP server APP: agent-core spine + JSONL persistence + the + acp bridge, NO stdout logger + a bin (the demo:acp front door) + support/ dev/test/example infrastructure (lower compat expectations) + invariants/ dev-mode event-contract invariants + session-log freeze + ui-stdio/ minimal stdio (readline) UI plugin: renders agent/* events, + feeds stdin lines to the agent (shared by the demos) + llm-replay/ record/replay adapter: short-circuits llm/stream from a + recorded session JSONL (keyless snapshot tests) + util/ low-level zero-dependency utilities shared across groups + brand/ type-only Branded nominal-typing primitive (no runtime + code, no harness deps; owns the brand for cross-boundary ids) +examples/ Runnable demos (not workspaces; see examples/AGENTS.md). Each is a + THIN leaf cordis.yml: it picks the swappable backends (an LLM adapter, + a bash executor) and loads ONE app package (dsh-stdio-agent or + dsh-acp-agent), which bundles the agent-core spine + front-door + cluster + boot glue (a bin). No start.ts. echo-agent = mock model + + echo tool on dsh-stdio-agent (pnpm run demo:echo, no key). + coding-agent = the real thing: DeepSeek V4 + bash tools on the same + app (pnpm run demo:coding, needs DEEPSEEK_API_KEY). acp-agent = the + coding agent as an ACP server on dsh-acp-agent (pnpm run demo:acp, + needs DEEPSEEK_API_KEY). cordis.snapshot.yml = the acp leaf with + llm-replay for keyless snapshot replay. docs/ architecture.md — the design doc. module-graph.md — generated inter-package dependency graph (Mermaid; `pnpm run gen-module-graph`). rfc/ — design decisions and proposals, one kind of doc grouped by - lifecycle into proposed/ implemented/ rejected/ (the why behind - vendoring, event-sourcing, the schema DSL, …). See rfc/README.md. + lifecycle (proposed/ implemented/ rejected/) then by class + (feature/ bug-fix/ simplification/ architecture/ process/ testing/); + the why behind vendoring, event-sourcing, the schema DSL, …. See + rfc/README.md. postmortem/ — incident write-ups: a bug that escaped to a user/merge/release, why the safety nets missed it, the guardrails added. cookbook/ — step-by-step guides: adding a package, a tool, @@ -60,25 +125,44 @@ scripts/ repo maintenance scripts (vendor-manifest guard, publint runner). ```sh pnpm install # pnpm workspaces, node >= 24 -pnpm run test # vitest run (packages/*/tests/**/*.spec.ts) -pnpm run test:coverage # vitest run --coverage (per-file 100% gate on packages/*/src) +pnpm run test # vitest run (packages|examples/*/tests/**/*.spec.ts) +pnpm run test:coverage # vitest run --coverage (per-file 100% gate on packages/*/*/src) pnpm run test:e2e # real-API tests (packages|examples/*/tests/**/*.e2e.ts); # self-skips without DEEPSEEK_API_KEY — see Secrets below -pnpm run typecheck # tsc -b tsconfig.build.json (declarations) + tsc -p - # tsconfig.typecheck.json (tests/examples typecheck too) +pnpm run test:snapshot # ACP snapshot tests (examples/*/tests/**/*.snapshot.ts): + # boot the real acp-agent subprocess, replay a recorded + # session JSONL, diff the normalized stdout + re-persisted + # log against committed goldens. KEYLESS — runs in the + # default gate. Filter one by scenario name (no `--`, which + # vitest treats as a positional file filter): `pnpm run + # test:snapshot -t `. +pnpm run test:snapshot:record # re-record fixtures + goldens against the real + # API (needs DEEPSEEK_API_KEY); accept-the-diff = re-record + # (or `pnpm run test:snapshot -u` to refresh goldens only) +pnpm run typecheck # tsc -b tsconfig.json pnpm run lint # eslint . pnpm run lint:fix # eslint . --fix -pnpm run build # tsc -b tsconfig.build.json && tsdown (JS bundles into lib/) +pnpm run build # tsc emits lib/types, then tsdown bundles runtime lib/index.* pnpm run knip # dead-code / unused-dependency check -pnpm run publint # package.json publish-correctness check (publishable packages/*) -pnpm run hygiene # knip + publint + workspace constraints +pnpm run publint # package.json publish-correctness check (every packages/*/* package) +pnpm run hygiene # knip + publint + workspace constraints + NodeNext type-consumer check pnpm run doc-typecheck # typecheck every ```ts block in README.md, docs/**/*.md, - # packages/*/README.md (doc/code drift gate) -pnpm run verify-event-taxonomy # assert the event-taxonomy table in docs/architecture.md - # matches the interface Events declarations in source + # packages/*/*.md + packages/*/*/*.md (doc/code drift gate) +pnpm run gen-cordis-catalog # regenerate docs/cordis-catalog/events-and-services.md + # (events + services) from the interface Events / Context source +pnpm run verify-cordis-catalog # assert that generated catalog is not stale pnpm run verify-md-wrap # assert no hard-wrapped prose paragraphs in README.md, - # docs/**/*.md, packages/*/README.md, AGENTS.md (one line per paragraph) -pnpm run doc-sync # doc-typecheck + verify-event-taxonomy + verify-md-wrap (CI runs this) + # docs/**/*.md, packages/*/*.md, AGENTS.md (one line per paragraph) +pnpm run verify-doc-refs # assert every docs/*.md path cited in a packages|examples + # TypeScript comment resolves (catches a moved/renamed doc) +pnpm run verify-package-paths # assert every packages/ cited in Markdown or a + # TypeScript comment resolves when it names a real (moved) package +pnpm run verify-rfc-classification # assert every RFC lives in a valid + # {lifecycle}/{class}/ folder and docs/rfc/README.md lists it + # under the matching heading (closed class set + index completeness) +pnpm run verify-node-next-types # assert built declarations typecheck for a + # standard external NodeNext ESM TypeScript consumer +pnpm run doc-sync # doc-typecheck + verify-cordis-catalog + verify-md-wrap + verify-md-links + verify-doc-refs + verify-package-paths + verify-rfc-classification + verify-type-equiv (CI runs this) pnpm run demo:echo # run examples/echo-agent (no API key; type "echo hi" to # see a tool call) — the mock skeleton pnpm run demo:coding # run examples/coding-agent — the real agent (needs @@ -101,12 +185,12 @@ cordis.yml configs reference env vars with the `!!js` tag: `apiKey: !!js process **Lean on with-key e2e tests — we are DeepSeek and model inference is cheap.** A no-key test (mock adapter, or an operation that never reaches the model) is great for determinism and CI, but it can only prove the plumbing, not that the agent actually *works* against a real model. Do not ration real-API tests to save tokens: write many of them, cover the real flows (a real prompt that writes a file, a multi-turn conversation, tool use, cancellation mid-stream), and run them frequently while developing — locally and whenever you have a key in the environment. **Especially smoke tests**: a cheap with-key smoke test that boots the real example, sends one real prompt, and checks the world (a file on disk, a non-empty assistant turn) catches whole classes of "green unit tests, broken product" failures that mocks structurally cannot — the very gap that let the ACP inject bug ship (see [docs/postmortem/0001](docs/postmortem/0001-acp-default-export-drops-inject.md)). The self-skip rule is ONLY so CI (which has no secrets) stays green and so a contributor without a key isn't blocked — it is not a signal that real-API tests are expensive or second-class. When in doubt, add the with-key test AND run it. -Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root `tsconfig.json` (`vitest` resolves through `tsconfig.test.json`). Building is only needed for publishing/consumption outside the repo — with one exception: `pnpm run lint`'s type-aware rules resolve vendor packages through their built declarations (`tsconfig.typecheck.json` → `vendor/*/lib`), so run `pnpm run typecheck` once after a fresh clone (CI does the same) or lint reports unresolved-type `no-unsafe-*` errors. +Dev/test/demo run **unbuilt** via tsx + the source `paths` map in the root `tsconfig.json` (`vitest` resolves through that same root config). Building is only needed for publishing/consumption outside the repo. Non-published code (`examples`, tests, and scripts) is checked by root `tsconfig.json`, which sets `noEmit` and references the package/vendor graph so those sources stay checked under their own tsconfig boundaries. ## Conventions - **Package naming**: every npm package in this repo is `@deepseek-ai/dsh-` (vendored packages keep their upstream names and are `private: true`). -- **ESM everywhere** (`"type": "module"`); imports between workspace packages use package names, never relative paths across package boundaries. In-package imports use explicit `.ts` extensions (allowImportingTsExtensions). +- **ESM everywhere** (`"type": "module"`); imports between workspace packages use package names, never relative paths across package boundaries. In-package relative imports use explicit `.ts` extensions; `rewriteRelativeImportExtensions` turns those into `.js` in emitted JS, while declarations keep explicit `.ts` specifiers that NodeNext/Node16 TypeScript consumers can resolve to sibling `.d.ts` files. `lib/types/**/*.js` is a bundler-only intermediate, not a Node ESM entrypoint. - **`cordis` is a peerDependency** (+ devDependency) of every harness package, mirroring upstream convention. - **Registrations are effects**: anything a plugin contributes (adapter, tool, section, agent, event listener) goes through `ctx.effect()` / `ctx.on()` so disposal and HMR work. If you write a registry, `register()` must return the disposer. - **Typed events via declaration merging**: services declare their events in `declare module 'cordis' { interface Events { … } }`, and their ctx key in `interface Context`. Extensible unions use the merge-extensible-map pattern (see `ContentBlockMap`, `MessageSourceMap`). @@ -119,9 +203,11 @@ Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root `tsconfig.js - **An empty `catch` must name what it swallows and why nothing else can hit it**: a bare `catch {}` hides bugs. When you deliberately ignore a throw, the comment must (a) name the single expected failure, (b) say why ignoring it is correct — usually because the useful state was already captured *before* the `try` — and (c) make clear nothing else of consequence can reach the catch (ideally the `try` wraps a single statement). Example: the error-body `response.json()` parse in `dsh-llm-deepseek`'s adapter sets `code` + HTTP `status` from the status line before the `try`, so a malformed provider body can only cost a richer message, never the real error. - **Symmetry is usually more correct**: when two related values play parallel roles (a test fixture and its expected output, a request shape and its response shape, a buggy input and the test that checks the fix), give them parallel form — both named consts, or both inline, not one each way. Asymmetry is a smell that usually points at a missed extraction. - **Merging PRs**: always merge with a **merge commit** (`gh pr merge --merge`), never squash or rebase. The per-PR commit history is intentional — review-fix commits, regression-test commits, and the reasoning in each message are part of the record — and squashing flattens it away. +- **Never rewrite a pushed branch in a stacked chain.** Once a branch is pushed (and especially once it has a PR), do NOT `rebase`, `amend`, or force-push it. Update a child branch by **merging its parent down** (`git merge ` into the child, as a new merge commit), never by rebasing the child onto the parent's new tip. Rewriting a shared branch diverges it from what the parent and GitHub recorded, which breaks the stacked-merge graph and erases the review-fix history that documents what each round caught. Amending is fine ONLY for your own not-yet-pushed, not-yet-reviewed work. A corollary on WHERE a fix lands: a review fix belongs on the PR that **introduced** the issue, even when a downstream PR in the stack also carries the affected file — fix it on the originating branch, then merge that branch DOWN the chain, rather than originating the fix on the downstream PR (where it would be invisible to a reviewer of the PR that actually owns the code). - **TODO markers**: use `FIXME`/`TODO`/`XXX` to flag known issues by urgency — see [docs/development.md](docs/development.md) for the semantics of each. -- **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test (dispose the contributing fiber, assert cleanup). **Excessive tests are welcome** — when in doubt, write the test; err on the side of covering edge cases, error paths, event ordering, and concurrency races even if they seem unlikely. Review findings get regression tests (see `packages/agent-loop/tests/review-fixes.spec.ts`). The same generosity applies to **real-API (with-key) e2e tests — inference is cheap here (we are DeepSeek), so do not ration them**: cover the agent's real flows (a real prompt that writes a file, multi-turn, tool use, cancellation) and run them frequently while developing, especially cheap **smoke tests** that boot the real example and check the world. A green mock/no-key suite proves the plumbing, not the product — the with-key smoke test is what catches "green units, broken product". See § Secrets / .env for the with-key policy and why self-skip is a CI accommodation, not a verdict that real-API tests are expensive. +- **Tests**: vitest, colocated under `packages///tests/*.spec.ts`. Every registry needs an HMR-safety test (dispose the contributing fiber, assert cleanup). **Excessive tests are welcome** — when in doubt, write the test; err on the side of covering edge cases, error paths, event ordering, and concurrency races even if they seem unlikely. Review findings get regression tests (see `packages/core/agent-loop/tests/review-fixes.spec.ts`). The same generosity applies to **real-API (with-key) e2e tests — inference is cheap here (we are DeepSeek), so do not ration them**: cover the agent's real flows (a real prompt that writes a file, multi-turn, tool use, cancellation) and run them frequently while developing, especially cheap **smoke tests** that boot the real example and check the world. A green mock/no-key suite proves the plumbing, not the product — the with-key smoke test is what catches "green units, broken product". See § Secrets / .env for the with-key policy and why self-skip is a CI accommodation, not a verdict that real-API tests are expensive. - **Prefer the REAL implementation over a mock/stand-in in tests.** When the genuine collaborator is available in the repo, wire it up instead of hand-rolling a fake — a test that registers an inline `defineTool({ name: 'bash', … })` to stand in for `dsh-tool-bash` proves the *bridge* moves bytes but not that the *shipping tool* renders the way the test asserts; the two drift and the test passes while the product is wrong. Mock only the genuinely expensive/non-deterministic boundary (the LLM adapter, the network, the clock) and keep everything downstream real: a bridge tool-call test runs the scripted mock MODEL but the REAL tool + REAL executor (e.g. `makeBridgeHarness({ withBash: true })` plugs `dsh-bash-local` + `dsh-tool-bash` and runs an actual `echo`), so it verifies the actual `presentCall`/`presentResult` an editor sees. This is the unit-test echo of "verify the world, not a synthetic stand-in" (see § Defensive patterns) — a fake you wrote will agree with whatever you assumed; the real thing won't. +- **A change that affects the editor-facing transcript or end-to-end agent UX needs a snapshot test (or an explicit note in the PR why none applies).** The snapshot tier (`examples/*/tests/**/*.snapshot.ts`, `pnpm run test:snapshot`) boots the real example subprocess, replays a recorded session JSONL deterministically (keyless), and diffs the normalized stdout transcript + re-persisted session log against committed goldens — the full-transcript regression net that mock-level unit tests structurally cannot be (it is what catches a bridge-translation or loop-structure regression that leaves every unit green). When you change the ACP bridge, the agent loop's observable output, tool presentation, or anything an editor renders, add or update a scenario under `examples/acp-agent/tests/snapshots/` and re-record with `pnpm run test:snapshot:record`. Reviewing the golden diff is part of the review. The rule is scoped to transcript/UX-affecting changes — a pure internal refactor with no observable-output change does not need one, but say so. See [docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md](docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md). ## Defensive patterns (hard-won) @@ -134,20 +220,28 @@ Each bullet is a bug class that bit us; the rule prevents the reoccurrence. - **Contain callback exceptions at the boundary.** A user-supplied listener (`onTaskDone`, event handlers) that throws must not reject the promise it runs inside or starve the listeners after it. Wrap the dispatch loop in try/catch and log; never let one bad subscriber break core lifecycle. - **Never hand untrusted/model output the ambient environment or predictable paths.** Spawned commands get a scrubbed env (drop `*KEY*`/`*SECRET*`/ `*TOKEN*`) so the harness's own credentials can't leak into output, `env`, or spill files. Temp/spill files use a private (0700) dir, random names, and exclusive owner-only (`'wx'`, `0o600`) opens — predictable world-readable paths invite symlink races and disclosure. - **e2e tests own their resources.** Real-API/integration tests must create the harness in the test and dispose it in `afterEach` (even on failure/retry/timeout), so a flaky run doesn't leak processes or contexts. Shared fixtures live in a plain `tests/harness.ts` module, NOT another `*.e2e.ts` file — importing a spec file re-registers its `describe` and duplicates real API calls. Verify the WORLD, not the agent's self-report: re-run the command/check externally and assert files are byte-identical where they should be unchanged (a keyword probe lets a cheating agent pass). -- **Line coverage is not behavior coverage; test the REAL entry path, not a synthetic stand-in.** 100% per-file coverage and a green suite are necessary, not sufficient — they prove lines ran, not that the feature works the way it ships. A plugin shipped via `cordis.yml` is loaded by the cordis Loader, which calls `Loader.unwrapExports` (`exports.default ?? exports`) and then constructs a fiber from the module's `inject`/`name`/`Config` namespace exports. A test that mounts the plugin by hand-building `ctx.plugin({ name, inject, apply })` (or even `ctx.plugin(NamespaceImport)`) BYPASSES `unwrapExports` entirely, so it cannot catch a broken export shape. This bit us hard: a stray `export default apply` made `unwrapExports` collapse the module to the bare function, dropping `inject` — so every service read threw `cannot get property … without inject` the instant a real editor connected, while 178 hand-mounted tests stayed green. The guard is at least one test that drives the plugin through its REAL load path (a subprocess booting the example via the Loader, or the Loader API directly), exercising the headline operations end-to-end. It runs WITHOUT a key when the operation doesn't call the model (`session/new`/`session/load` reach the factory but never the LLM), so there is no excuse to skip it. Corollary: when an `*.e2e.ts` spawns the example from a temp cwd, set `TSX_TSCONFIG_PATH` to the repo-root tsconfig — the unbuilt `paths` map is found by searching UP from cwd, so a temp cwd outside the repo silently falls back to built `lib/`, which both hides source changes and only "works" when a stale build happens to exist. +- **Line coverage is not behavior coverage; test the REAL entry path, not a synthetic stand-in.** 100% per-file coverage and a green suite are necessary, not sufficient — they prove lines ran, not that the feature works the way it ships. A plugin shipped via `cordis.yml` is loaded by the cordis Loader, which calls `Loader.unwrapExports` (`exports.default ?? exports`) and then constructs a fiber from the module's `inject`/`name`/`Config` namespace exports. A test that mounts the plugin by hand-building `ctx.plugin({ name, inject, apply })` (or even `ctx.plugin(NamespaceImport)`) BYPASSES `unwrapExports` entirely, so it cannot catch a broken export shape. This bit us hard: a stray `export default apply` made `unwrapExports` collapse the module to the bare function, dropping `inject` — so every service read threw `cannot get property … without inject` the instant a real editor connected, while 178 hand-mounted tests stayed green. The guard is at least one test that drives the plugin through its REAL load path (a subprocess booting the example via the Loader, or the Loader API directly), exercising the headline operations end-to-end. It runs WITHOUT a key when the operation doesn't call the model (`session/new`/`session/load` reach the factory but never the LLM), so there is no excuse to skip it. Corollary: when an `*.e2e.ts` spawns the example from a temp cwd, set `TSX_TSCONFIG_PATH` to the repo-root tsconfig — the unbuilt `paths` map is found by searching UP from cwd, so a temp cwd outside the repo silently falls back to built `lib/`, which both hides source changes and only "works" when a stale build happens to exist. Two sharper corollaries this bit us with again: + - **A real-load-path test only GUARDS the export shape if a broken shape actually FAILS it.** The original crash (`cannot get property … without inject`) fired because that plugin HAS `inject`. A plugin with NO `inject` (a composition/bundle plugin that mounts children carrying their own inject, e.g. `dsh-agent-core` and the app packages) does NOT crash on a stray `export default` — `unwrapExports` silently drops `Config`/`name` and the plugin boots anyway — so a Loader smoke stays green while the export shape is broken. For such plugins add an EXPLICIT assertion that the regression fails: `expect('default' in mod).toBe(false)` plus running the module through the real `Loader.prototype.unwrapExports` and asserting `name`/`Config`/`apply` survive. Prove it: add `export default apply`, watch the test go red, revert. + - **"Real entry path" means the PUBLISHED ARTIFACT, not the dev runtime.** A test (or a `demo:*` smoke) that boots `src/bin.ts` under `tsx` is NOT the same code a consumer runs — the package `bin` field points at the built `lib/bin.js` under plain `node`. tsx masks failure modes the published artifact has: a boot settle-race that exits 0 before the app's handles attach, module-resolution differences (the unbuilt `paths` map vs node_modules), and a load failure that `loader.await()`'s `Promise.allSettled` SWALLOWS so a typo'd config silently exits 0. The guard is a smoke that runs the built `lib/bin.js` under plain `node` in a node_modules-shaped temp dir (symlinked workspace + vendor packages), asserts the real output, AND asserts a genuinely-missing config exits NON-ZERO. The tsx demo is necessary but not sufficient; the published-bin smoke is what catches "green under tsx, broken on install". - **Tag spelling and EOF hygiene.** cordis.yml interpolates env via the `!!js` tag (js-yaml resolves custom tags under `tag:yaml.org,2002:js`), not `!js` — keep code, comments, and docs consistent. Files end with exactly one trailing newline; `git diff --check` (a pre-push gate) rejects new blank lines at EOF. ## Type Safety and Documentation -This codebase aims to be **very type-safe and well documented** for maintainability. Code that fails to compile under `strict: true` (with `noImplicitAny` enabled for all `packages/*` source) is not acceptable. Every `any` that remains must have a specific justification (a comment explaining why a narrower type is infeasible). +This codebase aims to be **very type-safe and well documented** for maintainability. Code that fails to compile under `strict: true` (with `noImplicitAny` enabled for all `packages/*/*` source) is not acceptable. Every `any` that remains must have a specific justification (a comment explaining why a narrower type is infeasible). -In the **core** packages (`packages/llm`, `packages/tools`, `packages/agent`, `packages/agent-loop`, `packages/session`, `packages/system-prompt`), **type gymnastics are acceptable when they improve the DX of plugin authors** for common plugin types. The `defineTool` typed schema DSL in `dsh-tools` is the canonical example: the `SchemaSpec` to `InferArgs` type-level mapping gives tool authors zero-cast typed `execute` args, and the cost of the conditional types stays inside the core package. +**Almost always lean toward the stricter lint rule.** In the agentic-coding era the cost/benefit of strictness has inverted: a machine writes and reads most of the code, so the one-time cost of satisfying a stricter rule is cheap and paid by a tool, while the benefit — a whole class of error caught mechanically, a consistent foundation every agent can rely on, less reviewer attention spent on what a linter could have caught — compounds across every future change. When choosing whether to enable a rule, tighten an existing one, or add a new gate (a `verify-*` script, a constraint check), default to YES unless it has a concrete, recurring false-positive problem. Prefer a narrowly-scoped escape hatch (a justified inline disable with a reason, a per-path override) over leaving the rule off globally. The same reasoning motivates this repo's many bespoke gates (`doc-sync`, `verify-package-paths`, the workspace-shape constraint): encode the invariant in a check so no human or agent has to remember it. -Verbose documentation is fine **as long as docs and code stay strictly in sync**. Out-of-sync docs are worse than no docs. **When you change code, update its docs in the SAME change** — grep the package README and the module/JSDoc comments for the old behavior (config keys, defaults, error codes, wire field names, event names) and fix every hit. CI runs `pnpm run doc-sync` (`doc-typecheck` + `verify-event-taxonomy` + `verify-md-wrap` + `verify-md-links`), which typechecks every fenced `ts` block in `README.md`, `docs/**/*.md`, and `packages/*/README.md`, verifies the event-taxonomy table against source, asserts no hard-wrapped prose paragraphs, and checks that every relative Markdown cross-link resolves — across those files plus `AGENTS.md` / `packages/AGENTS.md` — but that scope does NOT catch prose drift in `AGENTS.md` / `packages/AGENTS.md` / `packages/README.md` (config keys, defaults, error codes), so keeping those in sync remains on the author. Every module has a module-level doc comment explaining its role. Every exported class, interface, type, function, and non-obvious method has a JSDoc that explains semantics (not just the name) — contracts (what events fire when), disposal behavior, error behavior, and extension intent. Internal helpers get docs only where non-obvious. Prefer one-liners when one line suffices. +In the **core** packages (`packages/llm/llm`, `packages/core/tools`, `packages/core/agent`, `packages/core/agent-loop`, `packages/core/session`, `packages/core/system-prompt`), **type gymnastics are acceptable when they improve the DX of plugin authors** for common plugin types. The `defineTool` typed schema DSL in `dsh-tools` is the canonical example: the `SchemaSpec` to `InferArgs` type-level mapping gives tool authors zero-cast typed `execute` args, and the cost of the conditional types stays inside the core package. -**Write an RFC when — and only when — a PR makes a decision that is durable, contested, and surprising.** RFCs (`docs/rfc/`, grouped into `proposed/` / `implemented/` / `rejected/`) record the *why* behind choices a future reader would otherwise re-litigate (the vendoring policy, event-sourcing, the schema DSL are the existing examples). A PR that introduces such a decision — a new third-party runtime dependency over the vendoring default, a cross-package contract, a security/isolation model, a deviation from a documented architecture rule — writes the RFC in `implemented/` **in the same PR**, and links it from the relevant code. A proposal for future work not yet built goes in `proposed/`. A PR whose changes are mechanical, self-evident, or already covered by an existing RFC needs none — do not manufacture an RFC for a routine change. When unsure, the test is: would a competent maintainer six months from now ask "why was it done this way?" and be unable to answer from the code alone? If yes, write it. See [docs/rfc/README.md](docs/rfc/README.md) for the naming scheme and [docs/AGENTS.md](docs/AGENTS.md) for the cross-link convention. +Verbose documentation is fine **as long as docs and code stay strictly in sync**. Out-of-sync docs are worse than no docs. **When you change code, update its docs in the SAME change** — grep the package README and the module/JSDoc comments for the old behavior (config keys, defaults, error codes, wire field names, event names) and fix every hit. CI runs `pnpm run doc-sync` (`doc-typecheck` + `verify-cordis-catalog` + `verify-md-wrap` + `verify-md-links` + `verify-doc-refs` + `verify-package-paths` + `verify-rfc-classification` + `verify-type-equiv`), which typechecks every fenced `ts` block in `README.md`, `docs/**/*.md`, and `packages/*/*.md`, regenerates the cordis events/services catalog from source and fails if the committed copy is stale, asserts no hard-wrapped prose paragraphs, checks that every relative Markdown cross-link resolves, checks that every `docs/*.md` path cited in a source comment resolves, checks that every `packages/` reference naming a real package resolves, checks that every RFC is filed under a valid class folder and listed in its index, and checks that every ` ```ts type-equiv ` doc block still matches its source type — across those files plus `AGENTS.md` / `packages/AGENTS.md` — but that scope does NOT catch prose drift in `AGENTS.md` / `packages/AGENTS.md` / `packages/README.md` (config keys, defaults, error codes), so keeping those in sync remains on the author. Every module has a module-level doc comment explaining its role. Every exported class, interface, type, function, and non-obvious method has a JSDoc that explains semantics (not just the name) — contracts (what events fire when), disposal behavior, error behavior, and extension intent. Internal helpers get docs only where non-obvious. Prefer one-liners when one line suffices. -**Markdown is not hard-wrapped**: write one line per paragraph and let the editor soft-wrap. Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. This applies to prose only: leave fenced code blocks, tables, and list structure intact (a wrapped list item folds to one line per bullet). Code comments / JSDoc are exempt — they stay under the linter's column limit. `pnpm run verify-md-wrap` (part of `doc-sync`) enforces this across `README.md`, `docs/**/*.md`, `packages/*/README.md`, and `AGENTS.md` / `packages/AGENTS.md`; `pnpm run verify-md-links` (also part of `doc-sync`) checks that every relative cross-link in those files resolves. +**Tag every new event with `@mode`.** The cordis events/services catalog ([docs/cordis-catalog/events-and-services.md](docs/cordis-catalog/events-and-services.md)) is GENERATED from source by `scripts/gen-cordis-catalog.ts` — never hand-edit it; run `pnpm run gen-cordis-catalog` and commit the result. When you add an event to an `interface Events` block, its JSDoc MUST carry a `@mode emit|waterfall|parallel` tag (the generator hard-errors without it): use `waterfall` when the signature ends with a `next: () => …` parameter (the listener transforms or vetoes via `next()`), `parallel` when the loop awaits a fan-out with no veto (e.g. an awaited `Promise | void` checkpoint like `session/flush`), and `emit` for plain fire-and-forget notifications. The generator also cross-checks the tag against the signature where the shape is conclusive (a trailing `next` ⇒ waterfall) and hard-errors on a contradiction. Write the rest of the event's JSDoc to stand alone — it is the catalog entry's prose. + +**The core-data-structures catalog is a maintained surface, not a write-once artifact.** [docs/core-data-structures/](docs/core-data-structures/core.md) catalogs the spine vocabulary (core.md) and the per-seam types (sub-pages). When a change adds, removes, or reshapes a type the catalog documents — a new `…Map` variant, a new content-block or session-event type, a field on `GenerateOptions`/`Agent`/`ToolDefinition`/a bash type, or a whole new core/seam type — update the catalog in the SAME change: edit the prose, and for a pasted ` ```ts type-equiv ` block, re-copy it verbatim and keep `scripts/type-equiv.manifest.json` 1:1 with the blocks. The `verify-type-equiv` gate catches a *drifted paste* of an already-documented type, but it canNOT tell you a brand-new core type was never documented — that judgment is on the author and the reviewer. The definition of "core" (the spine-vs-seam line) is in [core.md § What counts as "core"](docs/core-data-structures/core.md#what-counts-as-core); a genuinely spine-level new type belongs in core.md, a new capability's vocabulary on a sub-page. See [development.md](docs/development.md#documenting-types-verbatim-ts-type-equiv) for the `ts type-equiv` mechanics. + +**Document the CURRENT state — the "what" and "why" — never the PROCESS or HISTORY of how it got there.** A comment, JSDoc, or doc paragraph describes what the code *is* and why it is that way, as if it had always been so. Do NOT narrate the change that produced it: no "previously X, now Y", "changed from", "used to", "this replaces", "the old map", "renamed", "moved here", "as of this PR", or "(was …)". **In particular, NEVER name the change unit a reader cannot see — the PR, commit, or stack position that introduced the code — in a comment, JSDoc, OR a test name/description.** A `// (PR D's per-agent teardown)` aside, a `* Tests for the cancel primitive (PR C).` module doc, or an `it('… identity no longer matters')` title that only makes sense relative to a prior design are all the same violation: the reader of the current tree has no "PR D" or "old design" to anchor against, and the reference rots the moment the stack merges. Name the *mechanism* (`the session's AgentHandle teardown`), not the PR. Such phrasing rots the instant the next change lands, and a reader of the current code does not need the diff narrated in prose — that belongs in the commit message, the PR description, or an RFC (the durable home for "why we moved away from X"). Write "the owner token lives on the task in the executor" — not "ownership *now* lives on the executor instead of a plugin-local map". When a contrast genuinely aids understanding (a non-obvious choice between live alternatives), frame it against the alternative as a standing fact ("stored on the executor, NOT the tool plugin, so it survives an HMR reload"), not against the codebase's past. The same rule governs review-fix commits: the *commit message* records what the review caught; the *code comment* it touches states only the resulting truth. RFCs (`docs/rfc/`, grouped into `proposed/` / `implemented/` / `rejected/`) record the *why* behind choices a future reader would otherwise re-litigate (the vendoring policy, event-sourcing, the schema DSL are the existing examples). A PR that introduces such a decision — a new third-party runtime dependency over the vendoring default, a cross-package contract, a security/isolation model, a deviation from a documented architecture rule — writes the RFC in `implemented/` **in the same PR**, and links it from the relevant code. A proposal for future work not yet built goes in `proposed/`. A PR whose changes are mechanical, self-evident, or already covered by an existing RFC needs none — do not manufacture an RFC for a routine change. When unsure, the test is: would a competent maintainer six months from now ask "why was it done this way?" and be unable to answer from the code alone? If yes, write it. See [docs/rfc/README.md](docs/rfc/README.md) for the naming scheme and [docs/AGENTS.md](docs/AGENTS.md) for the cross-link convention. + +**Markdown is not hard-wrapped**: write one line per paragraph and let the editor soft-wrap. Hard line breaks mid-paragraph make docs harder to edit and diff — a one-word change reflows and re-diffs the whole paragraph. This applies to prose only: leave fenced code blocks, tables, and list structure intact (a wrapped list item folds to one line per bullet). Code comments / JSDoc are exempt — they stay under the linter's column limit. `pnpm run verify-md-wrap` (part of `doc-sync`) enforces this across `README.md`, `docs/**/*.md`, `packages/*/*.md`, and `AGENTS.md` / `packages/AGENTS.md`; `pnpm run verify-md-links` (also part of `doc-sync`) checks that every relative cross-link in those files resolves. **Editing these instructions**: `AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it (at the repo root and in `packages/`). Always edit `AGENTS.md` — never write through the `CLAUDE.md` symlink or replace it with a regular file. diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 6d23352bff..7571209aa2 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -4,12 +4,12 @@ Conventions for authoring everything under `docs/` (architecture, RFCs, cookbook ## Cross-reference with machine-checkable links, never free prose -When one doc refers to another doc, an RFC, a package README, or any file in the repo, link it with a **relative Markdown link** to the actual path — `[capability seams](rfc/implemented/2026-06-13-capability-seams.md)`, `[architecture.md](architecture.md)`. Do NOT refer to it by bare prose or by a number ("see ADR 0009", "per RFC 005"): a number is not checkable, goes stale the moment a file is renamed, and forces the reader to go hunting. A relative link is verified mechanically — `pnpm run verify-md-links` (part of `doc-sync`, see [the cross-link lint RFC](rfc/implemented/2026-06-18-markdown-cross-link-lint.md)) fails CI and the pre-push hook if any relative target does not exist, so a rename that orphans a link is caught before review rather than rotting silently. +When one doc refers to another doc, an RFC, a package README, or any file in the repo, link it with a **relative Markdown link** to the actual path — `[capability seams](rfc/implemented/architecture/2026-06-13-capability-seams.md)`, `[architecture.md](architecture.md)`. Do NOT refer to it by bare prose or by a number ("see ADR 0009", "per RFC 005"): a number is not checkable, goes stale the moment a file is renamed, and forces the reader to go hunting. A relative link is verified mechanically — `pnpm run verify-md-links` (part of `doc-sync`, see [the cross-link lint RFC](rfc/implemented/process/2026-06-18-markdown-cross-link-lint.md)) fails CI and the pre-push hook if any relative target does not exist, so a rename that orphans a link is caught before review rather than rotting silently. -This is why the RFC tree carries no stable numbers: files are named `yyyy-mm-dd-topic-title.md` and referred to by link, so they survive moves between `proposed/`/`implemented/`/`rejected/` without a dangling reference. When you move or rename a doc, the gate tells you every inbound link you still need to fix. +This is why the RFC tree carries no stable numbers: files are named `yyyy-mm-dd-topic-title.md` and referred to by link, so they survive moves between lifecycle folders (`proposed/`/`implemented/`/`rejected/`) and class folders without a dangling reference. When you move or rename a doc, the gate tells you every inbound link you still need to fix. The gate checks file *existence*, not `#anchor` validity — a link to a real file with a wrong heading fragment still passes. Prefer linking the file (and a heading when it helps the reader), but don't rely on the gate to catch a stale anchor. ## RFCs -Design decisions and proposals live in [rfc/](rfc/) — one kind of doc, grouped by lifecycle into `proposed/`/`implemented/`/`rejected/`. See [rfc/README.md](rfc/README.md) for the naming scheme and when to write one. +Design decisions and proposals live in [rfc/](rfc/) — one kind of doc, grouped by lifecycle (`proposed/`/`implemented/`/`rejected/`) then by class (`feature`/`bug-fix`/`simplification`/`architecture`/`process`/`testing`). See [rfc/README.md](rfc/README.md) for the class definitions, the naming scheme, and when to write one. diff --git a/docs/architecture.md b/docs/architecture.md index 04b7f111f6..2b05e613b1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -8,6 +8,8 @@ The harness core is deliberately tiny: a handful of abstract services plus one c Requirement context: [Coding Harness MVP 需求分析][mvp-doc]. +For a catalog of the **data structures** this architecture moves around — the core vocabulary types, their literal shapes, and the seam types grouped by capability — see [core-data-structures/](core-data-structures/core.md). This document covers behavior; that one covers the types. + **Contents:** [Layering](#layering) · [Service map](#service-map) · [Capability seams](#capability-seams-interface--implementation--consumer) · [The vocabulary (dsh-llm)](#the-vocabulary-dsh-llm) · [Event-sourced sessions](#event-sourced-sessions-dsh-session) · [Prompt assembly](#prompt-assembly-dsh-system-prompt) · [Tool pipeline](#tool-pipeline-dsh-tools) · [Agents and the loop](#agents-dsh-agent-and-the-loop-dsh-agent-loop) ([lifecycle](#loop-lifecycle-session--turn--step), [event taxonomy](#event-taxonomy), [waterfall semantics](#cordis-waterfall-semantics-important)) · [Plugin sanity checklist](#plugin-sanity-checklist) · [Extension cookbook](#extension-cookbook) · [Deferred work](#deferred-work-todo) [microkernel-doc]: https://trtgsjkv6r.feishu.cn/wiki/VS9Lw1kQki6mDJk2UHocyuphnsc @@ -37,23 +39,25 @@ Requirement context: [Coding Harness MVP 需求分析][mvp-doc]. └─────────────────────────────────────────────────────────────┘ ``` -Dependency rule: plugins depend on interface packages, never on `dsh-agent-loop`. The loop itself is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. +Dependency rule: **extension** plugins depend on interface packages, never on `dsh-agent-loop`. The loop itself is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. The one sanctioned exception is a **composition/bundle** package whose job IS to assemble the concrete spine: `dsh-agent-core` bundles `dsh-agent-loop` (and the other concrete spine plugins) by design, so it depends on the concrete loop on purpose. The rule constrains plugins that EXTEND the system, not the bundle that COMPOSES it — swapping the loop means publishing a different bundle, not rewiring every extension. ## Service map | ctx key | Class | Package | Role | |---|---|---|---| -| `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` / `streamBlocks()` / `generate()` | +| `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` | | `ctx.sessions` | `SessionStore` | dsh-session | creates/holds event-sourced `Session`s | -| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list/update sessions | +| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list sessions | | `ctx.systemPrompt` | `SystemPrompt` | dsh-system-prompt | ordered sections + tool schemas → `assemble()` | | `ctx.tools` | `ToolRegistry` | dsh-tools | tool definitions; `execute()` through waterfall | -| `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam | -| `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `LoopAgent`s and drives their loops | +| `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam (returns an `AgentHandle` = `{ agent, dispose() }` for owned per-agent teardown) | +| `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `ReactLoopAgent`s and drives their loops | | `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks | All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically. +For each service's full public interface (every method signature, generated from source), plus the inherited cordis-core/loader/hmr/timer surface a plugin also sees, see the `## Services` section of [cordis-catalog/events-and-services.md](cordis-catalog/events-and-services.md). This table is the at-a-glance role summary; that catalog is the exhaustive reference. + ## Capability seams: interface / implementation / consumer Swappable capabilities are split into **three packages** so each part evolves independently. The bash capability is the template: @@ -79,13 +83,13 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta A `Session` is an append-only log of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`): - `user/message` → user message -- `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation) +- `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation; an empty-content `assistant/message`, which exists only to host a max-tokens step's `usage`, is skipped too) - `tool/result` → user message carrying a `tool-result` block - `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: the real adapters now exist (the original precondition); the envelope still wants a deliberate review against live model behavior (`TODO(review)` in dsh-session). Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`. -**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list/update over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionMeta`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic. +**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic. ## Prompt assembly (dsh-system-prompt) @@ -107,9 +111,9 @@ Tool schemas are deliberately **part of the assembly**: "what the model is told - `send(content)` — queued message; starts a turn when idle, else next turn - `steer(content)` — mid-turn injection, drained **between steps**; behaves like `send` when idle -- `inject(content)` — in-session context (`context/message` event); the next request sees it (Claude Code attachment / system-reminder analog). An inject made while the agent is *running* joins the open turn; an inject while *idle* is wrapped in a one-shot turn (`turn/start{trigger:injection}` → `context/message` → `turn/end`) so every event stays turn-enclosed (see [the turn-enclosure invariant](rfc/implemented/2026-06-15-turn-enclosure-invariant.md)). -- `abort(reason)` — aborts the in-flight step via `AbortSignal` -- `whenIdle()` — resolves once the agent reaches quiescence after settling out of `running` (resolves immediately when already idle; awaits the loop exit when disposed). The teardown signal: `abort()` then `await whenIdle()` guarantees the in-flight turn has fully stopped. Observes the transition without disposing the agent. +- `inject(content)` — in-session context (`context/message` event); the next request sees it (Claude Code attachment / system-reminder analog). An inject made while the agent is *running* joins the open turn; an inject while *idle* is wrapped in a one-shot turn (`turn/start{trigger:injection}` → `context/message` → `turn/end`) so every event stays turn-enclosed (see [the turn-enclosure invariant](rfc/implemented/architecture/2026-06-15-turn-enclosure-invariant.md)). +- `cancel(reason)` — the single public stop primitive: clears queued + steering work, aborts the in-flight step, and drops a turn about to start (the pre-step window) so a queued-but-not-started prompt never runs and cannot be batched into the cancelled turn. A UI/ACP `session/cancel` maps to it. +- `whenIdle()` — resolves once the agent reaches quiescence after settling out of `running` (resolves immediately when already idle; awaits the loop exit when disposed). A non-owner's quiescence-observation hook: it lets a consumer await the current work settling **without** disposing the agent. It is NOT teardown — it does not stop queued work, unregister the agent, or detach the session; a lifecycle owner tears an agent down with `await AgentHandle.dispose()` (which stops the loop, awaits its exit, and unregisters). - `session`, `status`, `options` **TODO(sub-agents)**: `spawn`/`fork` land on `AgentLoop.create()` — fork seeds the child Session with the parent's event log, spawn starts fresh; children are ordinary `Agent` handles so `steer()` and event subscription work uniformly. Inter-agent channels beyond these primitives are deliberately deferred. @@ -138,7 +142,7 @@ forever: step error (turn ends error/aborted, not a normal completed message) msg = waterfall agent/step-result ⟵ runs BEFORE the log append, so the - session('assistant/message', 'usage') log records what tool dispatch uses + session('assistant/message' {content, usage?}) log records what tool dispatch uses each tool-call (sequential, abort-checked between calls): session('tool/call'); ctx.tools.execute() ⟵ waterfall tools/execute session('tool/result') @@ -154,36 +158,17 @@ forever: emit agent/status(idle) unless more queued ``` -Error containment: a throwing `agent/turn-continuation` listener or a broken step ends the **turn** with an `error` event (appended INSIDE the turn, before `turn/end`) — never the driver loop. An adapter that ends its stream with a `finish {kind:'error'}` or `{kind:'aborted'}` chunk (the in-band error path, for adapters that can't throw mid-stream) is likewise translated into a step error, so the turn ends `error`/`aborted` instead of logging a normal `completed` assistant message. `abort()` is honored mid-stream **and** between tool calls; disposal mid-turn ends the turn with reason `disposed` and emits `agent/status('disposed')`. +Error containment: a throwing `agent/turn-continuation` listener or a broken step ends the **turn** with `turn/end { reason: { kind: 'error', step, message, code? } }` — the failure's step number rides on the durable turn reason (there is no separate session `error` event); live diagnostics fire via `agent/error`. Never the driver loop. An adapter that ends its stream with a `finish {kind:'error'}` or `{kind:'aborted'}` chunk (the in-band error path, for adapters that can't throw mid-stream) is likewise translated into a step error, so the turn ends `error`/`aborted` instead of logging a normal `completed` assistant message. A `cancel()` is honored mid-stream **and** between tool calls; disposal mid-turn ends the turn with reason `disposed` and emits `agent/status('disposed')`. Turn-end reasons: a turn ends with one `TurnEndReason` — `completed`, `aborted`, `error`, `disposed`, or `max-tokens`. `max-tokens` mirrors the model-call `FinishReason` of the same name (DeepSeek's `length`): a step that hit the output-token ceiling makes the turn end `max-tokens` rather than `completed`, by the rule *any `max-tokens` step in the turn surfaces as `max-tokens`* (a continuation plugin may run further steps after one, but the cut-short fact wins; the `disposed`/`aborted`/`error` outcomes still take precedence). This lets a consumer distinguish a clean stop from a truncated one (the ACP bridge maps it to the `max_tokens` stop reason). `TurnEndReason` is merge-extensible; `refusal` and `max_turn_requests` are the next variants to add when an adapter/loop first emits them. -A failure that happens once the turn is already closed has no in-turn position for a session `error` event (appending one after `turn/end` would put it past the persistence commit boundary, where it is dropped as a crash tail — [the turn-enclosure invariant](rfc/implemented/2026-06-15-turn-enclosure-invariant.md)). So a rejecting `session/flush` (the post-`turn/end` durability checkpoint) and a throwing `agent/turn-end` listener are reported via `agent/error` + the logger only, NOT as a session event; the turn stays balanced and the persistence backend keeps its buffered events for the next flush. +A failure that happens once the turn is already closed has no in-turn position for a turn-end error reason (the turn already ended). So a rejecting `session/flush` (the post-`turn/end` durability checkpoint) and a throwing `agent/turn-end` listener are reported via `agent/error` + the logger only, NOT as a session event; the turn stays balanced and the persistence backend keeps its buffered events for the next flush. -**Turn-enclosure invariant**: every session event lives inside a turn (between a `turn/start` and its `turn/end`). The loop appends queued `user/message` events *after* `turn/start`, and an idle `agent.inject()` wraps its `context/message` in a one-shot `injection` turn. This makes the turn the single durability/replay boundary: a persistence backend can treat anything after the last `turn/end` as an interrupted-crash tail without risking the loss of legitimately-recorded between-turn context. The `dsh-invariants` plugin enforces it in dev (a message event outside an open turn throws). See [the turn-enclosure invariant](rfc/implemented/2026-06-15-turn-enclosure-invariant.md). +**Turn-enclosure invariant**: every session event lives inside a turn (between a `turn/start` and its `turn/end`). The loop appends queued `user/message` events *after* `turn/start`, and an idle `agent.inject()` wraps its `context/message` in a one-shot `injection` turn. This makes the turn the single durability/replay boundary: a persistence backend can treat anything after the last `turn/end` as an interrupted-crash tail without risking the loss of legitimately-recorded between-turn context. The `dsh-invariants` plugin enforces it in dev (a message event outside an open turn throws). See [the turn-enclosure invariant](rfc/implemented/architecture/2026-06-15-turn-enclosure-invariant.md). ### Event taxonomy -The `agent/*` events are declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package); each other service declares its own events (`tools/*`, `llm/*`, `system-prompt/*`, `session/*`). The table below is CI-verified against the `interface Events` declarations in source (`scripts/verify-event-taxonomy.ts`). - -| Event | Mode | Purpose | -|---|---|---| -| `agent/created` / `agent/disposed` / `agent/status` / `agent/queued` | emit | lifecycle + inbox notifications | -| `agent/turn-start` / `agent/turn-end` / `agent/step-start` / `agent/step-end` | emit | boundaries | -| `agent/request` | **waterfall** | mutate the final `GenerateOptions` before the model call | -| `agent/stream-chunk` | emit | token-level UI/log feed | -| `agent/step-result` | **waterfall** | post-process the assistant message before tool dispatch | -| `agent/steering` | emit | steering content injected | -| `agent/turn-continuation` | **waterfall** | override the continue/stop decision | -| `agent/error` | emit | step/turn errors | -| `tools/execute` (dsh-tools) | **waterfall** | wrap/veto/sandbox tool execution | -| `tools/change` (dsh-tools) | emit | a tool was registered/unregistered | -| `llm/stream` / `llm/generate` (dsh-llm) | **waterfall** | model-call interception | -| `llm/adapter-change` (dsh-llm) | emit | an adapter was registered/unregistered | -| `system-prompt/assemble` (dsh-system-prompt) | **waterfall** | mutate the assembly | -| `system-prompt/change` (dsh-system-prompt) | emit | a section/tool-provider changed | -| `session/created` / `session/event` (dsh-session) | emit | session lifecycle + log feed | -| `session/flush` (dsh-session) | parallel (awaited) | durability checkpoint | +The `agent/*` events are declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package); each other service declares its own events (`tools/*`, `llm/*`, `system-prompt/*`, `session/*`). The full catalog — every event's exact signature, dispatch mode, and prose — is **generated from source** and lives in [cordis-catalog/events-and-services.md](cordis-catalog/events-and-services.md) (the `## Events` section), alongside the `ctx.` service interfaces. That file is regenerated by `scripts/gen-cordis-catalog.ts` and frozen by the `verify-cordis-catalog` freshness gate (part of `doc-sync`), so it cannot drift from the `interface Events` declarations. ### Cordis waterfall semantics (important) diff --git a/docs/cookbook/adding-a-package.md b/docs/cookbook/adding-a-package.md index ed40f242e4..3c9cdb07d5 100644 --- a/docs/cookbook/adding-a-package.md +++ b/docs/cookbook/adding-a-package.md @@ -5,29 +5,33 @@ The file-by-file checklist for a new `@deepseek-ai/dsh-` package. (Verifie ## 1. Create the package ``` -packages// - package.json # copy from packages/tools, adjust name/description/deps - tsconfig.json # extends ../../tsconfig.base.json, rootDir src, outDir lib, - # references: vendor/cosmokit, vendor/cordis (+ vendor/schemastery - # if you use Config, + ../ for each dsh dependency) +packages/// + package.json # copy from packages/core/tools, adjust name/description/deps + tsconfig.json # extends ../../../tsconfig.base.json, rootDir src, + # outDir lib/types, references: ../../../vendor/cosmokit, + # ../../../vendor/cordis (+ ../../../vendor/schemastery if + # you use Config, + ../..// for each dsh dep) src/index.ts # service default export or plugin (name/inject/apply/Config) tests/.spec.ts README.md # service API, events, extension points, design notes ``` -package.json invariants (enforced by `pnpm run constraints` / `scripts/check-workspace-constraints.ts`): `private: true`, `version: 0.0.1`, `type: module`, `cordis` in BOTH peerDependencies and devDependencies (same range). Mirror every dsh peer dependency in devDependencies. `schemastery` goes in `dependencies` (it is a runtime validator), matching agent-loop. +Choose an existing group when one matches the package's role (`core`, `llm`, `bash`, `session-persistence`, `ui`, `util`, or `support`). A new group is allowed, but it is a pure container: no `package.json`, no source files, and packages still sit exactly one level below it. + +package.json invariants (enforced by `pnpm run constraints` / `scripts/check-workspace-constraints.ts`): `private: true`, `version: 0.0.1`, `type: module`, `main: "lib/index.js"`, `types: "lib/types/index.d.ts"`, `exports["."].types: "./lib/types/index.d.ts"`, `exports["."].default: "./lib/index.js"`, `cordis` in BOTH peerDependencies and devDependencies (same range). Mirror every dsh peer dependency in devDependencies. `schemastery` goes in `dependencies` (it is a runtime validator), matching agent-loop. The `files` list is precise: `lib/index.js`, `lib/types/**/*.d.ts`, `lib/types/**/*.d.ts.map`, and `src`; do not publish `lib/types` JS or JS-map intermediates or stale root declaration files. CLI app packages with a package `bin` include `lib/bin.js` immediately after `lib/index.js` in `files`. + +In-package relative imports use explicit `.ts` specifiers in source (for example, `export * from './types.ts'`). The compiler rewrites those to `.js` in emitted JS and leaves explicit `.ts` specifiers in declarations, which standard NodeNext/Node16 TypeScript consumers resolve to the sibling `.d.ts` files. ## 2. Register it in the root configs | File | Change | |---|---| -| `tsconfig.base.json` | add `"@deepseek-ai/dsh-": ["./packages//src"]` to `paths` | -| `tsconfig.typecheck.json` | same entry (this file overrides the map wholesale) | -| `tsconfig.build.json` | add `{ "path": "./packages/" }` to `references` | -| `scripts/publint-all.ts` | add `'packages/'` to the array | -| `knip.json` | only if the package has non-`*.spec.ts` entries (e.g. `*.e2e.ts` → add a per-workspace override like `packages/llm-deepseek`) | +| `tsconfig.base.json` | no edit for an existing group; for a new group, add a `./packages//*/src` candidate to the `@deepseek-ai/dsh-*` wildcard | +| `tsconfig.json` | add `{ "path": "./packages//" }` to `references` | +| `tsconfig.build.json` | add `{ "path": "./packages//" }` to `references` | +| `knip.json` | only if the package has non-`*.spec.ts` entries (e.g. `*.e2e.ts` → add a per-workspace override like `packages/llm/llm-deepseek`) | -Covered automatically by globs — no edits needed: root `package.json` workspaces, `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`. +Covered automatically by globs or package-manifest discovery — no edits needed: root `package.json` workspaces, `scripts/publint-all.ts`, `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`, `scripts/check-workspace-constraints.ts`. ## 3. Decide the package topology @@ -39,7 +43,7 @@ For a swappable capability, split interface / implementation / consumer into sep pnpm install # registers the workspace pnpm run constraints && pnpm run typecheck && pnpm run lint pnpm run test:coverage # 100% per-file over src (types.ts exempt) -pnpm run build && pnpm run knip && pnpm run publint +pnpm run build && pnpm run hygiene ``` Test expectations: every registry/registration needs an HMR-safety test (register from a child fiber, dispose it, assert cleanup). Excessive tests are welcome — see AGENTS.md. diff --git a/docs/cookbook/adding-a-tool.md b/docs/cookbook/adding-a-tool.md index 8deb9510ae..73706c84f1 100644 --- a/docs/cookbook/adding-a-tool.md +++ b/docs/cookbook/adding-a-tool.md @@ -1,6 +1,6 @@ # Cookbook: adding a tool -How to give the model a new capability. Reference implementations: `examples/echo-agent/src/echo-tool.ts` (minimal) and `packages/tool-bash` (production-grade, three-package seam). +How to give the model a new capability. Reference implementations: `examples/echo-agent/src/echo-tool.ts` (minimal) and `packages/bash/tool-bash` (production-grade, three-package seam). ## The minimal shape @@ -33,7 +33,7 @@ Registration is effect-based: disposing the plugin fiber unregisters the tool (w ## Rules of the execute() contract -- **Args are validated for you.** `defineTool` validates the model-generated `arguments` against the `SchemaSpec` before `execute` runs (type, required keys, enum membership, nested objects/arrays — [runtime arg validation](../rfc/implemented/2026-06-11-runtime-arg-validation.md)), so inside `execute` the args already match `InferArgs`. You still hand-check value constraints the DSL can't express (non-empty strings, positive numbers, cross-field rules); throw a descriptive Error for those. Raw JSON-Schema tools registered directly (MCP) are NOT validated by the harness — they validate their own input. +- **Args are validated for you.** `defineTool` validates the model-generated `arguments` against the `SchemaSpec` before `execute` runs (type, required keys, enum membership, nested objects/arrays — [runtime arg validation](../rfc/implemented/architecture/2026-06-11-runtime-arg-validation.md)), so inside `execute` the args already match `InferArgs`. You still hand-check value constraints the DSL can't express (non-empty strings, positive numbers, cross-field rules); throw a descriptive Error for those. Raw JSON-Schema tools registered directly (MCP) are NOT validated by the harness — they validate their own input. - **Throwing means isError.** The registry catches anything `execute()` throws and returns `{isError: true}` to the model. Use that for infrastructure failures (bad input, spawn errors, aborts) — but REPORT domain failures in the result text instead (e.g. tool-bash returns `[exit code: 9]` with `isError: false`: the model decides what a failing command means). - **Honor `exec.signal`.** Cancel in-flight work when it fires. - **Use `exec.agent` for async notifications.** `agent.inject(content, {source: {kind: 'plugin', plugin: ''}})` appends durable context the NEXT model request sees — it is not a wake-up (an idle agent stays idle). Guard against disposed agents (try/catch). @@ -50,4 +50,4 @@ Prefer not to build policy into the tool. The seam is the `tools/execute` waterf ## Tests every tool needs -Arg-validation rejections, result shaping for every outcome, the HMR disposal test, and — for tools with side effects — an integration spec that drives the tool through the agent loop with a scripted `MockAdapter` (`packages/agent-loop/tests/mock-adapter.ts`), asserting the `tool/call` / `tool/result` session events. +Arg-validation rejections, result shaping for every outcome, the HMR disposal test, and — for tools with side effects — an integration spec that drives the tool through the agent loop with a scripted `MockAdapter` (`packages/core/agent-loop/tests/mock-adapter.ts`), asserting the `tool/call` / `tool/result` session events. diff --git a/docs/cookbook/adding-a-vendored-package.md b/docs/cookbook/adding-a-vendored-package.md index fa3c754cb8..59df2f617b 100644 --- a/docs/cookbook/adding-a-vendored-package.md +++ b/docs/cookbook/adding-a-vendored-package.md @@ -1,6 +1,6 @@ # Cookbook: adding a vendored package -When the harness needs another upstream Cordis package (e.g. `@cordisjs/plugin-http`), it is **vendored** as pinned source under `vendor/`, not added as an npm dependency — see [the vendoring decision](../rfc/implemented/2026-06-11-vendor-cordis-as-source.md) for why. [vendor/README.md](../../vendor/README.md) covers *updating* an already-vendored package; this guide is the file-by-file checklist for adding a **new** one. (Verified against the existing vendored set; if it drifts, fix it here.) +When the harness needs another upstream Cordis package (e.g. `@cordisjs/plugin-http`), it is **vendored** as pinned source under `vendor/`, not added as an npm dependency — see [the vendoring decision](../rfc/implemented/process/2026-06-11-vendor-cordis-as-source.md) for why. [vendor/README.md](../../vendor/README.md) covers *updating* an already-vendored package; this guide is the file-by-file checklist for adding a **new** one. (Verified against the existing vendored set; if it drifts, fix it here.) ## 1. Copy the source in @@ -12,13 +12,13 @@ vendor// README.md LICENSE # if upstream ships them ``` -`tsconfig.json` mirrors the other vendored packages — `rootDir: src`, `outDir: lib`, the strictness relaxations upstream code needs, and a `references` entry for every other vendored package it imports: +`tsconfig.json` mirrors the other vendored packages — `rootDir: src`, `outDir: lib/types`, the strictness relaxations upstream code needs, and a `references` entry for every other vendored package it imports: ```jsonc { "extends": "../../tsconfig.base.json", "compilerOptions": { - "rootDir": "src", "outDir": "lib", + "rootDir": "src", "outDir": "lib/types", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, "noImplicitOverride": false, "noUnusedLocals": false, "noUnusedParameters": false }, @@ -27,19 +27,21 @@ vendor// } ``` -`package.json` invariants: `"private": true` (vendored packages are never published), keep upstream's `name`/`version`/`exports`/`type`, and list its cordis deps in `peerDependencies` (matching the upstream manifest). Transitive upstream deps must themselves be vendored or already present — vendoring one package often means vendoring its dependency tree (e.g. `@cordisjs/plugin-http` pulls `@cordisjs/fetch-file`). +`package.json` invariants: `"private": true` (vendored packages are never published), keep upstream's `name`/`version`/`exports`/`type`, point declaration metadata at `lib/types`, publish `.d.ts` and `.d.ts.map` declaration outputs, and list its cordis deps in `peerDependencies` (matching the upstream manifest). Transitive upstream deps must themselves be vendored or already present — vendoring one package often means vendoring its dependency tree (e.g. `@cordisjs/plugin-http` pulls `@cordisjs/fetch-file`). + +Local relative imports/exports in vendored TypeScript source use explicit `.ts` specifiers after copying. This is a repo-local build-shape divergence from upstream: `rewriteRelativeImportExtensions` emits `.js` runtime imports while declarations keep explicit `.ts` specifiers that NodeNext/Node16 TypeScript consumers can resolve. ## 2. Register it in the root configs | File | Change | |---|---| | `tsconfig.base.json` | add `"": ["./vendor//src"]` to `paths` | -| `tsconfig.typecheck.json` | add `"": ["./vendor//lib"]` — this file points at built declarations, not src. If the package's `types` entry isn't `lib/index.d.ts`, point at that built file instead (e.g. `logger-console` maps to `./vendor/logger-console/lib/shared`, matching its `"types": "lib/shared.d.ts"`). | +| `tsconfig.json` | add `{ "path": "./vendor/" }` to `references` | | `tsconfig.build.json` | add `{ "path": "./vendor/" }` to `references` (before the `packages/*` entries) | | `vendor/README.md` | add a manifest table row (dir, npm name, version, upstream repo, commit SHA) and log any local modifications | | `scripts/publint-all.ts` | only if the vendored package is itself published from here (vendored deps normally are not — skip) | -Covered automatically by globs — no edits needed: root `package.json` workspaces (`vendor/*`), `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`. A per-package `vendor//tsdown.config.ts` is needed ONLY if the build shape diverges from the root default (dual ESM/CJS or multiple entries — see `vendor/schemastery` and `vendor/logger-console`). +Covered automatically by globs — no edits needed: root `package.json` workspaces (`vendor/*`), `tsdown.config.ts`, `vitest.config.ts`, `eslint.config.mjs`. A per-package `vendor//tsdown.config.ts` is needed ONLY if the build shape diverges from the root default (dual ESM/CJS or multiple entries — see `vendor/schemastery` and `vendor/logger-console`); its entry should read the JS emitted under `lib/types`. ## 3. Mind the manifest guard @@ -49,8 +51,8 @@ Covered automatically by globs — no edits needed: root `package.json` workspac ```sh pnpm install # registers the workspace -pnpm run typecheck # the base→lib path split means: run once after a fresh add +pnpm run typecheck pnpm run build && pnpm run test && pnpm run constraints ``` -Note the `tsconfig` two-map split (called out in [AGENTS.md](../../AGENTS.md) § Secrets/.env): `lint`'s type-aware rules resolve vendored packages through their built `lib/` declarations, so run `pnpm run typecheck` (which builds them) once after adding the package or lint reports unresolved-type errors. +The source `paths` map is shared by build and root typecheck configs. The important isolation boundary is the project-reference graph: vendored source must be referenced through its own `vendor//tsconfig.json`, not pulled into a root strict program. diff --git a/docs/cookbook/adding-an-llm-adapter.md b/docs/cookbook/adding-an-llm-adapter.md index 19bc4e9e05..aae6b0bb8b 100644 --- a/docs/cookbook/adding-an-llm-adapter.md +++ b/docs/cookbook/adding-an-llm-adapter.md @@ -1,6 +1,6 @@ # Cookbook: adding an LLM adapter -How to connect a new model provider. Reference implementations: `packages/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm-pi-ai` (wrapping an LLM library). Read the `StreamChunk` doc in `packages/llm/src/types.ts` first — it records the protocol conventions both adapters were verified against. +How to connect a new model provider. Reference implementations: `packages/llm/llm-deepseek` (hand-rolled HTTP/SSE) and `packages/llm/llm-pi-ai` (wrapping an LLM library). Read the `StreamChunk` doc in `packages/llm/llm/src/types.ts` first — it records the protocol conventions both adapters were verified against. ## The shape diff --git a/docs/cookbook/extension-cookbook.md b/docs/cookbook/extension-cookbook.md index df40f1384a..e6c0378361 100644 --- a/docs/cookbook/extension-cookbook.md +++ b/docs/cookbook/extension-cookbook.md @@ -38,6 +38,7 @@ A UI plugin consumes `agent/stream-chunk` and session events for rendering, and ```ts import type { Context } from 'cordis' +import { AgentId } from '@deepseek-ai/dsh-agent' declare function render(text: string): void declare function onUserInput(handler: (text: string) => void): void @@ -49,15 +50,15 @@ export function apply(ctx: Context) { ctx.on('agent/stream-chunk', (agent, turn, step, chunk) => { if (chunk.type === 'text-delta') render(chunk.text) }) - onUserInput(text => ctx.agents.get('main')?.send([{ type: 'text', text }])) + onUserInput(text => ctx.agents.get(AgentId('main'))?.send([{ type: 'text', text }])) } ``` ## A client-driver plugin (external protocol bridge) -A *client driver* is a UI plugin whose "user" is another program speaking a wire protocol rather than a human at a terminal. It owns the process's stdio (so it must run with **no stdout logger** — every non-protocol byte corrupts the stream), creates/resumes agents on demand through the `dsh-agent` factory seam, translates harness events (`session/event`, `agent/*`) into outbound protocol messages, and translates inbound requests back into `agent.send()` / `agent.abort()`. Two harness-specific contracts make it correct: resolve each request exactly once off a settle signal (the turn can end without its `agent/turn-end` event firing — fall back through the logged `turn/end` record), and on disposal reach quiescence (`await agent.whenIdle()` after `abort()`), not just request it. +A *client driver* is a UI plugin whose "user" is another program speaking a wire protocol rather than a human at a terminal. It owns the process's stdio (so it must run with **no stdout logger** — every non-protocol byte corrupts the stream), creates/resumes agents on demand through the `dsh-agent` factory seam, translates harness events (`session/event`, `agent/*`) into outbound protocol messages, and translates inbound requests back into `agent.send()` / `agent.cancel()`. Two harness-specific contracts make it correct: resolve each request exactly once off a settle signal (the turn can end without its `agent/turn-end` event firing — fall back through the logged `turn/end` record), and tear each agent down through its `AgentHandle.dispose()` (which stops the loop, `await`s its exit, and unregisters), not just `cancel()` — disposal must *reach* quiescence, not merely request it. -`packages/acp` is the worked example: it bridges the agent to the Agent Client Protocol (JSON-RPC over stdio) so Zed and other ACP editors can drive it. See its README for the full method surface and the deferred-permission-gate note. +`packages/ui/acp` is the worked example: it bridges the agent to the Agent Client Protocol (JSON-RPC over stdio) so Zed and other ACP editors can drive it. See its README for the full method surface and the deferred-permission-gate note. ```ts import type { Context } from 'cordis' @@ -76,10 +77,10 @@ export function apply(ctx: Context) { } }) // Inbound "prompt": create/resume an agent and feed it; settle on turn end. - // Disposal awaits quiescence: agent.abort() then await agent.whenIdle(). + // Teardown reaches quiescence via AgentHandle.dispose() (stop + await exit). } ``` ## Runnable wirings -Three complete examples load their plugin trees from `cordis.yml` with HMR: [`examples/echo-agent`](../../examples/echo-agent) (mock model + echo tool — the all-mock skeleton check, `pnpm run demo:echo`), [`examples/coding-agent`](../../examples/coding-agent) (DeepSeek V4 + the bash tool suite — the real thing, `pnpm run demo:coding`), and [`examples/acp-agent`](../../examples/acp-agent) (the same coding agent exposed as an ACP server over JSON-RPC stdio — the client-driver shape, `pnpm run demo:acp`). The two real demos share their provider/tool core via [`examples/base.yml`](../../examples/base.yml). +Three complete examples load their plugin trees from `cordis.yml`: [`examples/echo-agent`](../../examples/echo-agent) (mock model + echo tool — the all-mock skeleton check, `pnpm run demo:echo`), [`examples/coding-agent`](../../examples/coding-agent) (DeepSeek V4 + the bash tool suite — the real thing, `pnpm run demo:coding`), and [`examples/acp-agent`](../../examples/acp-agent) (the same coding agent exposed as an ACP server over JSON-RPC stdio — the client-driver shape, `pnpm run demo:acp`). Each leaf is now just its swappable backends plus an app-package entry: the stdio demos load [`@deepseek-ai/dsh-stdio-agent`](../../packages/ui/stdio-agent), the ACP demo loads [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent), and both app packages share the spine via the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) bundle. diff --git a/docs/cordis-catalog/events-and-services.md b/docs/cordis-catalog/events-and-services.md new file mode 100644 index 0000000000..083d3295f6 --- /dev/null +++ b/docs/cordis-catalog/events-and-services.md @@ -0,0 +1,455 @@ + + +# Cordis Events & Services Catalog + +An index reference to the **wiring** a plugin author works against: every cordis event you can listen to (exact signature + dispatch mode) and every `ctx.` service you can call (exact public interface). It complements [core-data-structures/](../core-data-structures/core.md), which catalogs the *data structures* these signatures move around — this page is the verbs, that page is the nouns. + +This file is GENERATED from source (`scripts/gen-cordis-catalog.ts`) and verified fresh by `pnpm run verify-cordis-catalog` (part of `doc-sync`) — do not edit it by hand. Signature blocks use a `ts cordis-catalog` fence (skipped by doc-typecheck, since a bare signature is not standalone-compilable). Type names in a signature link to the page that documents them. + +The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary this repo owns. The **inherited tier** at the end is the cordis-core + loader/hmr/timer surface a plugin also sees — pinned vendor source, summarized tersely. + +## Events + +Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 22 events across 5 scopes. + +### `agent/*` + +#### `agent/created` — emit + +An agent was registered in the AgentRegistry and is ready to receive messages. + +```ts cordis-catalog +'agent/created'(agent: Agent): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:136`](../../packages/core/agent/src/types.ts) + +#### `agent/disposed` — emit + +An agent was disposed and removed from the registry; its fiber and any in-flight turn have been torn down. + +```ts cordis-catalog +'agent/disposed'(agent: Agent): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:142`](../../packages/core/agent/src/types.ts) + +#### `agent/error` — emit + +A step or turn errored. The loop reports a failure here (plus the logger) even when the error has no in-turn position for a session `error` event. + +```ts cordis-catalog +'agent/error'(agent: Agent, turn: number, step: number, error: Error): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:219`](../../packages/core/agent/src/types.ts) + +#### `agent/queued` — emit + +A message entered the agent's inbox (queued or steering). `source` is the resolved source (defaults applied), not the caller's raw options. + +```ts cordis-catalog +'agent/queued'(agent: Agent, content: ContentBlock[], info: { source: MessageSource; steering: boolean }): void +``` + +Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:155`](../../packages/core/agent/src/types.ts) + +#### `agent/request` — waterfall + +Waterfall: mutate the fully-assembled GenerateOptions before the model call (hooks, compaction, model switching, tool filtering, …). Call `next()` to delegate, or return without it to short-circuit. + +```ts cordis-catalog +'agent/request'(agent: Agent, turn: number, step: number, options: GenerateOptions, next: () => Promise): Promise +``` + +Types: [Agent](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:188`](../../packages/core/agent/src/types.ts) + +#### `agent/status` — emit + +Agent status changed (`idle` ⇄ `running`, or → `disposed`). Drive lifecycle off this transition, never off a status you just requested — `send()` does not flip status to `running` before it returns. + +```ts cordis-catalog +'agent/status'(agent: Agent, status: AgentStatus): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:149`](../../packages/core/agent/src/types.ts) + +#### `agent/steering` — emit + +Steering content was injected into a running turn. + +```ts cordis-catalog +'agent/steering'(agent: Agent, turn: number, content: ContentBlock[], source: MessageSource): void +``` + +Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:213`](../../packages/core/agent/src/types.ts) + +#### `agent/step-end` — emit + +A step ended. + +```ts cordis-catalog +'agent/step-end'(agent: Agent, turn: number, step: number): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:179`](../../packages/core/agent/src/types.ts) + +#### `agent/step-result` — waterfall + +Waterfall: post-process the assembled assistant Message before tool dispatch (validation, content rewriting, …). + +```ts cordis-catalog +'agent/step-result'(agent: Agent, turn: number, step: number, message: Message, next: () => Promise): Promise +``` + +Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:194`](../../packages/core/agent/src/types.ts) + +#### `agent/step-start` — emit + +A step (one model call plus its tool dispatch) began. `step` is 1-based within the turn; a turn runs one or more steps. + +```ts cordis-catalog +'agent/step-start'(agent: Agent, turn: number, step: number): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:174`](../../packages/core/agent/src/types.ts) + +#### `agent/stream-chunk` — emit + +A raw StreamChunk arrived from the model (token-level UI/log feed). + +```ts cordis-catalog +'agent/stream-chunk'(agent: Agent, turn: number, step: number, chunk: StreamChunk): void +``` + +Types: [Agent](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) + +Source: [`packages/core/agent/src/types.ts:208`](../../packages/core/agent/src/types.ts) + +#### `agent/turn-continuation` — waterfall + +Waterfall: override the turn-continuation decision. The default (computed by the loop) is `hadToolCalls || steeringInjected`. Listeners can force-continue (/goal, /loop) or force-stop (budget guards). + +```ts cordis-catalog +'agent/turn-continuation'(agent: Agent, turn: number, defaultDecision: boolean, next: () => Promise): Promise +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:201`](../../packages/core/agent/src/types.ts) + +#### `agent/turn-end` — emit + +A turn ended. `reason` distinguishes a clean stop from a truncated or aborted one (`completed` | `aborted` | `error` | `disposed` | `max-tokens`). + +```ts cordis-catalog +'agent/turn-end'(agent: Agent, turn: number, reason: TurnEndReason): void +``` + +Types: [Agent](../core-data-structures/core.md) · [TurnEndReason](../core-data-structures/session.md) + +Source: [`packages/core/agent/src/types.ts:168`](../../packages/core/agent/src/types.ts) + +#### `agent/turn-start` — emit + +A turn began. `turn` is the 1-based turn number within the session. + +```ts cordis-catalog +'agent/turn-start'(agent: Agent, turn: number): void +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/types.ts:162`](../../packages/core/agent/src/types.ts) + +### `llm/*` + +#### `llm/stream` — waterfall + +Waterfall around every streaming model call (retry, caching, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit. + +```ts cordis-catalog +'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable +``` + +Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) + +Source: [`packages/llm/llm/src/index.ts:31`](../../packages/llm/llm/src/index.ts) + +### `session/*` + +#### `session/created` — emit + +A session was created in the store. + +```ts cordis-catalog +'session/created'(session: Session): void +``` + +Source: [`packages/core/session/src/index.ts:30`](../../packages/core/session/src/index.ts) + +#### `session/event` — emit + +An event was appended to a session log (sync, fire-and-forget). This is the per-append feed a UI or invariant plugin tails. + +```ts cordis-catalog +'session/event'(session: Session, event: SessionEvent): void +``` + +Types: [SessionEvent](../core-data-structures/core.md) + +Source: [`packages/core/session/src/index.ts:36`](../../packages/core/session/src/index.ts) + +#### `session/flush` — parallel + +Awaited durability checkpoint. The agent loop awaits `ctx.parallel('session/flush', session)` at every turn end; persistence plugins (JSONL, SQLite) drain their write-behind buffers here and on fiber dispose. Awaited (parallel), not a waterfall: every listener runs and the loop waits for all of them, but none can veto. + +```ts cordis-catalog +'session/flush'(session: Session): Promise | void +``` + +Source: [`packages/core/session/src/index.ts:45`](../../packages/core/session/src/index.ts) + +### `system-prompt/*` + +#### `system-prompt/assemble` — waterfall + +Waterfall around prompt assembly — mutate or extend the PromptAssembly (sections + tool schemas) before it is rendered. Bound to the SystemPrompt service; call `next()` to delegate. + +```ts cordis-catalog +'system-prompt/assemble'(this: SystemPrompt, assembly: PromptAssembly, next: () => Promise): Promise +``` + +Source: [`packages/core/system-prompt/src/index.ts:24`](../../packages/core/system-prompt/src/index.ts) + +#### `system-prompt/change` — emit + +A section or tool provider was registered or unregistered (the assembly inputs changed). + +```ts cordis-catalog +'system-prompt/change'(): void +``` + +Source: [`packages/core/system-prompt/src/index.ts:30`](../../packages/core/system-prompt/src/index.ts) + +### `tools/*` + +#### `tools/change` — emit + +A tool was registered or unregistered (the available tool set changed). + +```ts cordis-catalog +'tools/change'(): void +``` + +Source: [`packages/core/tools/src/index.ts:48`](../../packages/core/tools/src/index.ts) + +#### `tools/execute` — waterfall + +Waterfall around every tool execution — the single seam where sandbox, permission, hook, and plan-mode plugins wrap or veto a call. Listeners receive `(exec, next)`: call `next()` to proceed (possibly around your own logic), or return a ToolExecutionResult without calling `next()` to short-circuit (veto). + +```ts cordis-catalog +'tools/execute'(this: ToolRegistry, exec: ToolExecution, next: () => Promise): Promise +``` + +Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) + +Source: [`packages/core/tools/src/index.ts:43`](../../packages/core/tools/src/index.ts) + +## Services + +The 8 `ctx.` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against. + +### `ctx.agentLoop` — `AgentLoop` + +The agent-loop plugin (`ctx.agentLoop`): creates ReactLoopAgents, runs their loops, and registers them in `ctx.agents`. Also implements the AgentFactory seam, so plugins create/resume agents through `ctx.agents` (the interface) without depending on this concrete package. + +The loop itself is deliberately thin — every behavior beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy declared in @deepseek-ai/dsh-agent. + +```ts cordis-catalog +create(id: AgentId, options: AgentOptions = {}): ReactLoopAgent +createAgent(options: CreateAgentOptions): AgentHandle +async resume(options: ResumeAgentOptions): Promise +``` + +Source: [`packages/core/agent-loop/src/index.ts:63`](../../packages/core/agent-loop/src/index.ts) + +### `ctx.agents` — `AgentRegistry` + +Agent registry (`ctx.agents`): tracks live agents so UI, hook, and orchestrator plugins can find them without depending on the concrete loop package. Agent *creation* is provided by whichever plugin implements the AgentFactory (phase 1: `@deepseek-ai/dsh-agent-loop`), registered via setFactory. + +```ts cordis-catalog +setFactory(factory: AgentFactory): () => void +create(options: CreateAgentOptions): AgentHandle +async resume(options: ResumeAgentOptions): Promise +register(agent: Agent): () => void +get(id: AgentId): Agent | undefined +list(): Agent[] +``` + +Types: [Agent](../core-data-structures/core.md) + +Source: [`packages/core/agent/src/index.ts:105`](../../packages/core/agent/src/index.ts) + +### `ctx.bash` — `BashExecutor` (abstract seam) + +Abstract bash execution service. Subclass, implement the abstract methods, and load the subclass as a plugin — it registers as `ctx.bash` (one implementation per context; loading a second throws, which is cordis' standard duplicate-service behavior). + +Semantics every implementation must honor: + +- run REJECTS only for infrastructure failures (unusable workdir, missing shell, pre-aborted signal). Nonzero exits, timeout kills, and abort kills RESOLVE with a descriptive BashRunResult — reporting a failed command is the tool layer's job, not an exception. +- start returns immediately; no timeout applies to background tasks (callers stop them via kill or the spec's AbortSignal). Completion must fire the onTaskDone listeners exactly once per task, and must NOT fire after the service is disposed. +- readOutput is incremental: consecutive reads never re-deliver output. Implementations bound their buffers; reads that lost data flag `lossy` and point at full-stream spill files when available. +- Disposal kills every running task and awaits their exit (no orphan processes survive `fiber.dispose()`). + +```ts cordis-catalog +abstract resolve(request: BashExecRequest): BashExecSpec +abstract run(spec: BashExecSpec): Promise +abstract start(spec: BashExecSpec): BashTask +abstract get(id: BashTaskId): BashTask | undefined +abstract ownerOf(id: BashTaskId): OwnerToken | undefined +abstract list(): BashTask[] +abstract readOutput(id: BashTaskId): BashTaskRead +abstract kill(id: BashTaskId): boolean +onTaskDone(listener: BashTaskListener): () => void +``` + +Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../core-data-structures/bash.md) · [BashRunResult](../core-data-structures/bash.md) · [BashTask](../core-data-structures/bash.md) · [BashTaskRead](../core-data-structures/bash.md) + +Source: [`packages/bash/bash/src/index.ts:59`](../../packages/bash/bash/src/index.ts) + +### `ctx.llm` — `LlmService` + +The abstract `llm` service: an adapter registry plus a streaming model-call surface, interceptable via the `llm/stream` waterfall. + +```ts cordis-catalog +registerAdapter(models: string[], adapter: LlmAdapter): () => void +models(): string[] +stream(options: GenerateOptions): AsyncIterable +``` + +Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) + +Source: [`packages/llm/llm/src/index.ts:69`](../../packages/llm/llm/src/index.ts) + +### `ctx.sessionPersistence` — `SessionPersistence` (abstract seam) + +Abstract durable session-persistence service. Subclass, implement the abstract methods, and load the subclass as a plugin — it registers as `ctx.sessionPersistence` (one implementation per context; loading a second throws, cordis' standard duplicate-service behavior). + +Contracts every implementation MUST honor (a DB backend asserts them inside a transaction; a file backend appends at EOF): + +- **Append-only; a crashed turn is closed, not truncated.** Committed events — those at or below a flushed `turn/end` — are never rewritten. A crash can leave an unclosed final turn whose events are real (and possibly large); load preserves them and closes the orphaned turn with synthetic boundary events (see load). Only a never-fully-written torn tail fragment is discarded. +- **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`. load rejects a parse error or a `seq` gap in the COMMITTED region (unloadable); append's first event `seq` MUST equal the backend's stored next-seq (after `load` has balanced any interrupted turn). +- **JSON-serializable data.** `SessionEventMap` is merge-extensible and `event.data` is typed only as `SessionEventMap[K]`, so append REJECTS non-JSON-serializable data with an error naming the offending event type. A backend snapshots (serializes/clones) each event when it buffers, since `session.events` hands out the live mutable object. +- **Durability.** append returns only once the batch is durable (the file backend fsyncs; a DB commits). create MAY defer the physical write until the first append (lazy materialization). + +```ts cordis-catalog +abstract create(meta: SessionHeader): Promise +abstract append(id: SessionId, events: readonly SessionEvent[]): Promise +abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> +abstract list(): Promise +``` + +Types: [SessionEvent](../core-data-structures/core.md) + +Source: [`packages/session-persistence/session-persistence/src/index.ts:98`](../../packages/session-persistence/session-persistence/src/index.ts) + +### `ctx.sessions` — `SessionStore` + +In-memory session store (`ctx.sessions`). + +Persistence is intentionally not implemented here — persistence plugins subscribe to `session/event` and flush on `session/flush` / dispose. + +```ts cordis-catalog +create(id?: SessionId, options?: CreateSessionOptions): Session +prepare(id?: SessionId, options?: CreateSessionOptions): Session +enter(session: Session): () => void +announce(session: Session): void +get(id: SessionId): Session | undefined +list(): Session[] +``` + +Source: [`packages/core/session/src/index.ts:229`](../../packages/core/session/src/index.ts) + +### `ctx.systemPrompt` — `SystemPrompt` + +Registry service (`ctx.systemPrompt`): plugins contribute ordered text sections and tool-schema providers; the agent loop calls `assemble()` once per step. + +```ts cordis-catalog +section(section: PromptSection): () => void +tools(provider: () => ToolSchema[]): () => void +assemble(): Promise +``` + +Source: [`packages/core/system-prompt/src/index.ts:71`](../../packages/core/system-prompt/src/index.ts) + +### `ctx.tools` — `ToolRegistry` + +Tool registry (`ctx.tools`): tool plugins register definitions; the agent loop executes calls through the `tools/execute` waterfall. The registry contributes its schemas into the system-prompt assembly. + +```ts cordis-catalog +register(definition: ToolDefinition): () => void +get(name: string): ToolDefinition | undefined +schemas(): ToolSchema[] +async execute(exec: ToolExecution): Promise +``` + +Types: [ToolDefinition](../core-data-structures/tools.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md) + +Source: [`packages/core/tools/src/index.ts:277`](../../packages/core/tools/src/index.ts) + +## Inherited tier (cordis core + loader/hmr/timer) + +The framework surface every plugin inherits, beyond the harness vocabulary above. This is pinned vendor source ([vendoring policy](../../vendor/README.md)); it is summarized here so the catalog is a complete picture of what `ctx` and the event bus offer, without elevating framework internals to the harness tier's prominence. + +### Inherited events + +- `internal/plugin` — A plugin fiber was created. ([`vendor/cordis/src/events.ts:197`](../../vendor/cordis/src/events.ts)) +- `internal/status` — A fiber changed lifecycle state. ([`vendor/cordis/src/events.ts:198`](../../vendor/cordis/src/events.ts)) +- `internal/service` — Interception hook for a service binding (no core producer). ([`vendor/cordis/src/events.ts:199`](../../vendor/cordis/src/events.ts)) +- `internal/update` — Waterfall: a fiber config update is being applied. ([`vendor/cordis/src/events.ts:200`](../../vendor/cordis/src/events.ts)) +- `internal/get` — Waterfall: a service is being read from the store. ([`vendor/cordis/src/events.ts:201`](../../vendor/cordis/src/events.ts)) +- `internal/set` — Waterfall: a service is being written to the store. ([`vendor/cordis/src/events.ts:202`](../../vendor/cordis/src/events.ts)) +- `internal/listener` — A listener was registered. ([`vendor/cordis/src/events.ts:203`](../../vendor/cordis/src/events.ts)) +- `internal/dispatch` — An event is being dispatched to listeners. ([`vendor/cordis/src/events.ts:204`](../../vendor/cordis/src/events.ts)) +- `hmr/change` — A watched source file changed on disk. ([`vendor/hmr/src/index.ts:20`](../../vendor/hmr/src/index.ts)) +- `hmr/reload` — Plugins are being reloaded after a change. ([`vendor/hmr/src/index.ts:21`](../../vendor/hmr/src/index.ts)) +- `exit` — The process is exiting on a signal. ([`vendor/loader/src/index.ts:23`](../../vendor/loader/src/index.ts)) +- `loader/config-update` — The loader config tree changed. ([`vendor/loader/src/index.ts:24`](../../vendor/loader/src/index.ts)) +- `loader/entry-init` — A config entry is being initialized. ([`vendor/loader/src/index.ts:25`](../../vendor/loader/src/index.ts)) +- `loader/partial-dispose` — An entry is being partially disposed on reload. ([`vendor/loader/src/index.ts:26`](../../vendor/loader/src/index.ts)) +- `loader/patch-context` — A context is being patched during a reload. ([`vendor/loader/src/index.ts:27`](../../vendor/loader/src/index.ts)) + +### Inherited `ctx` members + +- `ctx.on / ctx.once` — Register an event listener (disposable). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts)) +- `ctx.emit / ctx.parallel / ctx.serial / ctx.bail / ctx.waterfall` — Dispatch an event (sync / awaited / first-non-nullish / veto-chain). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts)) +- `ctx.plugin / ctx.inject` — Load a plugin / declare required services. ([`vendor/cordis/src/registry.ts:144`](../../vendor/cordis/src/registry.ts)) +- `ctx.effect` — Register a disposable side effect tied to the fiber. ([`vendor/cordis/src/fiber.ts:9`](../../vendor/cordis/src/fiber.ts)) +- `ctx.get / ctx.set / ctx.provide / ctx.accessor / ctx.mixin` — Low-level service-store access and binding. ([`vendor/cordis/src/reflect.ts:7`](../../vendor/cordis/src/reflect.ts)) +- `ctx.extend / ctx.isolate / ctx.intercept` — Derive a child context (scoped services / isolation / interception). ([`vendor/cordis/src/context.ts:35`](../../vendor/cordis/src/context.ts)) +- `ctx.root / ctx.scope / ctx.fiber / ctx.registry / ctx.reflect / ctx.events / ctx.logger` — Ambient handles onto the running context graph. ([`vendor/cordis/src/context.ts:16`](../../vendor/cordis/src/context.ts)) +- `ctx.timer (+ interval / timeout / throttle / debounce / setTimeout / setInterval)` — Disposable timer helpers. The `timer` key is provided at runtime; the six helpers are mixed onto ctx directly (declared via Pick). ([`vendor/timer/src/index.ts:4`](../../vendor/timer/src/index.ts)) +- `ctx.loader` — The config Loader that booted the app (present under the loader). ([`vendor/loader/src/index.ts:30`](../../vendor/loader/src/index.ts)) +- `ctx.hmr` — The hot-module-reload watcher (present under the hmr plugin). ([`vendor/hmr/src/index.ts:15`](../../vendor/hmr/src/index.ts)) diff --git a/docs/core-data-structures/bash.md b/docs/core-data-structures/bash.md new file mode 100644 index 0000000000..807c7401fc --- /dev/null +++ b/docs/core-data-structures/bash.md @@ -0,0 +1,125 @@ +# Bash Executor + +The bash execution seam — the canonical [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) example, split across three packages: interface ([dsh-bash](../../packages/bash/bash), `ctx.bash`), implementation ([dsh-bash-local](../../packages/bash/bash-local), local subprocesses), and consumer ([dsh-tool-bash](../../packages/bash/tool-bash), the `bash`/`bash_output`/`bash_kill` tool schemas). Bash is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A sandboxed, containerized, or remote backend is a sibling package implementing the same interface. + +Source: [`packages/bash/bash/src/types.ts`](../../packages/bash/bash/src/types.ts) + +## Request vs. spec: the `resolve()` split + +The seam separates the **model-/plugin-facing request** (optional `workdir`/`timeoutMs`, filled from config) from the **fully-resolved spec** the executor acts on (those fields required). The tool layer calls `ctx.bash.resolve(request)` between them — this is the repo's "explicit > implicit at package seams" rule made concrete: the reader of a `BashExecSpec` never wonders where the working directory came from. + +```ts type-equiv +interface BashExecRequest { + command: string + /** Working directory override (default: implementation-configured). */ + workdir?: string | undefined + /** Timeout override in milliseconds (implementations cap it). */ + timeoutMs?: number | undefined + /** Abort signal — implementations kill the command when it fires. */ + signal?: AbortSignal | undefined + /** + * Opaque OWNER token for a background task — the consumer's isolation key + * (the tool layer passes the owning agent's `session.header.id`). The + * executor stores it on the task and exposes it via {@link BashExecutor.ownerOf}; + * the executor itself NEVER interprets it (no access policy lives in the + * seam — that is the consumer's job). Absent for foreground runs and for an + * ownerless background start (a non-agent caller). + */ + owner?: OwnerToken | undefined +} +``` + +```ts type-equiv +interface BashExecSpec { + command: string + workdir: string + timeoutMs: number + /** Abort signal — implementations kill the command when it fires. */ + signal?: AbortSignal | undefined + /** + * Opaque owner token, REQUIRED-but-nullable (mirrors `workdir`/`timeoutMs` + * being required on the resolved spec): {@link BashExecutor.resolve} carries + * the request's `owner` through, defaulting a missing one to `undefined`. A + * required field makes a forgotten owner a VISIBLE `undefined` rather than a + * silently-absent property that yields an unowned (cross-session-readable) + * task. `start()` stores it; `run()` (foreground) ignores it. + */ + owner: OwnerToken | undefined +} +``` + +The `owner` token is the isolation key: the executor stores it but never interprets it (access policy is the consumer's job), so a background task started by one agent isn't readable cross-session. A required-but-nullable field makes a forgotten owner a visible `undefined` rather than a silently-unowned task. + +Both ids the seam handles are [branded](core.md) (zero-cost `string` brands, the same machinery as `SessionId`/`AgentId`): `BashTaskId` (a tracked background task, generated `bash-N` by the local executor) and `OwnerToken` (the opaque isolation key). `OwnerToken` is deliberately a DISTINCT brand from `SessionId`, not an alias: the bash seam is a capability seam that must not know what an owner token *means*, so it never imports `dsh-session`'s vocabulary — the `dsh-tool-bash` consumer is the single boundary that casts the owning agent's `SessionId` into an `OwnerToken`. Branding both stops a raw `string` (or a `BashTaskId` where an `OwnerToken` is expected, or vice versa) from slipping through the type checker on the model-facing `task_id` path. + +## Foreground runs: `BashRunResult` + +The outcome of one completed (or killed) foreground run. Orthogonal outcomes are reported **independently** — a process can both time out AND exit 0 because it trapped the signal — so `timedOut`, `aborted`, `signal`, and `exitCode` are each their own field; a caller never reads a cut-short run as a clean success. + +```ts type-equiv +interface BashRunResult { + /** Exit code; null when the process died from a signal. */ + exitCode: number | null + /** Terminating signal (e.g. 'SIGTERM'); null on normal exit. */ + signal: NodeJS.Signals | null + /** True when the executor's own timeout killed the command. */ + timedOut: boolean + /** True when the caller's AbortSignal killed the command. */ + aborted: boolean + /** The effective timeout applied to this run (after defaulting/capping). */ + timeoutMs: number + stdout: CollectedOutput + stderr: CollectedOutput +} +``` + +Each stream is a `CollectedOutput` — the (possibly truncated) text plus recovery info. When truncated, `text` is the **tail** and the complete stream spills to a private file: + +```ts type-equiv +interface CollectedOutput { + /** Collected text — the TAIL of the stream when truncated. */ + text: string + /** True when bytes were dropped from `text`. */ + truncated: boolean + /** Path to a file holding the COMPLETE stream, when truncated and available. */ + spillPath?: string +} +``` + +## Background tasks: `BashTask` + +A long-running command started with `start()` is tracked as a `BashTask`. `BashTaskStatus` is `'running' | 'completed' | 'killed'`; `done` resolves when the underlying process closes and never rejects. + +```ts type-equiv +interface BashTask { + readonly id: BashTaskId + readonly command: string + status: BashTaskStatus + /** Exit code once finished (null = killed by signal / still running). */ + exitCode: number | null + /** Terminating signal name, when signal-killed. */ + signal: NodeJS.Signals | null + /** Resolves when the underlying process closes (never rejects). */ + readonly done: Promise +} +``` + +`readOutput()` returns an incremental `BashTaskRead` — the output produced since the previous read, with a `lossy` flag when truncation dropped unread bytes: + +```ts type-equiv +interface BashTaskRead { + task: BashTask + /** Output produced since the previous read (stderr in a marked section). */ + delta: string + /** True when truncation dropped unread bytes the delta cannot include. */ + lossy: boolean + /** Full stdout spill file, when stdout truncation occurred and a safe path is available. */ + stdoutSpillPath?: string + /** Full stderr spill file, when stderr truncation occurred and a safe path is available. */ + stderrSpillPath?: string +} +``` + +## The service + +`BashExecutor` (`ctx.bash`, abstract — defined in [`packages/bash/bash/src/index.ts`](../../packages/bash/bash/src/index.ts)) mirrors the `LlmService`/`LlmAdapter` split: `resolve` (request → spec), `run` (foreground), `start` (background), `get`/`ownerOf`/`list`/`readOutput`/`kill`, and `onTaskDone` (a `BashTaskListener` completion callback). Spawned commands get a **scrubbed env** (dropping `*KEY*`/`*SECRET*`/`*TOKEN*`) and spill files use a private 0700 dir with random names and owner-only opens — model output never gets the ambient environment or a predictable path. The implementation that provides all this is `dsh-bash-local`; the model-facing `bash`/`bash_output`/`bash_kill` schemas that call it are in `dsh-tool-bash` (and present as terminals via the [tool-presentation vocabulary](tools.md#tool-presentation-ui-vocabulary)). diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md new file mode 100644 index 0000000000..b50c3483e4 --- /dev/null +++ b/docs/core-data-structures/core.md @@ -0,0 +1,289 @@ +# Core Data Structures + +This folder catalogs the **data structures** of the DeepSeek Harness — what each core type represents, its literal shape, and where the full detail lives. It complements [architecture.md](../architecture.md), which describes *behavior* (the service map, the session/turn/step lifecycle, the event taxonomy); this page describes the *vocabulary* that behavior moves around. + +## What counts as "core" + +The harness is a microkernel: a tiny core plus many plugins. Most types belong to one plugin or one capability. A handful, though, are the **spine** — the language the agent loop and its events traffic in on *every* turn, no matter which optional plugins are loaded. Those are "core". + +Precisely, a data structure is **core** if either: + +1. it flows through the agent-loop spine — the loop holds it, derives it, streams it, or logs it on every turn (a `Message`, a `StreamChunk`, a `SessionEvent`, the `Agent` handle itself), independent of which plugins are present; **or** +2. it is the single headline type a plugin author writes against a pipeline — `ToolDefinition` (what every tool *is*). + +Everything else is documented on a **sub-page**, not here. The rule that draws the line: *the type you write, hold, or receive is core; the machinery that types it, renders it, or persists it is a sub-page detail.* So `ToolDefinition` is core, but the `SchemaSpec`/`InferArgs` DSL that types it, the `ToolCallPresentation` vocabulary that renders it, and the `SessionPersistence` seam that stores the event log are not — they live on the sub-pages below. + +| Sub-page | Owns | +|---|---| +| [llm-streaming.md](llm-streaming.md) | the `StreamChunk` wire protocol + adapter contract, `BlockAssembler`, the `LlmAdapter` seam | +| [session.md](session.md) | the full `SessionEventMap` variant catalog, `TurnTrigger`/`TurnEndReason`, `deriveMessages()`, the turn-enclosure invariant | +| [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` | +| [tools.md](tools.md) | `ToolDefinition` full fields, the schema DSL, `ToolExecution`/`ToolResult`, tool-presentation UI types, the `tools/execute` waterfall | +| [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, background `BashTask`s | + +> Type definitions on this page are pasted **verbatim** from source and drift-checked by `pnpm run verify-type-equiv` (see [development.md](../development.md#documenting-types-verbatim-ts-type-equiv)). Inline JSDoc is omitted for readability; follow the source link for the full contracts. + +## The `…Map → derived-union` pattern + +Almost every extensible sum type in the harness follows one shape: an interface keyed by a discriminant tag (the `…Map`), from which the union is derived with `keyof`. Plugins add variants by **declaration merging** — no edit to the owning package. + +```ts ignore-check +// The pattern, schematically: +interface ThingMap { + 'a': { kind: 'a'; /* … */ } + 'b': { kind: 'b'; /* … */ } +} +type ThingKind = keyof ThingMap // 'a' | 'b' +type Thing = ThingMap[keyof ThingMap] // the discriminated union + +// A plugin extends it without touching the source package: +declare module '@deepseek-ai/dsh-llm' { + interface ThingMap { + 'c': { kind: 'c'; /* … */ } + } +} +``` + +Six canonical maps use this pattern; a plugin author extends these: + +| Map | Package | Derives | Catalog | +|---|---|---|---| +| `ContentBlockMap` | dsh-llm | `ContentBlock` | [below](#content-blocks-and-messages) | +| `MessageSourceMap` | dsh-llm | `MessageSource` | [below](#content-blocks-and-messages) | +| `FinishReasonMap` | dsh-llm | `FinishReason` | [below](#the-model-request-and-result) | +| `TurnTriggerMap` | dsh-session | `TurnTrigger` | [session.md](session.md) | +| `TurnEndReasonMap` | dsh-session | `TurnEndReason` | [session.md](session.md) | +| `SessionEventMap` | dsh-session | `SessionEvent` | [session.md](session.md) | + +Two large discriminated unions are the ones consumers `switch` over most: **`StreamChunk`** (the streaming protocol) and **`SessionEvent`** (the log entry). Per the repo convention, `switch` on the tag — don't chain `if`s — so each arm narrows and a typo'd tag fails to compile. + +## Branded IDs + +IDs that cross package boundaries are **branded** — structurally strings, but non-interchangeable at the type level (an `AgentId` can't be passed where a `CallId` is expected). Construction goes through a per-type factory; comparison, logging, and JSON behave as ordinary strings. + +The `Branded` primitive lives in its own type-only package, [dsh-brand](../../packages/util/brand) (no runtime code, no harness-package dependency), so any package can brand the ids it owns without depending on an unrelated capability package (e.g. dsh-bash brands `BashTaskId`/`OwnerToken` via dsh-brand alone, never pulling in dsh-llm). + +Source: [`packages/util/brand/src/index.ts`](../../packages/util/brand/src/index.ts) + +```ts type-equiv +type Branded = string & { readonly [BRAND]: B } +``` + +The three core IDs: `CallId` (correlates a tool call with its result; dsh-llm), `SessionId` (dsh-session), `AgentId` (dsh-agent). Each is `Branded<'CallId'>` etc. plus a same-named factory function. Capability seams brand their own ids too — see `BashTaskId`/`OwnerToken` in [bash.md](bash.md). + +## Content blocks and messages + +A conversation is `Message`s; a message is an array of typed **content blocks**. The block union derives from `ContentBlockMap`. + +Source: [`packages/llm/llm/src/types.ts`](../../packages/llm/llm/src/types.ts) + +```ts type-equiv +interface ContentBlockMap { + 'text': TextBlock + 'reasoning': ReasoningBlock + 'tool-call': ToolCallBlock + 'tool-result': ToolResultBlock + 'image': ImageBlock +} +``` + +The block interfaces (full fields in source): `TextBlock` (`text`), `ReasoningBlock` (thinking, distinct from visible text), `ToolCallBlock` (`id: CallId`, `name`, raw-JSON `arguments`), `ToolResultBlock` (`toolCallId`, nested `content: ContentBlock[]`, `isError?`), `ImageBlock` (`url`, `mimeType?`). `ContentBlock = ContentBlockMap[ContentBlockType]`. + +A `Message` is a role plus blocks: + +```ts type-equiv +interface Message { + role: 'system' | 'user' | 'assistant' + content: ContentBlock[] +} +``` + +Where a message came from is itself a merge-extensible sum type: + +```ts type-equiv +interface MessageSourceMap { + user: { kind: 'user' } + plugin: { kind: 'plugin'; plugin: string } + agent: { kind: 'agent'; agentId: string } +} +``` + +## Streaming + +Adapters emit a raw **chunk** protocol; the loop logs the chunks (replay fidelity) while feeding the same chunks through a `BlockAssembler` to rebuild blocks and messages. `StreamChunk` is a closed discriminated union over `type` — `block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`. + +The full union, the adapter contract (usage-before-finish, raw-JSON tool arguments, the two sanctioned error paths), and `BlockAssembler` live on **[llm-streaming.md](llm-streaming.md)**. + +## The model request + +One model call is a fully-assembled `GenerateOptions`. The adapter answers with a raw `StreamChunk` stream; the consumer assembles it with `BlockAssembler` (see [llm-streaming.md](llm-streaming.md)). + +Source: [`packages/llm/llm/src/types.ts`](../../packages/llm/llm/src/types.ts) + +```ts type-equiv +interface GenerateOptions { + model: string + messages: Message[] + /** System prompt text (adapters map to the provider's system slot). */ + system?: string + /** Tool schemas (adapters map to the provider's `tools` field). */ + tools?: ToolSchema[] + /** Assistant prefix continuation (prefill). */ + prefill?: ContentBlock[] + temperature?: number + maxTokens?: number + /** + * Stop sequences: generation halts as soon as the model produces any one of + * these strings (adapters map to the provider's stop field, e.g. OpenAI + * `stop`). The stop string itself is not included in the output. + */ + stop?: string[] + signal?: AbortSignal +} +``` + +Why a model response stopped is a merge-extensible reason: + +```ts type-equiv +interface FinishReasonMap { + 'stop': { kind: 'stop' } + 'tool-calls': { kind: 'tool-calls' } + 'max-tokens': { kind: 'max-tokens' } + 'aborted': { kind: 'aborted' } + 'error': { kind: 'error'; message: string; code?: string } +} +``` + +`FinishReason = FinishReasonMap[keyof FinishReasonMap]`. `TokenUsage` (per-call accounting with disjoint cache fields) is detailed on [llm-streaming.md](llm-streaming.md). + +`GenerateOptions.tools` carries `ToolSchema` — the JSON-schema description of a tool, as sent to the model. It is declared in dsh-llm (not dsh-tools) precisely because it is part of the request the loop assembles every step: + +```ts type-equiv +interface ToolSchema { + name: string + description: string + /** JSON Schema object for the arguments. */ + parameters: Record + strict?: boolean +} +``` + +The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition` that produces it (schema + `execute`) is on [tools.md](tools.md). + +## Sessions + +A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`), not stored separately. The event vocabulary derives from `SessionEventMap`: + +Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/types.ts) + +```ts type-equiv +type SessionEvent = { + [K in SessionEventType]: { + type: K + /** Monotonic sequence number within the session. */ + seq: number + /** Unix epoch milliseconds. */ + time: number + data: SessionEventMap[K] + } +}[T] +``` + +The eleven event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**. + +## The agent handle + +`Agent` is the surface every plugin (UI, hooks, orchestrators) programs against. The concrete implementation is `ReactLoopAgent` in dsh-agent-loop; nothing outside the loop depends on the implementation. + +Source: [`packages/core/agent/src/types.ts`](../../packages/core/agent/src/types.ts) + +```ts type-equiv +interface Agent { + readonly id: AgentId + readonly options: AgentOptions + readonly session: Session + readonly status: AgentStatus + + /** Queue a user message. Starts a turn when idle; otherwise waits for the next turn. */ + send(content: ContentBlock[], options?: SendOptions): void + + /** + * Steer a running turn: content is injected between steps of the current + * turn. When idle, behaves like {@link send}. + */ + steer(content: ContentBlock[], options?: SendOptions): void + + /** + * Inject in-session context (file-change notices, skill content, cron + * notifications, …): appends a `context/message` session event the next model + * request sees at its chronological position, rendered as tagged synthetic + * context rather than a user prompt. Does not run the model. + * + * Turn-enclosure (the turn-enclosure RFC): an inject while a turn is open joins that turn; + * an inject while idle wraps its `context/message` in a one-shot `injection` + * turn (`turn/start` → `context/message` → `turn/end`) and checkpoints it for + * durability, so every event stays inside a turn and a persistence backend + * never loses a between-turn notice. The idle checkpoint is fire-and-forget + * (inject is synchronous): a failing flush is reported via `agent/error` + * (step `0`) and the logger, never thrown into the caller. + * + * Live-adapter review has validated the tagged-envelope rendering against + * current DeepSeek behavior; provider-specific mismatches belong in that + * adapter, not in the canonical session vocabulary. + */ + inject(content: ContentBlock[], options?: SendOptions): void + + /** + * Cancel ALL pending work for the agent. `cancel()`: + * + * - clears the queued FIFO (un-started prompts never run) and the steering + * FIFO (steering for the cancelled turn is dropped, not re-enqueued); + * - aborts the in-flight step if one is running (the turn ends `aborted`); + * - drops a turn that is about to start (a `cancel()` landing in the + * pre-step window — after a `send()` queued but before the loop flips to + * `running`, or after `running` is emitted but before the first step) so + * that queued prompt does not run and cannot be batched into the cancelled + * turn. + * + * After `cancel()`, `whenIdle()` resolves on the post-cancel quiescent state. + * `cancel()` on an idle agent with nothing queued or running is a safe no-op + * — it does NOT arm anything that would drop a later legitimate prompt. + */ + cancel(reason?: string): void + + /** + * Resolve once the agent has reached quiescence after settling out of + * `running`, or immediately if it is already idle with no queued work. A + * non-owner's quiescence-observation hook: a consumer that does NOT own the + * agent's lifecycle awaits this to proceed only after queued/running work has + * fully stopped, rather than returning while the driver is still streaming or + * about to start a queued turn — without itself tearing the agent down. (A + * lifecycle OWNER does not need it: `AgentHandle.dispose()` already awaits the + * loop-exit promise directly as part of stopping and unregistering. So this is + * for a non-owning observer — e.g. a test awaiting a turn to settle, or a + * monitor — that wants the settle signal but must not dispose the agent.) + * + * "Quiescence", not merely "status changed": a disposed agent emits + * `agent/status('disposed')` from inside its disposer, BEFORE the driver loop + * has unwound — so `whenIdle()` resolving on `disposed` must wait for the loop + * to actually exit (the implementation chains the loop-exit promise), not just + * observe the status flip. A mid-step disposal that never reaches `idle` still + * unblocks the await this way. + */ + whenIdle(): Promise + + // TODO(sub-agents): spawn/fork seams — semantics deliberately deferred. + // The intended shape: a creation option referencing a parent agent + // (fork = seed the child Session with the parent's event log; spawn = + // fresh Session), with the child returned as an Agent handle so steer() + // and event subscription work uniformly. See docs/architecture.md. +} +``` + +`AgentStatus` is `'idle' | 'running' | 'disposed'`. `AgentId` is a branded string. `AgentOptions` (`model?`, `systemPrompt?`) is merge-extensible — plugins add creation options by declaration merging. The `agent/*` event taxonomy (lifecycle, turn/step boundaries, the `agent/request`/`agent/step-result`/`agent/turn-continuation` waterfalls) is in [architecture.md § Event taxonomy](../architecture.md#event-taxonomy). + +## `ToolDefinition` + +The one pipeline-authoring type that is core: what every registered tool *is* — a model-facing `ToolSchema` plus an `execute` function and optional UI presenters. A tool author rarely constructs it by hand (the `defineTool` DSL builds it with typed args), but it is the contract the registry holds and the loop dispatches through. + +Its full fields, the `defineTool`/`SchemaSpec`/`InferArgs` typed schema DSL, the `ToolExecution`/`ToolExecutionResult` waterfall shapes, and the tool-presentation UI vocabulary are on **[tools.md](tools.md)**. diff --git a/docs/core-data-structures/llm-streaming.md b/docs/core-data-structures/llm-streaming.md new file mode 100644 index 0000000000..7439eb14a6 --- /dev/null +++ b/docs/core-data-structures/llm-streaming.md @@ -0,0 +1,66 @@ +# LLM Streaming + +The wire-level streaming vocabulary of [dsh-llm](../../packages/llm/llm). [core.md](core.md) introduces `StreamChunk`, `Message`, and `ContentBlock`; this page owns the full chunk protocol, the adapter contract every adapter must obey, and the shared assembler. + +Source: [`packages/llm/llm/src/types.ts`](../../packages/llm/llm/src/types.ts) + +## `StreamChunk` — the raw protocol + +A streaming response interleaves several typed blocks (text, reasoning, multiple tool calls). `index` ties each delta to its block; `block-end` carries the fully-assembled `ContentBlock` so consumers don't have to re-assemble deltas themselves. It is a **closed** discriminated union — a `switch` over `type` ends with `assertNever`, so adding a variant breaks compilation at every consumer that must handle it. + +```ts type-equiv +type StreamChunk = + | { type: 'block-start'; index: number; blockType: ContentBlockType } + | { type: 'text-delta'; index: number; text: string } + | { type: 'reasoning-delta'; index: number; text: string } + | { type: 'tool-call-delta'; index: number; id: CallId; name?: string; argumentsDelta: string } + | { type: 'block-end'; index: number; block: ContentBlock } + | { type: 'usage'; usage: TokenUsage } + | { type: 'finish'; reason: FinishReason } +``` + +## The adapter contract + +Every adapter MUST obey these, and every consumer may rely on them: + +- **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering. +- **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`. +- **Two sanctioned error paths.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted'}` (provider in-band errors, for adapters that can't throw mid-stream). Consumers must handle *both*. The agent loop translates a finish-error/aborted into a turn error — it never logs a normal completed assistant message for a failed step. + +This contract is why two adapters exist as a deliberate pair: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (the same endpoint through `@earendil-works/pi-ai`). Two independent internals over one contract is what pinned the protocol down — the library-backed adapter can't throw mid-stream, so it exercises the finish-chunk error path the hand-rolled one might not. + +## `TokenUsage` + +Per-call token accounting. Counts are **disjoint**: `inputTokens` is uncached input only; cached input is reported separately, and billed input is the sum of the three. Adapters whose providers fold cache hits into a single prompt total (DeepSeek's `prompt_tokens`) subtract them back out. + +```ts type-equiv +interface TokenUsage { + inputTokens: number + outputTokens: number + cacheReadTokens?: number + cacheWriteTokens?: number + reasoningTokens?: number +} +``` + +## `BlockAssembler` + +`BlockAssembler` ([`packages/llm/llm/src/assembler.ts`](../../packages/llm/llm/src/assembler.ts)) is the single shared implementation that folds a `StreamChunk` stream back into `ContentBlock`s and a final `Message`. The loop logs the raw chunks (for replay fidelity) while feeding the same chunks through an assembler — so the canonical log keeps token-level detail and the derived message is rebuilt deterministically. A consumer that needs the assembled result without re-implementing the fold uses this. + +## The seam + +`LlmAdapter` is the provider seam: subclass, implement `stream()`, register with `ctx.llm.registerAdapter(models, adapter)`. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § The vocabulary](../architecture.md#the-vocabulary-dsh-llm). + +`ContentBlockType` (the key set the `index`-correlated blocks carry) derives from `ContentBlockMap`: + +```ts type-equiv +interface ContentBlockMap { + 'text': TextBlock + 'reasoning': ReasoningBlock + 'tool-call': ToolCallBlock + 'tool-result': ToolResultBlock + 'image': ImageBlock +} +``` + +See [core.md § Content blocks and messages](core.md#content-blocks-and-messages) for the block interfaces. diff --git a/docs/core-data-structures/persistence.md b/docs/core-data-structures/persistence.md new file mode 100644 index 0000000000..45c1d8dd6b --- /dev/null +++ b/docs/core-data-structures/persistence.md @@ -0,0 +1,67 @@ +# Session Persistence + +The **durability seam** for the event log. [session.md](session.md) describes the in-memory `Session` — the append-only `SessionEvent` log that is the source of truth. This page describes how that log is made durable: the abstract `SessionPersistence` service, its backends, the flush checkpoint, crash recovery, and the metadata header that travels alongside the log. + +The seam is a textbook [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md): one abstract service ([dsh-session-persistence](../../packages/session-persistence/session-persistence), `ctx.sessionPersistence`) defining create/append/load/list over the existing `SessionEvent` — **no parallel persisted type** — and two interchangeable backends that pass the same `runPersistenceContract` suite. See the [session-persistence RFC](../rfc/implemented/architecture/2026-06-14-session-persistence.md). + +## The flush checkpoint + +`session/event` is a *synchronous* notification; persistence plugins buffer it (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. Flush is `ctx.parallel` (awaited): a turn's events are durably committed before the next turn starts, and the turn boundary is the commit boundary. A rejecting flush is reported via `agent/error` and the logger — never as a session event (it would land past the commit boundary), so the backend keeps its buffered events for the next flush. + +## Crash recovery preserves an interrupted turn + +A backend that reloads a log crashed mid-turn finds an open `turn/start` with no `turn/end`. It does **not** truncate — a single turn can be huge in a long-horizon task (many steps, large tool output), and those events were durably appended before the crash. Instead it closes the orphaned turn with a synthetic `turn/end { reason: { kind: 'interrupted' } }`, keeping the log balanced and the turn-enclosure invariant intact. `interrupted` is the one `TurnEndReason` no loop emits (see [session.md](session.md#why-a-turn-ended-turnendreasonmap)). + +## `SessionHeader` — metadata beside the log + +Per-session metadata travels **separately** from the event log: format version, cwd, and lineage are storage concerns, not conversation events, so they stay out of `SessionEventMap` and never reach `deriveMessages()`. The header is attached to a `Session` via `session.header`. + +Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/types.ts) + +```ts type-equiv +interface SessionHeader { + /** + * On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the + * session is created. A persistence backend rejects any other version on load + * (no migration — see the constant). + */ + version: number + /** The session's id (mirrors the {@link Session}'s id). */ + id: SessionId + /** Unix epoch milliseconds when the session was created. */ + createdAt: number + /** Absolute working directory the session was created in (if any). */ + cwd?: string + /** The session this one was forked from (seed lineage), if any. */ + parentSession?: SessionId +} +``` + +## `CreateSessionOptions` — seeding and metadata + +Creating a `Session` through the store takes a `seed` (replay/fork an existing event log) and `meta` (the storage-level fields the store folds into a `SessionHeader`). The store fills in `version`/`id` and defaults `createdAt`; the caller supplies the validated absolute `cwd`, the `parentSession` lineage, and — only when reconstructing a persisted session — the original `createdAt` to preserve it. + +```ts type-equiv +interface CreateSessionOptions { + /** Events to seed the new session with (replay/fork). */ + seed?: SessionEvent[] + /** + * Creation metadata. The store fills in `version`/`id` and defaults + * `createdAt` to now; the caller supplies the storage-level fields (validated + * absolute `cwd`, `parentSession` lineage, and — when reconstructing a + * persisted session — the original `createdAt` to preserve it). + */ + meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number } +} +``` + +Replay/fork is therefore `ctx.sessions.create(id, { seed: seedEvents })`; resuming a *persisted* session into a live agent is `ctx.agents.resume({ resumeSessionId })`. + +## The backends + +Both implement the same abstract `SessionPersistence` (create/append/load/list over `SessionEvent`) and pass `runPersistenceContract`, proving the seam is genuinely backend-agnostic: + +- **[dsh-session-persistence-jsonl](../../packages/session-persistence/session-persistence-jsonl)** — an append-only JSONL log per session with crash-safe atomic writes, the interrupted-turn crash recovery above, and a read/replay path. +- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data)` maps 1:1 onto the event, so there is no parallel persisted schema to keep in sync. + +Multiple backends sharing one on-disk session coordinate writes through the [shared persistence write-coordinator](../rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md). diff --git a/docs/core-data-structures/session.md b/docs/core-data-structures/session.md new file mode 100644 index 0000000000..d73ad0cb4e --- /dev/null +++ b/docs/core-data-structures/session.md @@ -0,0 +1,128 @@ +# Sessions + +The in-memory, event-sourced model of [dsh-session](../../packages/core/session). A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth for an agent's whole interaction history. The LLM message history is *derived* from the log, never stored separately; replay is re-derivation from the same events. How the log is made **durable** (the persistence seam, backends, crash recovery) is the sibling concern on [persistence.md](persistence.md). + +Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/types.ts) + +## `SessionEventMap` — the event vocabulary + +The append-only event types. Merge-extensible: a plugin (e.g. compaction) declares extra event types via declaration merging. + +```ts type-equiv +interface SessionEventMap { + 'turn/start': { turn: number; trigger: TurnTrigger } + 'turn/end': { turn: number; reason: TurnEndReason } + 'step/start': { turn: number; step: number } + 'step/end': { turn: number; step: number } + /** A user-visible prompt (queued message drained at turn start). */ + 'user/message': { content: ContentBlock[]; source: MessageSource } + /** + * In-session context injection (file-change notices, subdir AGENTS.md, + * skill content, cron notifications, …). Rendered into the derived history + * as tagged synthetic context — NOT a user prompt. + */ + 'context/message': { content: ContentBlock[]; source: MessageSource } + /** Raw stream chunk — token-level replay fidelity. */ + 'assistant/chunk': { turn: number; step: number; chunk: StreamChunk } + /** + * Assembled assistant message for one step (derived history uses this). + * Carries the step's `usage` when the adapter reported token accounting, so + * the model output and its accounting travel together (there is no separate + * usage record). `usage` is absent when the adapter reported none. + */ + 'assistant/message': { turn: number; step: number; content: ContentBlock[]; usage?: TokenUsage } + 'tool/call': { turn: number; step: number; callId: CallId; name: string; arguments: string } + 'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string } } + /** Steering content injected between steps of a running turn. */ + 'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource } +} +``` + +## `SessionEvent` — one log entry + +A proper discriminated union over `type` (not independent `type`/`data` unions), so `switch (event.type)` narrows `event.data` without casts. `seq` is the monotonic position in the log (`seq = log.length`); `time` is epoch ms. + +```ts type-equiv +type SessionEvent = { + [K in SessionEventType]: { + type: K + /** Monotonic sequence number within the session. */ + seq: number + /** Unix epoch milliseconds. */ + time: number + data: SessionEventMap[K] + } +}[T] +``` + +`SessionEventType = keyof SessionEventMap`. Because `SessionEventMap` is merge-extensible, switches over `SessionEvent` must NOT use `assertNever` — a plugin-added variant is a valid unknown value; handle the known cases and fall through `default`. + +## Derived history: `deriveMessages()` + +`Session.deriveMessages()` projects the event log into the `Message[]` the model sees. The projection rules: + +- `user/message` → a user message. +- `assistant/message` → an assistant message. Raw `assistant/chunk` events are replay/UI data and are **skipped** in derivation (the assembled message is authoritative). An **empty-content** `assistant/message` is also skipped — a max-tokens step cut off with no content still records an `assistant/message` to host its `usage`, but a content-less assistant turn must not enter the provider transcript. +- `tool/result` → a user message carrying a `tool-result` block. +- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`…`) at their chronological position — the "system-reminder" pattern; the model distinguishes them from real prompts by the envelope. + +Everything else (`turn/*`, `step/*`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`. + +## What started a turn: `TurnTriggerMap` + +```ts type-equiv +interface TurnTriggerMap { + message: { kind: 'message'; source: MessageSource } + continuation: { kind: 'continuation' } + /** + * An out-of-band context injection (`agent.inject()`) made while the agent + * was idle. The loop wraps the injected `context/message` in a one-shot turn + * (`turn/start` → `context/message` → `turn/end`) so every event in the log + * stays turn-enclosed — the durability/replay boundary is the turn, and a + * bare event between turns would otherwise be indistinguishable from a crash + * tail on reload. + */ + injection: { kind: 'injection'; source: MessageSource } +} +``` + +## Why a turn ended: `TurnEndReasonMap` + +```ts type-equiv +interface TurnEndReasonMap { + completed: { kind: 'completed' } + aborted: { kind: 'aborted'; reason?: string } + /** + * The turn failed: a step threw or the model reported a failure. `step` is the + * step number the failure occurred on (the operational error's location — the + * single durable record of an in-turn failure; live diagnostics also fire via + * `agent/error`). `code` is the error's code when one was attached. + */ + error: { kind: 'error'; step: number; message: string; code?: string } + disposed: { kind: 'disposed' } + 'max-tokens': { kind: 'max-tokens' } + /** + * The turn never ended on its own: the process crashed mid-turn and a + * persistence backend later closed the orphaned (open) turn on reload so the + * log stays balanced. SYNTHESIZED by the backend's crash-recovery repair — no + * loop ever emits this. Its events are real (they were durably appended before + * the crash) and are PRESERVED, not discarded: a single turn can be huge in a + * long-horizon task (many steps, large tool output), so truncating it would + * lose real work. The marker records that the turn was cut short, not that the + * model completed it. See the session-persistence RFC. + */ + interrupted: { kind: 'interrupted' } +} +``` + +`max-tokens` mirrors the model-call `FinishReason` of the same name: any `max-tokens` step in a turn makes the whole turn end `max-tokens` (the cut-short fact wins over a later continuation), so a consumer can tell a clean stop from a truncated one. `interrupted` is the one reason no loop emits — it is synthesized by crash recovery (see [persistence.md](persistence.md)). Both maps are merge-extensible. + +## The turn-enclosure invariant + +Every session event lives **inside** a turn (between a `turn/start` and its `turn/end`). The loop appends queued `user/message` events *after* `turn/start`, and an idle `agent.inject()` wraps its `context/message` in a one-shot `injection` turn. This makes the turn the single durability/replay boundary: a backend can treat anything after the last `turn/end` as an interrupted-crash tail without risking the loss of legitimately-recorded between-turn context. The `dsh-invariants` plugin enforces it in dev (a message event outside an open turn throws). See [the turn-enclosure invariant RFC](../rfc/implemented/architecture/2026-06-15-turn-enclosure-invariant.md). + +## Durability contract + +What a persistence backend relies on: the durable log persists every event verbatim, **including** `assistant/chunk` — `seq` must stay contiguous, so chunks cannot be filtered out of the canonical log. All `event.data` must be JSON-serializable; `Session.append` enforces this at the source (throwing on non-serializable data), so a bad event never enters the log and `session.events` always equals what a backend can persist. Adding an event type that carries non-serializable data, or that breaks the turn/step nesting the invariants plugin checks, is a breaking change to the on-disk format. + +The backends that consume this contract are on [persistence.md](persistence.md). diff --git a/docs/core-data-structures/tools.md b/docs/core-data-structures/tools.md new file mode 100644 index 0000000000..f255fdbc32 --- /dev/null +++ b/docs/core-data-structures/tools.md @@ -0,0 +1,112 @@ +# Tools + +The tool pipeline of [dsh-tools](../../packages/core/tools). [core.md](core.md) introduces `ToolDefinition` as the one pipeline-authoring type promoted to the spine and `ToolSchema` as the model-facing wire shape. This page owns the full `ToolDefinition`, the typed schema DSL that builds it, the waterfall execution shapes, and the UI-presentation vocabulary. + +Source: [`packages/core/tools/src/index.ts`](../../packages/core/tools/src/index.ts) · [`packages/core/tools/src/schema.ts`](../../packages/core/tools/src/schema.ts) + +## `ToolDefinition` — a registered tool + +A `ToolSchema` (the model-facing fields) plus the `execute` function and optional UI presenters. The registry holds these; the loop dispatches calls through them. The registry's `schemas()` builds the model-facing `ToolSchema[]` by an explicit allowlist — `execute`/`presentCall`/`presentResult` must never leak into a model request. + +```ts type-equiv +interface ToolDefinition extends ToolSchema { + execute(args: unknown, exec: ToolExecution): Promise + /** + * Optional: how to present the PENDING state of one call in a UI, derived + * from the call's `args` (parsed arguments, `unknown` — the tool validates/ + * narrows its own input). Returning `undefined` (or omitting the method) tells + * a UI to fall back to a generic presentation (title = tool name, raw args as + * input). Pure and side-effect-free: a UI may call it during live streaming + * AND a session-log replay, so it must depend only on `args`. + */ + presentCall?(args: unknown): ToolCallPresentation | undefined + /** + * Optional: how to present the COMPLETED state, given the same `args` and the + * `result` (`execute`'s content + whether it errored). Returning `undefined` + * (or omitting the method) tells a UI to keep the pending title and render the + * raw result content. Pure and side-effect-free for the same replay reason. + */ + presentResult?(args: unknown, result: ToolResult): ToolResultPresentation | undefined +} +``` + +`execute` receives `args: unknown` — a raw `ToolDefinition` validates its own input. First-party tools don't write that by hand; they use `defineTool`, which validates and narrows for them. + +## The typed schema DSL + +Plugin authors write per-property specs with a boolean `required: true`, and a type-level helper maps the spec to the `execute` argument type — zero casts. The DSL is *machinery that types* `ToolDefinition`; it is intentionally a sub-page detail, not core. + +Source: [`packages/core/tools/src/schema.ts`](../../packages/core/tools/src/schema.ts) + +```ts type-equiv +interface SchemaProp { + type: SchemaType + /** Per-property required flag (NOT the JSON Schema top-level required array). */ + required?: true + /** Human-readable description, surfaced in the JSON Schema as well. */ + description?: string + /** Enum of allowed values (strings only). */ + enum?: string[] + /** Default value. */ + default?: unknown + /** Nested properties for type: 'object'. */ + properties?: SchemaSpec + /** Items schema for type: 'array'. */ + items?: SchemaProp +} +``` + +```ts type-equiv +type SchemaSpec = Record +``` + +`SchemaType` is the primitive union `'string' | 'number' | 'boolean' | 'object' | 'array'`. `InferArgs` maps a `SchemaSpec` to the TS argument type — `required: true` props become required keys, everything else genuinely optional: + +```ts type-equiv +type InferArgs = Simplify< + & { [K in RequiredKeys]: InferPropValue } + & { [K in Exclude>]?: InferPropValue } +> +``` + +`defineTool({ name, description, parameters, execute, … })` ties it together: `parameters` is a `SchemaSpec`, `execute(args, exec)` gets `args: InferArgs`, and the helper converts the spec to JSON Schema (`schemaSpecToJsonSchema`) for the wire and validates model-generated args (`validateArgs`) before the typed body runs. A mismatch throws `ToolArgsError` (`code: 'INVALID_ARGS'`), which the registry turns into an `isError` result so the model can self-correct. Why a custom DSL and not schemastery: tool parameters need JSON Schema (the LLM wire format), not validation/transformation — the lightweight DSL gives the best authoring DX with the smallest surface. + +## Execution: the `tools/execute` waterfall shapes + +`ctx.tools.execute()` runs each call through the `tools/execute` waterfall — the single seam where sandbox, permission, hook, and plan-mode plugins wrap or veto. The pending call is a `ToolExecution`; the outcome is a `ToolExecutionResult`. + +```ts type-equiv +interface ToolExecution { + callId: CallId + name: string + /** Parsed JSON arguments (unknown — tools validate their own input). */ + arguments: unknown + /** The agent on whose behalf the call runs (set by the agent loop). */ + agent?: Agent + signal?: AbortSignal +} +``` + +```ts type-equiv +interface ToolExecutionResult { + callId: CallId + content: ContentBlock[] + isError: boolean + /** + * Set when the call failed with a {@link HarnessError}: machine-routable + * `{ name, code }` for retry/sandbox plugins and replay. The model-facing + * text in `content` is always present; this is extra structure for code. + */ + error?: ToolErrorInfo +} +``` + +A waterfall listener receives `(exec, next)`: call `next()` to proceed (possibly around your own logic), or return a `ToolExecutionResult` without calling `next()` to veto. An unregistered tool routes through the same catch as a tool-thrown error, so both failure classes get a structured `{ name, code }` (`ToolNotFoundError` → `UNKNOWN_TOOL`) — the loop records a failed tool call instead of failing the whole turn. + +## Tool-presentation UI vocabulary + +How a tool wants its call shown in a UI (an editor tool-call card, a CLI log line), provider-neutral so a tool describes itself without depending on any client protocol. `presentCall` returns a `ToolCallPresentation` (pending state: `title`, `kind`, `rawInput`, `content`, optional `terminal`); `presentResult` returns a `ToolResultPresentation` (completed state: replacement `title`, reformatted `content`, terminal `output`/exit). `ToolCallKind` (`'read' | 'edit' | 'delete' | 'move' | 'search' | 'execute' | 'fetch' | 'other'`) picks an icon. A `ToolTerminal` asks a capable UI to render the call as a terminal card (cwd header, output, exit-status pill). + +> These shapes carry a `FIXME(tool-presentation)` in source: they grew incrementally and the call-vs-result terminal split is muddy. Before more tools/UIs depend on them, they will be redesigned (a tagged union over card kinds) and pinned in an RFC, migrating `dsh-tool-bash` and the ACP bridge together. Treat the field-level shapes here as provisional; the source is authoritative. + +The full presentation field docs live in [`packages/core/tools/src/index.ts`](../../packages/core/tools/src/index.ts). The bash tool's own schemas (`bash`/`bash_output`/`bash_kill`) and the executor they drive are on [bash.md](bash.md). diff --git a/docs/development.md b/docs/development.md index 45d23912ca..99f67e671d 100644 --- a/docs/development.md +++ b/docs/development.md @@ -31,7 +31,7 @@ Run typecheck once after a fresh clone: pnpm run typecheck ``` -That first typecheck builds declaration output used by type-aware linting for vendored packages. Without it, `pnpm run lint` can report unresolved-type `no-unsafe-*` errors even when source code is fine. +That first typecheck runs the package/vendor build graph and the root no-emit `tsconfig.json` graph for examples, tests, and scripts. The root graph uses the same source `paths` map but relies on project references so vendored code is checked under its own tsconfig settings. If you are preparing to push from a fresh clone or worktree, also build once: @@ -39,7 +39,7 @@ If you are preparing to push from a fresh clone or worktree, also build once: pnpm run build ``` -`pnpm run hygiene` includes `publint`, which validates package entrypoints against the built `lib/*.js` files. A fresh worktree has no bundled JS until `pnpm run build` runs. +`pnpm run hygiene` includes `publint`, which validates package entrypoints against the built `lib/*.js` files, and `verify-node-next-types`, which validates built declarations against a temporary NodeNext consumer. A fresh worktree has no bundled JS or declarations until `pnpm run build` runs. ## Environment variables @@ -57,7 +57,7 @@ DEEPSEEK_BASE_URL=https://... # optional lefthook is configured in `lefthook.yml` as an early local checkpoint before review: - `pre-commit` runs staged-file ESLint fixes, `pnpm run typecheck`, and the vendor manifest guard. -- `pre-push` runs `pnpm run test`, `pnpm run hygiene`, `pnpm run doc-sync`, and `pnpm run verify-module-graph`. +- `pre-push` runs `pnpm run test`, `pnpm run test:snapshot`, `pnpm run hygiene`, `pnpm run doc-sync`, and `pnpm run verify-module-graph`. The vendor manifest guard checks that changes under `vendor/*/src` are staged with the matching `vendor/README.md` manifest update. See `vendor/README.md` before editing vendored code. @@ -74,11 +74,12 @@ The GitHub workflow runs these gates on each pull request: - `pnpm run doc-sync` - `pnpm run verify-module-graph` - `pnpm run test:coverage` +- `pnpm run test:snapshot` - `pnpm run build` -- `pnpm run knip && pnpm run publint` +- `pnpm run hygiene` - an echo-agent smoke test that checks the demo's tool call, tool result, and JSONL output -`pnpm run hygiene` is the local shorthand for `pnpm run knip && pnpm run publint && pnpm run constraints`; CI splits `pnpm run constraints` into its own earlier step, then runs `pnpm run knip && pnpm run publint` after `pnpm run build`. +`pnpm run hygiene` is the local shorthand for `pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types`; CI also runs `pnpm run constraints` as an earlier fail-fast step, then runs the full hygiene script after `pnpm run build`. ## Daily commands @@ -88,20 +89,23 @@ Use these from the repo root: pnpm run test # unit tests pnpm run test:coverage # unit tests with per-file coverage gates pnpm run test:e2e # real-API tests; self-skips without DEEPSEEK_API_KEY -pnpm run typecheck # build declarations, then typecheck source, tests, and examples +pnpm run typecheck # build package/vendor outputs, then typecheck examples, tests, and scripts pnpm run lint # eslint . pnpm run lint:fix # eslint . --fix pnpm run doc-typecheck # compile checked TypeScript snippets in Markdown docs -pnpm run verify-event-taxonomy # compare docs/architecture.md event names with source +pnpm run gen-cordis-catalog # regenerate docs/cordis-catalog/events-and-services.md from source +pnpm run verify-cordis-catalog # fail if the cordis events/services catalog is stale pnpm run verify-md-wrap # fail on hard-wrapped prose paragraphs in docs/README markdown -pnpm run doc-sync # doc-typecheck, event taxonomy, and markdown wrap verification +pnpm run verify-type-equiv # fail if a ```ts type-equiv doc block drifts from its source type +pnpm run doc-sync # doc-typecheck, cordis-catalog freshness, markdown wrap/link, and type-equiv verification pnpm run gen-module-graph # regenerate docs/module-graph.md from package peerDeps pnpm run verify-module-graph # fail if docs/module-graph.md is stale -pnpm run build # build declarations and JS bundles -pnpm run hygiene # knip, publint, and workspace constraints +pnpm run build # emit lib/types intermediates, then bundle lib/index.* runtime files +pnpm run verify-node-next-types # fail if built declarations are not NodeNext-consumable +pnpm run hygiene # knip, publint, workspace constraints, and NodeNext declaration check ``` -When changing package public behavior, update the relevant README or JSDoc in the same change. `pnpm run doc-sync` catches checked TypeScript snippets, event-taxonomy drift, and hard-wrapped markdown prose, but broader prose/API sync still needs review. +When changing package public behavior, update the relevant README or JSDoc in the same change. `pnpm run doc-sync` catches checked TypeScript snippets, cordis events/services catalog drift, and hard-wrapped markdown prose, but broader prose/API sync still needs review. ## Demos @@ -127,6 +131,16 @@ Use one of three comment tags to flag known issues in the code, ordered by urgen Pick the tag that matches the urgency so anyone scanning the code can tell a release blocker from a someday-maybe. +## Documenting types verbatim (`ts type-equiv`) + +The [core data structures](core-data-structures/core.md) docs paste real type definitions so a reader sees the exact shape. To keep a paste from drifting when source changes, fence it as ` ```ts type-equiv ` (instead of ` ```ts `) and register it in `scripts/type-equiv.manifest.json` with the source file and symbol it mirrors: + +```json +{ "doc": "docs/core-data-structures/session.md", "symbol": "SessionEvent", "source": "packages/core/session/src/types.ts" } +``` + +`pnpm run verify-type-equiv` (part of `doc-sync`) then extracts that symbol's declaration from source via the TypeScript parser and asserts the block matches it (whitespace- and comment-insensitive, so a doc block may show a clean definition and the prose can carry the semantics). It also enforces a 1:1 correspondence: every `ts type-equiv` block has exactly one manifest entry and vice-versa, so a block can't go silently unchecked and a stale entry can't linger. `doc-typecheck` skips `ts type-equiv` blocks (they aren't standalone-compilable) and excludes them from its opt-out ratio. When you change a documented type, the gate fails until you update the paste; when you add or remove a block, update the manifest in the same change. + ## Architecture context Read `docs/architecture.md` before changing anything under `packages/`. The codebase is built around Cordis plugins, event-sourced sessions, typed service seams, and explicit extension points. diff --git a/docs/module-graph.md b/docs/module-graph.md index 5110178e0e..9efe6e9669 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -7,13 +7,19 @@ Inter-package dependencies among the `@deepseek-ai/dsh-*` harness packages, deri ```mermaid graph TD + bash --> brand + llm --> brand bash-local --> bash llm-deepseek --> llm llm-pi-ai --> llm + session --> brand session --> llm system-prompt --> llm + agent --> brand agent --> llm agent --> session + llm-replay --> llm + llm-replay --> session session-persistence --> session invariants --> agent invariants --> llm @@ -25,6 +31,9 @@ graph TD tools --> agent tools --> llm tools --> system-prompt + ui-stdio --> agent + ui-stdio --> llm + ui-stdio --> session acp --> agent acp --> llm acp --> session @@ -40,23 +49,45 @@ graph TD tool-bash --> bash tool-bash --> llm tool-bash --> tools + agent-core --> agent + agent-core --> agent-loop + agent-core --> invariants + agent-core --> llm + agent-core --> session + agent-core --> system-prompt + agent-core --> tool-bash + agent-core --> tools + acp-agent --> acp + acp-agent --> agent-core + acp-agent --> session-persistence-jsonl + stdio-agent --> agent + stdio-agent --> agent-core + stdio-agent --> session + stdio-agent --> session-persistence-jsonl + stdio-agent --> ui-stdio ``` | Package | Depends on | | --- | --- | -| `bash` | — | -| `llm` | — | +| `brand` | — | +| `bash` | `brand` | +| `llm` | `brand` | | `bash-local` | `bash` | | `llm-deepseek` | `llm` | | `llm-pi-ai` | `llm` | -| `session` | `llm` | +| `session` | `brand`, `llm` | | `system-prompt` | `llm` | -| `agent` | `llm`, `session` | +| `agent` | `brand`, `llm`, `session` | +| `llm-replay` | `llm`, `session` | | `session-persistence` | `session` | | `invariants` | `agent`, `llm`, `session` | | `session-persistence-jsonl` | `session`, `session-persistence` | | `session-persistence-sqlite` | `session`, `session-persistence` | | `tools` | `agent`, `llm`, `system-prompt` | +| `ui-stdio` | `agent`, `llm`, `session` | | `acp` | `agent`, `llm`, `session`, `session-persistence`, `tools` | | `agent-loop` | `agent`, `llm`, `session`, `session-persistence`, `system-prompt`, `tools` | | `tool-bash` | `agent`, `bash`, `llm`, `tools` | +| `agent-core` | `agent`, `agent-loop`, `invariants`, `llm`, `session`, `system-prompt`, `tool-bash`, `tools` | +| `acp-agent` | `acp`, `agent-core`, `session-persistence-jsonl` | +| `stdio-agent` | `agent`, `agent-core`, `session`, `session-persistence-jsonl`, `ui-stdio` | diff --git a/docs/postmortem/0001-acp-default-export-drops-inject.md b/docs/postmortem/0001-acp-default-export-drops-inject.md index 0f72fa2d44..12eee7a2b7 100644 --- a/docs/postmortem/0001-acp-default-export-drops-inject.md +++ b/docs/postmortem/0001-acp-default-export-drops-inject.md @@ -24,7 +24,7 @@ The ACP server could not create or load a single session — the two RPCs an edi ## Root cause #1 — `export default apply` drops the plugin's `inject` (broke `session/new`) -`packages/acp/src/index.ts` is a *namespace plugin*: it exports `name`, `inject`, `Config`, and `apply` as separate named exports — the same shape as every other plugin in the repo (`invariants`, `llm-deepseek`, `tool-bash`, `stdio-chat`, …). But it *also* ended with one extra line no other plugin had: +`packages/ui/acp/src/index.ts` is a *namespace plugin*: it exports `name`, `inject`, `Config`, and `apply` as separate named exports — the same shape as every other plugin in the repo (`invariants`, `llm-deepseek`, `tool-bash`, `stdio-chat`, …). But it *also* ended with one extra line no other plugin had: ```ts ignore-check export const name = 'acp' @@ -97,8 +97,8 @@ Both bugs share one root process gap: **no test exercised the plugin through its ## Guardrails added -- **Removed `export default apply`** (`packages/acp/src/index.ts`) — the Bug #1 fix. -- **`AgentLoop.resume` reads `this.ctx.get('sessionPersistence')`** (`packages/agent-loop/src/index.ts`) — the Bug #2 fix, with a comment explaining the shadow-walk trap. +- **Removed `export default apply`** (`packages/ui/acp/src/index.ts`) — the Bug #1 fix. +- **`AgentLoop.resume` reads `this.ctx.get('sessionPersistence')`** (`packages/core/agent-loop/src/index.ts`) — the Bug #2 fix, with a comment explaining the shadow-walk trap. - **No-key `session/new` e2e over real stdio** (`examples/acp-agent/tests/acp.e2e.ts`): boots the example as a subprocess through the real Loader and asserts `session/new` resolves. This fails loudly on Bug #1 with no API key. Verified it fails when `export default apply` is restored. - **`TSX_TSCONFIG_PATH` in the e2e spawn**: the subprocess runs from a temp cwd, where tsx cannot find the repo-root tsconfig `paths` map by searching upward — so dsh-* imports silently fell back to built `lib/`. Pointing tsx at the repo tsconfig makes resolution cwd-independent and ensures the test runs *source*, not a possibly-stale build. - **AGENTS.md defensive pattern**: "Line coverage is not behavior coverage; test the REAL entry path, not a synthetic stand-in" — codifies the lesson for every future plugin. diff --git a/docs/rfc/README.md b/docs/rfc/README.md index f3794820d8..f46cb3495f 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -4,63 +4,163 @@ One kind of design doc lives here. An **RFC** records a decision or proposal tha ## Layout and naming -Files are grouped by lifecycle into three folders, and an RFC moves between them as its status changes: +Every RFC has two axes, both encoded in its **path** — `{lifecycle}/{class}/yyyy-mm-dd-topic-title.md`: -- **`proposed/`** — proposals reviewed before implementation; not yet built (or only partly). -- **`implemented/`** — the decision shipped. The file records what was decided and what was rejected. -- **`rejected/`** — the proposal was considered and declined. Kept for the record so the rejection isn't re-litigated. +- **Lifecycle** (the top-level folder) is the RFC's status, and an RFC moves between folders as that status changes: + - **`proposed/`** — proposals reviewed before implementation; not yet built (or only partly). + - **`implemented/`** — the decision shipped. The file records what was decided and what was rejected, and is **kept current with what actually shipped**: when the code later moves a file, renames a package, or changes a key/default, the RFC is updated in the same change to match (facts only — paths, names, structure — not the decision itself). See [implemented/AGENTS.md](implemented/AGENTS.md). + - **`rejected/`** — the proposal was considered and declined. Kept for the record so the rejection isn't re-litigated. +- **Class** (the nested folder) is the *kind* of decision — see [Classification](#classification) below. -Each file is named `yyyy-mm-dd-topic-title.md`, where the date is when the topic was **first proposed** (per git history). Cross-references between RFCs use relative markdown links (`[topic](../implemented/2026-…-….md)`) — never bare prose or numbers — so they are mechanically checkable and survive moves between folders. +The date in the filename is when the topic was **first proposed** (per git history). Cross-references between RFCs use relative markdown links (`[topic](../../implemented/architecture/2026-…-….md)`) — never bare prose or numbers — so they are mechanically checkable and survive moves between folders. + +## Classification + +Each RFC is filed under exactly one **class** — the kind of decision it records. The class is encoded in the path (the folder *is* the label, so a file's location declares its class) and the set is **closed**: `scripts/verify-rfc-classification.ts` rejects any folder outside the set and asserts this index lists every RFC under the heading matching its path. Adding a new class means amending that gate and this section, not just dropping a new folder. See [the classification RFC](implemented/process/2026-06-20-rfc-classification.md) for why the taxonomy is path-encoded and gated. + +| Class | What it covers | +|---|---| +| `feature` | A new user- or model-facing capability. | +| `bug-fix` | Corrects a defect or closes a gap a postmortem surfaced. | +| `simplification` | Removes code, behavior, or surface area without adding a capability. | +| `architecture` | A structural decision about the **shipped source** — how packages relate, what the runtime vocabulary is. | +| `process` | Tooling, policy, or workflow **around** the code — gates, the package manager, vendoring — not runtime behavior. | +| `testing` | Test infrastructure and strategy. | + +The `architecture` / `process` line: **architecture** is about the source we ship; **process** is the surrounding tooling and workflow. (`refactor` is deliberately absent — it overlaps `simplification`, whose discriminator, "does observable behavior change?", already covers it.) ## When to write one -Write an RFC when a decision is **durable** (it shapes the codebase beyond a single function or package), **contested** (there was a real alternative a reasonable engineer might have chosen), and **surprising** (a future reader would otherwise ask "why on earth is it done this way?"). A proposal for substantial future work starts in `proposed/`; a decision already made starts in `implemented/`. +Write an RFC when a decision is **durable** (it shapes the codebase beyond a single function or package), **contested** (there was a real alternative a reasonable engineer might have chosen), and **surprising** (a future reader would otherwise ask "why on earth is it done this way?"). A proposal for substantial future work starts in `proposed/`; a decision already made starts in `implemented/`. Pick the class folder that matches the decision (see [Classification](#classification)). -Do NOT write one for a mechanical or local choice (a variable name, a one-file refactor), for anything already enforced and explained by a gate or a convention in AGENTS.md, or for a still-provisional decision tagged `TODO(...)` in the code — record those as TODOs and promote to an RFC only once they settle. An RFC is never edited into a different decision: supersede it with a new one and cross-link. +Do NOT write one for a mechanical or local choice (a variable name, a one-file refactor), for anything already enforced and explained by a gate or a convention in AGENTS.md, or for a still-provisional decision tagged `TODO(...)` in the code — record those as TODOs and promote to an RFC only once they settle. An RFC is never edited into a *different decision*: supersede it with a new one and cross-link. (Editing an `implemented/` RFC to track where its already-made decision now *lives* — a moved file, a renamed package — is not a different decision and is required, not forbidden; see [implemented/AGENTS.md](implemented/AGENTS.md).) ## Proposed +### Feature + | Title | First proposed | |---|---| -| [Mutation testing as the coverage counterweight](proposed/2026-06-11-mutation-testing.md) | 2026-06-11 | -| [Deterministic tests, the replay invariant fixture, and race stress](proposed/2026-06-11-deterministic-and-stress-testing.md) | 2026-06-11 | -| [Architectural conformance — dependency rules and the adapter kit](proposed/2026-06-11-architectural-conformance.md) | 2026-06-11 | -| [API extractor reports](proposed/2026-06-11-api-extractor-reports.md) | 2026-06-11 | -| [Supply chain checks and vendor drift verification](proposed/2026-06-11-supply-chain-and-vendor-drift.md) | 2026-06-11 | -| [Agent Client Protocol (ACP) support for external editors](proposed/2026-06-14-acp-agent-client-protocol.md) | 2026-06-14 | -| [Multiplex concurrent ACP sessions over one connection](proposed/2026-06-14-acp-multi-session.md) | 2026-06-14 | -| [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/2026-06-15-optional-code-mode.md) | 2026-06-15 | -| [Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)](proposed/2026-06-16-typed-event-schemas.md) | 2026-06-16 | -| [Agent lifecycle and ownership seams](proposed/2026-06-18-agent-lifecycle-and-ownership-seams.md) | 2026-06-18 | -| [Shared persistence write coordinator](proposed/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 | +| [Agent Client Protocol (ACP) support for external editors](proposed/feature/2026-06-14-acp-agent-client-protocol.md) | 2026-06-14 | +| [Multiplex concurrent ACP sessions over one connection](proposed/feature/2026-06-14-acp-multi-session.md) | 2026-06-14 | +| [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/feature/2026-06-15-optional-code-mode.md) | 2026-06-15 | + +### Simplification + +| Title | First proposed | +|---|---| +| [Unify the agent id and the session id](proposed/simplification/2026-06-20-unify-agent-and-session-id.md) | 2026-06-20 | +| [Stop mirroring durable boundaries as agent events](proposed/simplification/2026-06-20-remove-agent-boundary-mirror-events.md) | 2026-06-20 | + +### Architecture + +| Title | First proposed | +|---|---| +| [Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)](proposed/architecture/2026-06-16-typed-event-schemas.md) | 2026-06-16 | +| [Extract a generic long-running tool runtime](proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md) | 2026-06-20 | + +### Process + +| Title | First proposed | +|---|---| +| [Architectural conformance — dependency rules and the adapter kit](proposed/process/2026-06-11-architectural-conformance.md) | 2026-06-11 | +| [API extractor reports](proposed/process/2026-06-11-api-extractor-reports.md) | 2026-06-11 | +| [Supply chain checks and vendor drift verification](proposed/process/2026-06-11-supply-chain-and-vendor-drift.md) | 2026-06-11 | +| [Discover package inventories instead of maintaining static lists](proposed/process/2026-06-20-discover-package-inventory.md) | 2026-06-20 | + +### Testing + +| Title | First proposed | +|---|---| +| [Mutation testing as the coverage counterweight](proposed/testing/2026-06-11-mutation-testing.md) | 2026-06-11 | +| [Deterministic tests, the replay invariant fixture, and race stress](proposed/testing/2026-06-11-deterministic-and-stress-testing.md) | 2026-06-11 | ## Implemented +### Feature + | Title | First proposed | |---|---| -| [Vendor Cordis as source, not npm dependencies](implemented/2026-06-11-vendor-cordis-as-source.md) | 2026-06-11 | -| [Microkernel: extension via Cordis event taxonomy, one concrete loop](implemented/2026-06-11-microkernel-event-taxonomy.md) | 2026-06-11 | -| [Event-sourced sessions with derived message history](implemented/2026-06-11-event-sourced-sessions.md) | 2026-06-11 | -| [Provider-neutral content-block vocabulary owned by dsh-llm](implemented/2026-06-11-content-block-vocabulary.md) | 2026-06-11 | -| [Custom typed tool-schema DSL instead of schemastery](implemented/2026-06-11-custom-schema-dsl.md) | 2026-06-11 | -| [Tool schemas are part of the system-prompt assembly](implemented/2026-06-11-tool-schemas-in-prompt-assembly.md) | 2026-06-11 | -| [Mechanical quality gates over prose guidelines](implemented/2026-06-11-quality-gates.md) | 2026-06-11 | -| [tsdown for JS bundling instead of dumble](implemented/2026-06-11-tsdown-over-dumble.md) | 2026-06-11 | -| [Runtime arg validation at the model boundary](implemented/2026-06-11-runtime-arg-validation.md) | 2026-06-11 | -| [Dev-mode invariants over compile-time deep-readonly](implemented/2026-06-11-dev-invariants-over-deep-readonly.md) | 2026-06-11 | -| [Property-based testing for protocol-shaped code](implemented/2026-06-11-property-based-testing.md) | 2026-06-11 | -| [Doc-sync enforcement](implemented/2026-06-11-doc-sync-enforcement.md) | 2026-06-11 | -| [Markdown cross-link validity linting](implemented/2026-06-18-markdown-cross-link-lint.md) | 2026-06-18 | -| [Structured error taxonomy](implemented/2026-06-11-structured-error-taxonomy.md) | 2026-06-11 | -| [Capability seams — interface / implementation / consumer split](implemented/2026-06-13-capability-seams.md) | 2026-06-13 | -| [Two LLM adapters as a design-verification twin](implemented/2026-06-13-twin-llm-adapters.md) | 2026-06-13 | -| [Session persistence as an abstract service over `SessionEvent`](implemented/2026-06-14-session-persistence.md) | 2026-06-14 | -| [Every session event is enclosed in a turn](implemented/2026-06-15-turn-enclosure-invariant.md) | 2026-06-15 | -| [pnpm as the package manager instead of Yarn 4](implemented/2026-06-16-pnpm-over-yarn.md) | 2026-06-16 | -| [Rich ACP bash rendering — the terminal card (`_meta`) and command classification](implemented/2026-06-18-acp-terminal-and-tool-rendering.md) | 2026-06-18 | +| [Rich ACP bash rendering — the terminal card (`_meta`) and command classification](implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md) | 2026-06-18 | + +### Simplification + +| Title | First proposed | +|---|---| +| [Drop the mutable session summary](implemented/simplification/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 | +| [Drop unconsumed assembled LLM convenience surfaces](implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md) | 2026-06-20 | +| [Drop the unconsumed `llm/adapter-change` event](implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md) | 2026-06-20 | +| [Prune dead methods from the persistence seam](implemented/simplification/2026-06-20-prune-dead-seam-methods.md) | 2026-06-20 | +| [Keep one public stop primitive](implemented/simplification/2026-06-20-public-agent-stop-surface.md) | 2026-06-20 | +| [Fold trace-only session facts into load-bearing events](implemented/simplification/2026-06-20-collapse-trace-only-session-events.md) | 2026-06-20 | + +### Architecture + +| Title | First proposed | +|---|---| +| [Microkernel: extension via Cordis event taxonomy, one concrete loop](implemented/architecture/2026-06-11-microkernel-event-taxonomy.md) | 2026-06-11 | +| [Event-sourced sessions with derived message history](implemented/architecture/2026-06-11-event-sourced-sessions.md) | 2026-06-11 | +| [Provider-neutral content-block vocabulary owned by dsh-llm](implemented/architecture/2026-06-11-content-block-vocabulary.md) | 2026-06-11 | +| [Custom typed tool-schema DSL instead of schemastery](implemented/architecture/2026-06-11-custom-schema-dsl.md) | 2026-06-11 | +| [Tool schemas are part of the system-prompt assembly](implemented/architecture/2026-06-11-tool-schemas-in-prompt-assembly.md) | 2026-06-11 | +| [Runtime arg validation at the model boundary](implemented/architecture/2026-06-11-runtime-arg-validation.md) | 2026-06-11 | +| [Dev-mode invariants over compile-time deep-readonly](implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md) | 2026-06-11 | +| [Structured error taxonomy](implemented/architecture/2026-06-11-structured-error-taxonomy.md) | 2026-06-11 | +| [Capability seams — interface / implementation / consumer split](implemented/architecture/2026-06-13-capability-seams.md) | 2026-06-13 | +| [Two LLM adapters as a design-verification twin](implemented/architecture/2026-06-13-twin-llm-adapters.md) | 2026-06-13 | +| [Session persistence as an abstract service over `SessionEvent`](implemented/architecture/2026-06-14-session-persistence.md) | 2026-06-14 | +| [Every session event is enclosed in a turn](implemented/architecture/2026-06-15-turn-enclosure-invariant.md) | 2026-06-15 | +| [Shared persistence write coordinator](implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 | +| [Agent lifecycle and ownership seams](implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md) | 2026-06-18 | +| [Reorganize packages into a modular hierarchy](implemented/architecture/2026-06-20-package-hierarchy.md) | 2026-06-20 | +| [Branded IDs everywhere they belong](implemented/architecture/2026-06-20-branded-ids.md) | 2026-06-20 | +| [Extract example apps into packages](implemented/architecture/2026-06-20-extract-example-app-packages.md) | 2026-06-20 | + +### Process + +| Title | First proposed | +|---|---| +| [Vendor Cordis as source, not npm dependencies](implemented/process/2026-06-11-vendor-cordis-as-source.md) | 2026-06-11 | +| [Mechanical quality gates over prose guidelines](implemented/process/2026-06-11-quality-gates.md) | 2026-06-11 | +| [tsdown for JS bundling instead of dumble](implemented/process/2026-06-11-tsdown-over-dumble.md) | 2026-06-11 | +| [Doc-sync enforcement](implemented/process/2026-06-11-doc-sync-enforcement.md) | 2026-06-11 | +| [pnpm as the package manager instead of Yarn 4](implemented/process/2026-06-16-pnpm-over-yarn.md) | 2026-06-16 | +| [TSC-first build and one tsconfig](implemented/process/2026-06-17-ts-build-config.md) | 2026-06-17 | +| [Markdown cross-link validity linting](implemented/process/2026-06-18-markdown-cross-link-lint.md) | 2026-06-18 | +| [Core-data-structures catalog and the `ts type-equiv` drift gate](implemented/process/2026-06-20-core-data-structures-catalog.md) | 2026-06-20 | +| [Generated cordis events + services catalog](implemented/process/2026-06-20-generated-cordis-catalog.md) | 2026-06-20 | +| [Classify RFCs by kind via path-encoded subdirectories](implemented/process/2026-06-20-rfc-classification.md) | 2026-06-20 | + +### Testing + +| Title | First proposed | +|---|---| +| [Property-based testing for protocol-shaped code](implemented/testing/2026-06-11-property-based-testing.md) | 2026-06-11 | +| [ACP snapshot tests — record-once / replay-deterministic](implemented/testing/2026-06-19-acp-snapshot-tests.md) | 2026-06-19 | +| [Real-API e2e in CI against the external DeepSeek API](implemented/testing/2026-06-19-real-api-e2e-ci.md) | 2026-06-19 | +| [Use `session.jsonl` as the only snapshot session-log artifact](implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md) | 2026-06-20 | ## Rejected +### Simplification + | Title | First proposed | |---|---| -| [Deep-readonly public surfaces](rejected/2026-06-11-immutable-public-surfaces.md) | 2026-06-11 | +| [Persist assembled assistant messages, not stream chunks](rejected/simplification/2026-06-20-assembled-assistant-messages-only.md) | 2026-06-20 | +| [Drop ACP session/load until resume has a product shape](rejected/simplification/2026-06-20-drop-acp-session-load.md) | 2026-06-20 | +| [Drop ACP terminal `_meta` rendering](rejected/simplification/2026-06-20-drop-acp-terminal-meta.md) | 2026-06-20 | +| [Drop bash full-output spill files](rejected/simplification/2026-06-20-drop-bash-output-spill-files.md) | 2026-06-20 | +| [Drop durable step boundary events](rejected/simplification/2026-06-20-drop-durable-step-boundaries.md) | 2026-06-20 | +| [Drop unused session lineage metadata](rejected/simplification/2026-06-20-drop-unused-session-lineage.md) | 2026-06-20 | +| [Fold the persistence interface into dsh-session](rejected/simplification/2026-06-20-fold-session-persistence-interface.md) | 2026-06-20 | +| [Collapse tool-owned UI presentation](rejected/simplification/2026-06-20-generic-tool-rendering.md) | 2026-06-20 | +| [Retire mid-turn steering](rejected/simplification/2026-06-20-retire-mid-turn-steering.md) | 2026-06-20 | +| [Return the ACP bridge to one live session per connection](rejected/simplification/2026-06-20-single-session-acp-bridge.md) | 2026-06-20 | +| [Truncate interrupted final turns on load](rejected/simplification/2026-06-20-truncate-interrupted-turns.md) | 2026-06-20 | + +### Architecture + +| Title | First proposed | +|---|---| +| [Deep-readonly public surfaces](rejected/architecture/2026-06-11-immutable-public-surfaces.md) | 2026-06-11 | +| [Make the shared example base providerless](rejected/architecture/2026-06-20-providerless-example-base.md) | 2026-06-20 | diff --git a/docs/rfc/implemented/AGENTS.md b/docs/rfc/implemented/AGENTS.md new file mode 100644 index 0000000000..e5ddc5eeea --- /dev/null +++ b/docs/rfc/implemented/AGENTS.md @@ -0,0 +1,15 @@ +# AGENTS.md — Implemented RFCs + +These are RFCs whose decision has **shipped**. The repo-wide and docs-wide rules still apply ([root AGENTS.md](../../../AGENTS.md) § "Type Safety and Documentation", [docs/AGENTS.md](../../AGENTS.md)); this file adds one rule specific to this folder. + +## Keep an implemented RFC current with what actually shipped + +An RFC in `implemented/` describes a decision that is now **live code**. Keep its description of the shipped reality accurate: when the implementation later moves a file, renames a package or symbol, changes a config key/default/error code, or relocates a plugin, update the RFC in the **same change** that touches the code — exactly as you would a package README. A stale implemented RFC (pointing at a path that no longer exists, naming a package that was renamed, describing a structure that was refactored) is worse than no RFC: a future reader trusts it and is misled. + +Update it **in place** to state the current truth. Do **not** leave the outdated text in and bolt on a "superseded / now actually…" note — that makes the document a changelog of its own drift and forces the reader to reconstruct the present from a pile of corrections. Write what is true now. + +### This is not a license to rewrite the *decision* + +Keeping the shipped-state description current is about **facts** (paths, names, structure, defaults) — not about silently flipping the **decision and its rationale** into a different one. If the underlying choice itself is reversed or materially changed (not just relocated), that is a new decision: write a new RFC and cross-link, per [rfc/README.md](../README.md) ("An RFC is never edited into a different decision"). The line: a refactor that moves where the decision is *realized* → edit this RFC to match; a reversal of *what was decided* → a new RFC. + +When in doubt, ask whether a reader following this RFC to the code would land on something real. If not, it needs updating. diff --git a/docs/rfc/implemented/CLAUDE.md b/docs/rfc/implemented/CLAUDE.md new file mode 120000 index 0000000000..47dc3e3d86 --- /dev/null +++ b/docs/rfc/implemented/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/docs/rfc/implemented/2026-06-11-content-block-vocabulary.md b/docs/rfc/implemented/architecture/2026-06-11-content-block-vocabulary.md similarity index 100% rename from docs/rfc/implemented/2026-06-11-content-block-vocabulary.md rename to docs/rfc/implemented/architecture/2026-06-11-content-block-vocabulary.md diff --git a/docs/rfc/implemented/2026-06-11-custom-schema-dsl.md b/docs/rfc/implemented/architecture/2026-06-11-custom-schema-dsl.md similarity index 100% rename from docs/rfc/implemented/2026-06-11-custom-schema-dsl.md rename to docs/rfc/implemented/architecture/2026-06-11-custom-schema-dsl.md diff --git a/docs/rfc/implemented/2026-06-11-dev-invariants-over-deep-readonly.md b/docs/rfc/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md similarity index 86% rename from docs/rfc/implemented/2026-06-11-dev-invariants-over-deep-readonly.md rename to docs/rfc/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md index 518cd7168d..b9a182a4e9 100644 --- a/docs/rfc/implemented/2026-06-11-dev-invariants-over-deep-readonly.md +++ b/docs/rfc/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md @@ -8,7 +8,7 @@ Status: implemented (accepted 2026-06-13) The session log is append-only by contract, but the types don't enforce it: `session.events` returns `readonly SessionEvent[]` whose *elements* are mutable, and `deriveMessages()` handed the logged `content` arrays/blocks out by reference. The loop then passes those derived messages into the `agent/request` waterfall and on to adapters, where mutating the request is sanctioned — so a request middleware could reach back and rewrite history, silently breaking replay equivalence and the derived-history guarantee. Separately, the event taxonomy (turn/step nesting, seq monotonicity, tool-call/result pairing, legal status transitions) was asserted only where individual tests happened to look. -Two ways to defend the log: make immutability part of the type (`DeepReadonly` on the way out), or catch corruption at runtime in dev. The runtime-validation proposal took the runtime route; [the deep-readonly proposal](../rejected/2026-06-11-immutable-public-surfaces.md) took the type route. +Two ways to defend the log: make immutability part of the type (`DeepReadonly` on the way out), or catch corruption at runtime in dev. The runtime-validation proposal took the runtime route; [the deep-readonly proposal](../../rejected/architecture/2026-06-11-immutable-public-surfaces.md) took the type route. ## Decision @@ -26,4 +26,4 @@ The invariants encode the *real* contract, not an idealized one: a `tool/call` m - History corruption is caught loudly in tests and demos, at zero production cost and zero type noise. The trade-off is that the guarantee is dynamic (a dev-mode tripwire) rather than static. - The invariants plugin doubles as executable documentation of the event taxonomy — the assertions are the contract. - `Session.events` keeps its `readonly SessionEvent[]` type; no consumer churn. -- This folds in [the deep-readonly proposal](../rejected/2026-06-11-immutable-public-surfaces.md) — there is no separate deep-readonly record; this records the decision to *not* pursue that approach. `InvariantError` is a plain `Error` with a `code` for now; a later taxonomy change can promote it. +- This folds in [the deep-readonly proposal](../../rejected/architecture/2026-06-11-immutable-public-surfaces.md) — there is no separate deep-readonly record; this records the decision to *not* pursue that approach. `InvariantError` is a plain `Error` with a `code` for now; a later taxonomy change can promote it. diff --git a/docs/rfc/implemented/2026-06-11-event-sourced-sessions.md b/docs/rfc/implemented/architecture/2026-06-11-event-sourced-sessions.md similarity index 100% rename from docs/rfc/implemented/2026-06-11-event-sourced-sessions.md rename to docs/rfc/implemented/architecture/2026-06-11-event-sourced-sessions.md diff --git a/docs/rfc/implemented/2026-06-11-microkernel-event-taxonomy.md b/docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md similarity index 96% rename from docs/rfc/implemented/2026-06-11-microkernel-event-taxonomy.md rename to docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md index 942445a429..c8869c0616 100644 --- a/docs/rfc/implemented/2026-06-11-microkernel-event-taxonomy.md +++ b/docs/rfc/implemented/architecture/2026-06-11-microkernel-event-taxonomy.md @@ -12,7 +12,7 @@ The product principle (see the 微内核Harness实现思路 design doc) is "ever Pure Cordis event taxonomy. The loop's extension seams are typed events with deliberate dispatch modes: -- **waterfall** (around-middleware) where plugins mutate or veto: `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/execute`, `llm/stream`, `llm/generate`, `system-prompt/assemble`. +- **waterfall** (around-middleware) where plugins mutate or veto: `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/execute`, `llm/stream`, `system-prompt/assemble`. - **emit** (sync fire-and-forget) for notifications: turn/step boundaries, stream chunks, lifecycle, errors. - **parallel** (awaited) for the one durability checkpoint: `session/flush`. diff --git a/docs/rfc/implemented/2026-06-11-runtime-arg-validation.md b/docs/rfc/implemented/architecture/2026-06-11-runtime-arg-validation.md similarity index 90% rename from docs/rfc/implemented/2026-06-11-runtime-arg-validation.md rename to docs/rfc/implemented/architecture/2026-06-11-runtime-arg-validation.md index 9998c85d35..5c0aad9eb2 100644 --- a/docs/rfc/implemented/2026-06-11-runtime-arg-validation.md +++ b/docs/rfc/implemented/architecture/2026-06-11-runtime-arg-validation.md @@ -17,6 +17,6 @@ The validator mirrors `schemaSpecToJsonSchema` semantics exactly — same struct ## Consequences - The model gets actionable feedback on its own malformed calls instead of an opaque crash, closing the gap between `InferArgs`'s promise and runtime reality. -- The validator and `InferArgs` must stay in agreement; that drift risk is to be closed by a property test ([property-based testing](2026-06-11-property-based-testing.md), not yet landed) generating args that satisfy `InferArgs` and asserting they pass `validateArgs`. Until then the agreement rests on the example tests and the shared converter structure. +- The validator and `InferArgs` must stay in agreement; that drift risk is to be closed by a property test ([property-based testing](../testing/2026-06-11-property-based-testing.md), not yet landed) generating args that satisfy `InferArgs` and asserting they pass `validateArgs`. Until then the agreement rests on the example tests and the shared converter structure. - `ToolArgsError` is a plain `Error` with a `code` field for now; if a harness-wide error taxonomy lands it becomes a subclass without changing callers that read `.message`. - Validation cost is negligible next to a model call. diff --git a/docs/rfc/implemented/2026-06-11-structured-error-taxonomy.md b/docs/rfc/implemented/architecture/2026-06-11-structured-error-taxonomy.md similarity index 100% rename from docs/rfc/implemented/2026-06-11-structured-error-taxonomy.md rename to docs/rfc/implemented/architecture/2026-06-11-structured-error-taxonomy.md diff --git a/docs/rfc/implemented/2026-06-11-tool-schemas-in-prompt-assembly.md b/docs/rfc/implemented/architecture/2026-06-11-tool-schemas-in-prompt-assembly.md similarity index 100% rename from docs/rfc/implemented/2026-06-11-tool-schemas-in-prompt-assembly.md rename to docs/rfc/implemented/architecture/2026-06-11-tool-schemas-in-prompt-assembly.md diff --git a/docs/rfc/implemented/2026-06-13-capability-seams.md b/docs/rfc/implemented/architecture/2026-06-13-capability-seams.md similarity index 90% rename from docs/rfc/implemented/2026-06-13-capability-seams.md rename to docs/rfc/implemented/architecture/2026-06-13-capability-seams.md index 3446fce9af..e66440c3ee 100644 --- a/docs/rfc/implemented/2026-06-13-capability-seams.md +++ b/docs/rfc/implemented/architecture/2026-06-13-capability-seams.md @@ -26,4 +26,4 @@ The split is not mandatory when the parts are genuinely one concern: the LLM sea ## Consequences -More packages and more boilerplate per capability (a `package.json`/`tsconfig`/README trio, the inject wiring). Bought: implementations and consumers ship and version independently, and a new backend never risks the model-facing contract. The rule is documented in [AGENTS.md](../../../AGENTS.md) § Conventions ("Capability seams are three packages") and [architecture.md](../../architecture.md) § "Capability seams"; the bash trio is the reference template. When to fold vs. split is a judgment call the architecture doc spells out — this RFC records *why* the default is to split. +More packages and more boilerplate per capability (a `package.json`/`tsconfig`/README trio, the inject wiring). Bought: implementations and consumers ship and version independently, and a new backend never risks the model-facing contract. The rule is documented in [AGENTS.md](../../../../AGENTS.md) § Conventions ("Capability seams are three packages") and [architecture.md](../../../architecture.md) § "Capability seams"; the bash trio is the reference template. When to fold vs. split is a judgment call the architecture doc spells out — this RFC records *why* the default is to split. diff --git a/docs/rfc/implemented/2026-06-13-twin-llm-adapters.md b/docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md similarity index 93% rename from docs/rfc/implemented/2026-06-13-twin-llm-adapters.md rename to docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md index 6d6ded31e4..ecd21b5cc3 100644 --- a/docs/rfc/implemented/2026-06-13-twin-llm-adapters.md +++ b/docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md @@ -21,4 +21,4 @@ Alternatives considered: **a single adapter** — less code and half the e2e cos ## Consequences -Double the adapter maintenance and double the key-gated e2e surface (both adapters cover V4 Flash and Pro across representative thinking/effort modes). Bought: a continuously-verified neutrality guarantee for the most leak-prone abstraction in the codebase, and a worked second example for adapter authors. The two share the core Config shape (`apiKey`/`baseURL`/`models`) so a deployment swaps mostly one line, but the reasoning knob differs — `dsh-llm-deepseek` takes `thinking`/`reasoningEffort`, `dsh-llm-pi-ai` takes a single `reasoning` level — so a swap translates that field. If the maintenance cost ever outweighs the verification value (e.g. once conformance tests from [architectural conformance](../proposed/2026-06-11-architectural-conformance.md) cover the contract mechanically), retiring the twin to a single adapter + the conformance kit would be a new RFC superseding this one. +Double the adapter maintenance and double the key-gated e2e surface (both adapters cover V4 Flash and Pro across representative thinking/effort modes). Bought: a continuously-verified neutrality guarantee for the most leak-prone abstraction in the codebase, and a worked second example for adapter authors. The two share the core Config shape (`apiKey`/`baseURL`/`models`) so a deployment swaps mostly one line, but the reasoning knob differs — `dsh-llm-deepseek` takes `thinking`/`reasoningEffort`, `dsh-llm-pi-ai` takes a single `reasoning` level — so a swap translates that field. If the maintenance cost ever outweighs the verification value (e.g. once conformance tests from [architectural conformance](../../proposed/process/2026-06-11-architectural-conformance.md) cover the contract mechanically), retiring the twin to a single adapter + the conformance kit would be a new RFC superseding this one. diff --git a/docs/rfc/implemented/2026-06-14-session-persistence.md b/docs/rfc/implemented/architecture/2026-06-14-session-persistence.md similarity index 74% rename from docs/rfc/implemented/2026-06-14-session-persistence.md rename to docs/rfc/implemented/architecture/2026-06-14-session-persistence.md index e676181465..7abe7ec8ed 100644 --- a/docs/rfc/implemented/2026-06-14-session-persistence.md +++ b/docs/rfc/implemented/architecture/2026-06-14-session-persistence.md @@ -8,7 +8,7 @@ Status: implemented (proposed 2026-06-14, accepted 2026-06-15) ## Context -Sessions lived only in memory. The example `session-jsonl.ts` plugin (duplicated byte-for-byte in both examples) was write-only telemetry: it buffered `session/event` and appended JSON lines, with no read/replay path, no crash-safety (no fsync, no atomic write, a fire-and-forget dispose drain), no listing, and no format versioning. Nothing could rehydrate a past session from disk into a live agent, so durable resume ("continue yesterday's task"), durable forking, and the ACP `session/load` method ([ACP support](../proposed/2026-06-14-acp-agent-client-protocol.md)) were all impossible. +Sessions lived only in memory. The example `session-jsonl.ts` plugin (duplicated byte-for-byte in both examples) was write-only telemetry: it buffered `session/event` and appended JSON lines, with no read/replay path, no crash-safety (no fsync, no atomic write, a fire-and-forget dispose drain), no listing, and no format versioning. Nothing could rehydrate a past session from disk into a live agent, so durable resume ("continue yesterday's task"), durable forking, and the ACP `session/load` method ([ACP support](../../proposed/feature/2026-06-14-acp-agent-client-protocol.md)) were all impossible. The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append-only log the single source of truth and derives LLM history from it. Persistence had to stay faithful to that: persist the existing `SessionEvent` directly, with no parallel "persisted message" type that the log is converted to and from. The backend also had to be swappable — a file store now, a database store later — behind one interface. @@ -16,19 +16,19 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append Persistence is an abstract **capability seam** ([capability seams](2026-06-13-capability-seams.md), the `dsh-bash` template), not loop or core logic: -1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`/`has`/`delete`/`update`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type. -2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**) plus an atomic `.summary.json` sidecar for the mutable `SessionSummary`. +1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type. +2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**). Key choices recorded here because they are durable, contested, and surprising: - **The canonical durable log persists every `SessionEvent` verbatim, including `assistant/chunk`.** `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log. - **Append-only; a crashed turn is closed, never truncated.** Committed events — those at or below a flushed `turn/end` — are never rewritten. The loop only flushes at `turn/end`, so a crash can leave a durable log whose final turn never closed: real, fully-written events sit after the last `turn/end`. **A single turn can be huge in a long-horizon task** (many steps, large tool output spanning a long autonomous run), so discarding the interrupted turn would silently destroy a large amount of real work — truncating a turn is wrong. Instead, on reload `load` PRESERVES those events and CLOSES the orphaned turn by durably appending the minimal synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered, then a `step/end` if a step was still open, then a `turn/end` carrying the merge-extensible `{ kind: 'interrupted' }` reason (a marker that records the turn was cut short by a crash, not completed by the model — no loop ever emits it). The synthetic tool results matter for resume correctness: the loop logs the `assistant/message` (carrying the `tool-call` blocks) BEFORE running the tools, so a crash mid-tool leaves calls without results; `deriveMessages()` would then replay a dangling assistant tool-call, which every provider rejects as an invalid transcript on the next request. Answering each orphaned call with an error result keeps the rehydrated history valid. `load` returns the balanced log, so a resumed session is immediately usable. The ONLY thing discarded is a never-fully-written **torn tail fragment** — a final record whose bytes (JSONL) or row were never completely flushed; that fragment is not a valid event and is dropped before the synthetic closers are written. A parse error or `seq` gap in the COMMITTED region (at or before the last real `turn/end`) is genuine corruption and makes the session unloadable. - **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — `append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, interrupted-turn close on load, contiguous-seq), expressed once over file bytes and once over rows. -- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionMeta` (`SessionHeader & SessionSummary`) owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. +- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](../simplification/2026-06-19-drop-mutable-session-summary.md).) - **Resume is an async factory, not a change to synchronous create.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and starts a fresh agent on the resumed id (NOT `${agentId}-session`). The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent. -Format versioning: the header carries a `version`; `load` rejects an unknown version (no v1 migration). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated on load) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later. +Format versioning: the header carries a `version`; `load` rejects any non-current version (no migration — the pre-release session format is pinned at `SESSION_FORMAT_VERSION = 0` and absorbs shape churn, per the AGENTS.md pre-release stance). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated on load) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later. ## Consequences -Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and the foundation the ACP `session/load` ([ACP support](../proposed/2026-06-14-acp-agent-client-protocol.md)) needs — all over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only / contiguous-seq / lazy-materialization / serializability semantics. This completes [event-sourced sessions](2026-06-11-event-sourced-sessions.md)'s deferred "real persistence backend" and resolves its `TODO(review)` on the event vocabulary: persisting the log freezes its shape, and the `assistant/chunk` fidelity question is answered above (persist verbatim). +Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and the foundation the ACP `session/load` ([ACP support](../../proposed/feature/2026-06-14-acp-agent-client-protocol.md)) needs — all over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only / contiguous-seq / lazy-materialization / serializability semantics. This completes [event-sourced sessions](2026-06-11-event-sourced-sessions.md)'s deferred "real persistence backend" and resolves its `TODO(review)` on the event vocabulary: persisting the log freezes its shape, and the `assistant/chunk` fidelity question is answered above (persist verbatim). diff --git a/docs/rfc/implemented/2026-06-15-turn-enclosure-invariant.md b/docs/rfc/implemented/architecture/2026-06-15-turn-enclosure-invariant.md similarity index 100% rename from docs/rfc/implemented/2026-06-15-turn-enclosure-invariant.md rename to docs/rfc/implemented/architecture/2026-06-15-turn-enclosure-invariant.md diff --git a/docs/rfc/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md b/docs/rfc/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md new file mode 100644 index 0000000000..35cb2eb6b3 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md @@ -0,0 +1,42 @@ +# RFC: Agent lifecycle and ownership seams + +Status: implemented + +## Problem + +Several ACP and tool-bash limitations were symptoms of the same missing seam: plugins could create or resume agents through `ctx.agents`, but they could not own and dispose one agent independently, and long-running bash tasks carried no stable owner in the executor itself. ACP aborted and awaited agents on disconnect but could not unregister just that session's agent; `session/cancel` could not cancel queued-but-not-yet-started work; and `tool-bash` kept task ownership in a plugin-local `Map`, so an HMR reload could make an old task look unowned. + +## What was implemented + +The three seams shipped across a stacked chain of PRs (the queue-aware cancel, the `AgentHandle` disposer, and the bash owner token), each converged independently. + +### 1. Queue-aware `Agent.cancel(reason?)` + +A new `cancel()` verb on the `Agent` interface — the single public stop primitive. (It originally shipped alongside a narrower step-only `abort()`; that verb was later removed as unused, leaving `cancel()` the only public way to stop work.) It clears the inbox's queued + steering FIFOs, aborts the in-flight step if any, and drives a **turn-scoped cancellation marker** the driver loop checks at every turn-decision point — so a prompt that is queued-but-not-yet-started never runs, a cancel landing in the pre-step / continuation window drops the about-to-run turn (ending it `aborted`), and a later prompt cannot be batched into the cancelled turn. `whenIdle()` reaches post-cancel quiescence. ACP `session/cancel` maps to `cancel()`. The marker is armed ONLY when there is something to cancel, so an idle no-op cancel cannot strand the next prompt. + +### 2. `AgentHandle` async disposer + +`ctx.agents.create`/`resume` (and the `AgentFactory` interface) return `AgentHandle = { agent: Agent; dispose(): Promise }`. The disposer is a **capability** — only the holder can tear down exactly this agent: stop its loop, `await` the loop's exit (true quiescence, not just the `disposed` status flip), unregister it, and remove its session from the store. `ctx.agents.get(id)` still returns a bare `Agent`. Config-created agents stay owned by the `AgentLoop` fiber (the handle is discarded). ACP holds each session's disposer in its `SessionRecord` and runs it on disconnect/teardown, so a bare client disconnect leaves no registered agent and no session-store entry — even when `session/load` races teardown (the just-resumed handle is disposed before the closed-guard throw). + +**Teardown ORDER is load-bearing for durability**, and the implementation folds the session lifecycle into the agent's SINGLE composite cordis effect (`SessionStore.prepare`/`enter`/`announce`, replacing a sibling-effect split). A fiber unload disposes sibling effects concurrently (`Promise.all`), which would race the session's `onAppend` detach against the loop's closing `session/flush` and drop the closing `turn/end`; inside one effect the disposers run as an ordered LIFO chain (loop stopped + `await agent.done` BEFORE the session detaches), so the loop's final flush is captured on BOTH the handle's `dispose()` and a fiber unload. The register disposer's `agent/disposed` emit is contained (a throwing listener must not reject the chain and skip the later session detach). + +### 3. Bash owner token in the seam + +Background-task ownership moved from a `tool-bash` plugin-local `Map` into the executor. `BashExecRequest` gains an optional `owner?: string`; the resolved `BashExecSpec` carries it as required-but-nullable `owner: string | undefined` (a forgotten owner is a visible `undefined`, never a silently-absent property). The executor stores the token on its task and exposes it via a new `BashExecutor.ownerOf(id): string | undefined` seam (NOT on the public `BashTask` — one read path, no redundant API). `tool-bash` deletes its `Map` entirely: it stamps `exec.agent?.session.header.id` as the owner at `start`, and `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to the caller's token with `!== undefined` semantics (an empty-string token is still a real owner). The completion notice finds the live agent by scanning `ctx.get('agents')?.list()` for `agent.session.header.id === ownerToken` (read via `ctx.get` — `onTaskDone` runs on the bash fiber, a foreign fiber, where the `ctx.agents` proxy would throw). Because ownership now lives on the task in the executor (disposed with the `dsh-bash` fiber), it SURVIVES a `tool-bash` HMR reload — closing the old `XXX(tool-bash-owner-hmr)` gap. (The `onTaskDone` listener is still effect-scoped to `tool-bash`'s `apply`, so a completion landing during the reload gap still drops its one notice — the pre-existing reload-gap drop — but the ownership fence itself is HMR-proof.) + +## Acceptance Criteria (met) + +- ACP disconnect/session close leaves no registered agent AND no session-store entry for that session, even when `session/load` races teardown. +- `session/cancel` before a queued prompt starts prevents that prompt from running and cannot batch the next prompt into the cancelled turn. +- A `tool-bash` HMR reload does NOT make an existing background task readable or killable by a different session (ownership survives on the executor). +- Existing non-ACP demos still work without managing handles explicitly; config-created agents remain owned by the `AgentLoop` plugin fiber. + +## Seam precondition (recorded) + +The bash owner-token comparison relies on `session.header.id` being unique among live agents. The agent registry does NOT enforce this — it rejects a duplicate *agentId*, not a duplicate session id, and `createAgent` accepts an arbitrary `sessionId`. This is NOT reachable via ACP (UUID sessionId, `agentId === sessionId`, duplicate-load rejected), so it is not a live product hole, but a programmatic caller that registers two agents with the same session id would break bash isolation and mis-route the completion notice. The access *policy* (token comparison) stays in `tool-bash` (the consumer); the bash seam stores only an opaque `owner` string and never interprets it — the correct interface/impl/consumer split. + +The planned resolution is to remove the precondition by construction — see [unify the agent id and the session id](../../proposed/simplification/2026-06-20-unify-agent-and-session-id.md): once an agent IS its session (one id), the registry's existing unique-`agentId` check is a unique-session-id guarantee and no two live agents can share a session token. + +## Notes + +This touched public interfaces (`Agent`, `AgentFactory`, the bash seam) deliberately, not as a local ACP patch. The simple synchronous `Agent.send()` ergonomics were preserved; the async lifecycle path is additive, for owners that need it. diff --git a/docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md b/docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md new file mode 100644 index 0000000000..44ae67f872 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md @@ -0,0 +1,37 @@ +# RFC: Shared persistence write coordinator + +Status: implemented (proposed and accepted 2026-06-18, implemented 2026-06-20) + +## Problem + +`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration was duplicated: per-session state, `session/created` adoption, backend-specific prefix reads, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. The pure seed-prefix collision and serializability guards had already moved into the seam package; the remaining orchestration was still correctness-heavy and received the same fixes twice. A code-level diff showed the two backends were byte-identical — or same-algorithm — for ALL of it: the four maps (`states`/`buffers`/`chains`/`inits`), `installWritePath`, `initFor`, `onCreated`'s four cases, `flush`, `drain`, `serialize`, `adopt`, `adoptLivePrefix`, `assertVersion`, and the `create`/`append`/`load` skeletons. Only the storage primitives (write bytes vs. INSERT rows) differed. + +## Decision + +Extract a backend-agnostic `PersistenceCoordinator` into `dsh-session-persistence`. The coordinator owns the orchestration once; each first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements a small `PersistenceBackend` hook interface, and delegates its four public service methods (`create`/`append`/`load`/`list`) to it. + +Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The RFC's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks; it cannot reach the coordinator's private orchestration state, and the public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator at all. + +### The hook interface (`PersistenceBackend`) + +Six methods (five required + an optional lifecycle hook) — the only seam between the coordinator and storage: + +- `name` — backend label for the dispose-failure `AggregateError`. +- `loadStored(id)` — read a stored prefix by id, scanning ANY storage scope (every JSONL cwd bucket; SQLite's id is globally unique). Used by resume/load and, via `!== undefined`, the create-collision probe. +- `loadLive(id, cwd)` — read a stored prefix SCOPED to `cwd`. **Deliberately distinct from `loadStored`**: HMR live-adoption must only adopt a persisted log at the SAME cwd as the live session; a same-id log at a different cwd is a collision, not a resume. Collapsing the two reintroduces a cross-cwd adoption bug. SQLite ignores `cwd`. +- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook). +- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`). +- `list()` — list all stored metadata. +- `close?()` — optional lifecycle teardown (SQLite closes its db handle; JSONL omits it), awaited in the dispose effect AFTER the quiescence drain so a close failure never masks a drain error. + +### The opaque torn marker + +The single design choice that keeps the seam clean: the crash-repair "where is the torn tail" token is OPAQUE to the coordinator. The coordinator computes the synthetic closers (it owns `interruptedTurnClosers` from `dsh-session`), but it only ever tests `tornMarker !== undefined` and passes the value straight back to `commitRepair` — it never inspects it. Each backend picks its own marker type: JSONL uses the byte offset to truncate to, SQLite the seq to delete from (both happen to be `number`). The JSONL backend folds its `committedBytes < buffer.byteLength` comparison INSIDE the hook so the returned marker is already `number | undefined`; without that fold the coordinator would have to know about byte lengths. + +## Testing + +The shared `runPersistenceContract` (public-API contract) keeps running for every backend. A new `runCoordinatorContract` (`tests/coordinator-contract.ts`) holds the write-path orchestration — adoption, HMR, collision, dispose-drain, crash-tail repair — and runs once per backend through a `CoordinatorFixture` (an in-memory reference + jsonl + sqlite). The per-backend specs shrank to storage mechanics only (JSONL: path safety, fsync rollback, bucket listing; SQLite: schema version, `scanRows`, transaction rollback). A through-coordinator torn-tail→load→`commitRepair` test per real backend (via a `corruptTail` fixture hook) keeps the coordinator's torn-marker repair branch covered under the 100% per-file gate — the contract crash test only produces synthetic closers, never a torn marker, so it could not reach that branch. + +## Risks and what we gave up + +The pre-extraction duplication was verbose but explicit — each backend read top-to-bottom. The coordinator adds one indirection (the hook seam) and one new concept (the opaque torn marker). This clears the bar because the centralized logic is the correctness-heavy part that was already being fixed twice, and the hook set is narrow (six methods, no inheritance). The hook surface was deliberately held to the minimum: the create-collision probe is NOT a separate hook — it folds into `loadStored(id) !== undefined`; there is no separate `materialize` hook (folded into `appendBatch` for atomicity); `list()` stays a backend method with no coordinator pass-through (listing needs none of the orchestration). The net effect is a reduction: one orchestration copy instead of two, the backends shrank by ~1200 lines of duplicated churn, and a future backend implements a handful of small primitives instead of copying the entire `session/event` → buffer → flush machinery. diff --git a/docs/rfc/implemented/architecture/2026-06-20-branded-ids.md b/docs/rfc/implemented/architecture/2026-06-20-branded-ids.md new file mode 100644 index 0000000000..0b4975089d --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-20-branded-ids.md @@ -0,0 +1,68 @@ +# RFC: Branded IDs everywhere they belong + +Status: implemented (proposed and accepted 2026-06-20) + +## Problem + +The harness already brands three identifiers — `CallId` (`packages/llm/llm/src/brand.ts`), `SessionId` (`packages/core/session/src/types.ts`), and `AgentId` (`packages/core/agent/src/types.ts`) — using the `Branded = string & { readonly [BRAND]: B }` machinery (owned by the type-only `@deepseek-ai/dsh-brand` package at `packages/util/brand/` — see its [README](../../../../packages/util/brand/README.md)) and a zero-cost cast factory per type. `dsh-brand` also states the governing policy: *"Branding is for ids that cross package boundaries and could plausibly be confused; not every string needs a brand."* That policy is right; the problem is that it is only half-applied. Two gaps let a structurally-identical-but-semantically-wrong string slip through the type checker today. + +**Gap 1 — unbranded cross-boundary IDs in the bash seam.** The background-task id is a plain `string`: `BashTask.id: string` (`packages/bash/bash/src/types.ts`), carried as `string` through the whole executor seam (`BashExecutor.get`/`ownerOf`/`readOutput`/`kill(id: string)` in `packages/bash/bash/src/index.ts`) and validated/passed as `string` by the model-facing tools (`validateTaskId`, `assertTaskAccess`, the `task_id` schema arg in `packages/bash/tool-bash/src/index.ts`). It is generated by a per-executor counter — `` `bash-${this.nextTaskId++}` `` in `packages/bash/bash-local/src/index.ts` — which gives it **exactly the same `name-N` shape as `SessionId`'s default** (`` `session-${++counter}` `` in `packages/core/session/src/index.ts`). A bash task id and a session id are trivially swappable at a call site and the compiler says nothing. This is the headline case the user asked about, and it is a model-facing id (the model passes `task_id` back to `bash_output`/`bash_kill`), so a confusion here is reachable from untrusted input. + +The bash **owner token** is the related sub-case: `BashExecRequest.owner?: string` and `BashExecSpec.owner: string | undefined` (`packages/bash/bash/src/types.ts`) are documented as a deliberately *opaque* isolation key, but in every live caller the value IS the owning agent's `session.header.id` (`callerToken = (exec) => exec.agent?.session.header.id` in `packages/bash/tool-bash/src/index.ts`) — i.e. a `SessionId` wearing a `string` disguise. It is compared for access control (`owner !== callerToken(exec)`), so a mismatched-but-well-typed string here is a cross-session isolation bug the type system currently cannot catch. This is the same `session.header.id`-as-owner alias that the [unify-the-agent-id-and-the-session-id](../../proposed/simplification/2026-06-20-unify-agent-and-session-id.md) proposal calls the "bash owner-token alias hole". + +**Gap 2 — brand erosion at the seams of the *already-branded* IDs.** Even `CallId`/`SessionId`/`AgentId` decay back to bare `string` at exactly the places confusion is most likely: the registry/store `Map` key types and most public method params. Representative sites: `SessionStore.store = new Map()` and `create`/`prepare(id?: string)`/`get(id: string)` (`packages/core/session/src/index.ts`); `AgentRegistry.store = new Map()` and `register`/`get(id: string)` (`packages/core/agent/src/index.ts`); `ToolPresenter.pending = new Map()` keyed by call id and `call(callId: string)`/`result(callId: string)` (`packages/ui/acp/src/index.ts`); the ACP session-id surface beyond the store map — `SessionRecord.sessionId: string`, `bySession = new WeakMap()`, `loadingIds = new Set()`, `requireSession(sessionId: string)`, and the exported `streamSessionEventUpdate(sessionId: string, …)` (`packages/ui/acp/src/index.ts`); and the persistence coordinator's `Map` keyed by session id (`packages/session-persistence/session-persistence/src/coordinator.ts`). A brand that is dropped at the `Map` key buys nothing on lookups — the value of the existing brands is partly unrealized. + +## Proposal + +A type-only change. Brands are zero-cost casts; nothing about runtime behavior, serialization, comparison, or the wire format changes. The work is in three parts, all honoring the existing "not every string" policy. + +- **Brand the bash task id.** Add `BashTaskId = Branded<'BashTaskId'>` plus its same-named factory in `packages/bash/bash/src/types.ts` (the package that *owns* the id), importing `Branded` from `@deepseek-ai/dsh-brand` exactly as `SessionId`/`AgentId` already do. The brand primitive lives in the dependency-free `dsh-brand` utility package precisely so `dsh-bash` can brand its ids by depending on it alone — it never pulls in `dsh-llm` (or `dsh-session`) just to reach `Branded`. Thread it through `BashTask.id`, the `BashExecutor` seam methods (`get`/`ownerOf`/`readOutput`/`kill`), the generation site in `dsh-bash-local` (brand the counter output once, at creation), and the `dsh-tool-bash` validate/access surface (`validateTaskId` returns a `BashTaskId`; `task_id` is branded at the tool boundary where the model's string arrives). + +- **Mint a distinct `OwnerToken` brand.** Add `OwnerToken = Branded<'OwnerToken'>` in `packages/bash/bash/src/types.ts`; type `BashExecRequest.owner` / `BashExecSpec.owner` / `BashExecutor.ownerOf` as `OwnerToken | undefined`. The `dsh-tool-bash` consumer casts the agent's `session.header.id` (a `SessionId`) into an `OwnerToken` at the boundary — the one place the two vocabularies meet. The bash seam never imports `dsh-session`. (Rationale in the next section.) + +- **Stop the brand erosion.** Propagate the existing brands to the `Map` key types and public method params listed under Gap 2 — `Map`, `get(id: SessionId)`, `Map`, `Map`, the ACP `SessionRecord.sessionId: SessionId` surface, the coordinator's `Map`. This is the larger mechanical share of the diff and the part that makes the *existing* brands actually load-bearing on lookups, not just on the struct fields. + +Illustrative shape (the factory pattern is identical to the three existing brands): + +```ts ignore-check +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** A background bash task handle (generated `bash-N` by the local executor). */ +export type BashTaskId = Branded<'BashTaskId'> +export function BashTaskId(id: string): BashTaskId { + return id as BashTaskId +} + +/** A bash task's opaque isolation key — the consumer's owner identity, NOT the bash seam's. */ +export type OwnerToken = Branded<'OwnerToken'> +export function OwnerToken(id: string): OwnerToken { + return id as OwnerToken +} +``` + +## Why a distinct OwnerToken brand (not SessionId) + +The obvious shortcut is to type `owner` as `SessionId` directly — it always *is* one. We reject that. The bash executor seam is a capability seam (interface `dsh-bash`, implementation `dsh-bash-local`, consumer `dsh-tool-bash`) and its owner token is *documented as deliberately opaque*: the executor "never interprets it (no access policy lives in the seam — that is the consumer's job)" (`packages/bash/bash/src/types.ts`). Typing the seam's field as `SessionId` would import `dsh-session`'s vocabulary into a package that must not know what an owner token *means* — it would couple a generic execution backend to the session model and contradict the opaque-token design. A sandboxed or remote executor that replaces `dsh-bash-local` should not inherit a session dependency. The distinct `OwnerToken` brand keeps the seam decoupled: `dsh-bash` knows only "an owner is some opaque branded token," and the `dsh-tool-bash` consumer — which already decides the access policy — is the single boundary that casts its `SessionId` into an `OwnerToken`. The brand still delivers the safety win (you cannot pass a `BashTaskId` or a raw string where an owner is expected) without the coupling. + +## Out of scope / possible extensions + +Kept deliberately narrow per the "not every string needs a brand" policy. Each of these is a plausible future brand, deferred with a reason, not a commitment: + +- **`ModelId`** (`GenerateOptions.model`, the `LlmService` adapter-registry key) — a real cross-package lookup key (config → agent → llm → adapter); a reasonable next brand, left out only to keep this RFC's blast radius focused. +- **`ToolName`** (the `ToolRegistry` key) — author-defined, human-readable, and rarely confused with another id; the weakest candidate, likely not worth a brand. +- **`ErrorCode`** (`HarnessError.code`) — a closed vocabulary (`ABORTED`, `NO_ADAPTER`, …), not a per-instance id; better served by a string-literal union than a brand, if anything. +- **Numeric ordinals** — turn number, step number, and the event `seq` are `number`, not `string`, so `Branded` does not apply; a parallel `number & { readonly [BRAND]: B }` variant could brand them, but they are positional ordinals rarely passed across boundaries, so the payoff is low. +- **Validated construction** — the brand factories are pure casts with no runtime check, and every boundary (ACP `sessionId`, provider-issued `call.id`, the empty-string fallback in `dsh-llm-deepseek`) trusts the raw string today. A `SessionId.parse()` / `isValid()` companion that throws on malformed input at boundaries is a genuine gap, but it is a *runtime-behavior* change with its own design (what is "malformed"? what do we do on failure?) and belongs in its own RFC, not bundled into this type-only pass. + +## Acceptance criteria + +- `BashTaskId` and `OwnerToken` are defined in `dsh-bash` and threaded end-to-end: the executor seam, the `dsh-bash-local` generation site, and the `dsh-tool-bash` model-facing surface all speak the brands; `dsh-bash` gains no dependency on `dsh-session`. +- No collection keyed by an in-scope branded id (`CallId`/`SessionId`/`AgentId`/`BashTaskId`) is keyed by bare `string` — this covers `Map`, `WeakMap` value slots, and `Set` membership (e.g. the ACP `bySession`/`loadingIds`), not just `Map`; the corresponding public method params and exported function signatures (e.g. `streamSessionEventUpdate`) take the brand, not `string`. +- Brands are constructed via the cast factory at each boundary where a raw string enters (provider call id, ACP session id, model-supplied `task_id`); no `as` casts scattered at call sites. +- `pnpm run typecheck` and `pnpm run doc-sync` are green; the change is observably type-only (no snapshot, no e2e behavioral diff). + +## Risks / what we give up + +- **Mechanical churn across two surfaces.** Propagating brands touches the bash seam (interface + impl + consumer) and the ACP session-id surface plus the persistence coordinator. The risk is broad but low-severity: a missed site is a compile error, not a silent bug. It ships as its own PR, converged with Codex, and stacks naturally near the [unify-the-agent-id-and-the-session-id](../../proposed/simplification/2026-06-20-unify-agent-and-session-id.md) work (both touch the session-id / owner-token boundary; if that proposal lands first, `OwnerToken` still stays distinct from the unified id for the decoupling reason above). +- **Brands do not validate.** A brand is a confusability guard, not a correctness proof: a *wrong* session id that is still a well-formed string passes the type checker exactly as before. This RFC does not close that gap (see Out of scope) — it only stops the *category* error of passing the wrong *kind* of id. +- **The "where to stop" line stays a judgment call.** Branding `BashTaskId` but not `ToolName`, `OwnerToken` but not `ModelId`, is a taste call about which strings "could plausibly be confused." Reasonable reviewers may want more or fewer; the policy in `brand.ts` is the tie-breaker, and this RFC errs toward the ids that are model-facing or used for access control. diff --git a/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md b/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md new file mode 100644 index 0000000000..b6c30819dd --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md @@ -0,0 +1,53 @@ +# RFC: Extract example apps into packages + +Status: implemented + +## Problem + +An example folder is supposed to be *thin* — the variable wiring of a demo, not the demo's machinery. Before this change it was thick. Each example carried a hand-rolled `start.ts` boot bootstrap, an infra preamble (`timer`, and — for the stdio demos — `logger` + `hmr`), nested includes of three shared YAML fragments (`base.yml` / `base-core.yml` / `acp-agent/acp-tail.yml`), and per-example `agent-loop`/persistence/system-prompt config. The actual app — the spine of services every agent needs — was spread across the leaf and those includes. + +The deeper problem was a **coupled front-door cluster** that lived at the leaf with nothing enforcing it. Choosing the ACP bridge over `ui-stdio` was not one swappable line: an ACP server must **drop the stdout console logger** (stdout is the JSON-RPC channel — a stray log corrupts the frames) and pre-create **no** agents (ACP `session/new` creates them on demand), whereas the stdio app needs a console logger and a pre-created `main`. (`timer` is the one infra plugin common to both — it writes nothing to stdout — so it belongs in the shared spine, not the cluster.) That coupling was enforced only by prose warnings in the leaf YAML. A leaf that wired a console logger into the ACP config was a one-line, comment-only mistake away — exactly the [stdout-purity footgun](../feature/2026-06-18-acp-terminal-and-tool-rendering.md) the examples guarded by hand. The three `start.ts` files also duplicated the Loader-boot tail, the `.env` loader, and (for ACP) snapshot-mode branching and the stdin-dispose lifecycle. + +## What shipped + +Each example is now **mostly an invocation of an app package**, splitting the wiring along the existing [interface / implementation / consumer seam](2026-06-13-capability-seams.md): the **app package owns the composition**, the leaf `cordis.yml` owns only the **swappable choices** (which LLM adapter, which bash executor, model, prompt, persistence root). + +- **`@deepseek-ai/dsh-agent-core`** ([packages/core/agent-core](../../../../packages/core/agent-core)) — a Cordis bundle plugin for the providerless, executor-less, UI-less spine: `timer` + `llm` + sessions + system-prompt + tools + agents + invariants + `tool-bash` + `agent-loop`, mounted as child plugins inside its `apply(ctx)` via `ctx.plugin(...)`. This is the old `base-core.yml` **minus** `bash-local`, **plus** `timer` and the loop, as code instead of a YAML include. The bundle **forwards** `agent-loop`'s `agents` list as its own config (`export const Config = AgentLoop.Config`, default `[]`, the existing `AgentLoop.Config` shape in [packages/core/agent-loop/src/index.ts](../../../../packages/core/agent-loop/src/index.ts)) — so each app supplies its own pre-created agents. This is precisely the reason the old `base-core.yml` gave for keeping `agent-loop` *out* of the shared core ("the examples disagree — stdio needs a pre-created `main`, acp needs none"); forwarding the config dissolves that objection — the loop is shared, the agents list is per-app. The bundle children register into the root service store, so a leaf-mounted sibling (the adapter, the executor) sees them exactly as a nested `plugin-include` subtree's services were seen before. Depending on the CONCRETE `dsh-agent-loop` (not just the `dsh-agent` interface) is deliberate and is the sanctioned exception to the "extension plugins depend on interfaces, never on the concrete loop" rule (packages/README.md, docs/architecture.md § Layering): the rule constrains plugins that EXTEND the system, whereas this bundle's whole job is to COMPOSE the concrete spine. Swapping the loop means publishing a different bundle, not rewiring every extension. +- **`@deepseek-ai/dsh-stdio-agent`** ([packages/ui/stdio-agent](../../../../packages/ui/stdio-agent)) and **`@deepseek-ai/dsh-acp-agent`** ([packages/ui/acp-agent](../../../../packages/ui/acp-agent)) — app packages, each consuming `dsh-agent-core` and **baking in its coupled front-door cluster**: stdio = `ui-stdio` + console logger + a pre-created `main`; acp = the `acp` bridge + JSONL persistence + **no stdout logger** + no pre-created agents. The leaf no longer carries the cluster, so it has no logger entry to copy wrong by default — the common stdout-purity mistake loses its foothold. (A leaf can still *add* a sibling logger entry — a package cannot forbid what a leaf author writes — so the rule "never add a stdout logger to an ACP leaf" stays documented at the leaf; what changed is that the default leaf has nothing to get wrong.) They land under the existing `ui` group alongside `acp`, so no new package group (and no `tsconfig`/`packages/README` group plumbing) was needed. +- **`start.ts` is gone.** Each app package exposes a `bin` (`dsh-stdio-agent` / `dsh-acp-agent`); the `demo:*` scripts invoke it (e.g. `dsh-stdio-agent ./cordis.yml`). The Loader-boot tail, `.env` loading, snapshot-mode selection, and stdin-dispose lifecycle moved into that bin, owned by the app. The `bin.ts` files are coverage-excluded (a self-executing CLI entry, like the old `start.ts`) and driven by the keyless Loader-path tests. +- **Each leaf `cordis.yml` collapses** to backends + config: the LLM adapter (`llm-deepseek` with apiKey/models, or `llm-replay`), the bash executor (`bash-local`), `hmr` for the stdio demos (see the amendment below), and one app entry carrying the app's config (model, system prompt, persistence root — surfaced as the app package's own `Config`, which routes each value to wherever the app wires it: stdio onto its pre-created agent, acp onto the bridge plugin). +- **echo-agent folds onto `dsh-stdio-agent`**, swapping the LLM backend to the local `mock-llm` and adding the local `echo-tool` (plus `bash-local`, which the spine's `tool-bash` injects) at the leaf — the clean demonstration of "swap the backend, keep the app". `mock-llm.ts` / `echo-tool.ts` stay as example-local teaching plugins. +- **`base.yml`, `base-core.yml`, and `acp-agent/acp-tail.yml` are retired** — the spine they shared now lives in `dsh-agent-core`. + +`bash-local` and the LLM adapter stay **leaf choices**: the bundle ships `tool-bash` (the consumer schema), the leaf picks the executor implementation, so a sandboxed executor or replay adapter swaps in without touching the app. + +### Amendment on implementation: `hmr` stays a leaf entry + +The proposal listed `hmr` among the stdio app's baked-in front-door cluster. Validating against the code, baking `hmr` into the `dsh-stdio-agent` package fights cordis in two ways, so it ships as a **leaf `cordis.yml` entry** instead: + +1. `@cordisjs/plugin-hmr` is a Loader-only, subprocess-only dev plugin — its constructor throws without `node --expose-internals` + a live `loader` service, so it can only run in the real `demo:*`/bin subprocess, never in the in-process unit/coverage tier. +2. The in-process test tier (vitest) cannot even *import* the vendored `hmr` module (its class-decorator `@Inject` form fails under Vite's transform), so a package whose `apply` statically imported it could never satisfy the per-file 100% coverage gate on its headline function. + +Crucially, `hmr` is **not** a stdout-purity footgun the way the console logger is — a stray `hmr` in the ACP config would not corrupt the JSON-RPC frames — so leaving it at the leaf costs none of the safety the coupling argument is about. The **logger** (the real coupling) stays baked in: the stdio app includes it, the ACP app omits it. + +## Why not keep the wiring in shared YAML includes? + +The old `base*.yml`/`acp-tail.yml` includes already deduped the *config*, but a YAML include cannot **encapsulate** the front-door coupling — it can only describe it in a comment and trust every leaf to obey. It also cannot own a `bin`, so the boot glue stayed copied across three `start.ts` files. A package turns "the ACP app never logs to stdout" from a prose warning into a property of the artifact: there is no logger entry in the leaf to get wrong. + +## Verification + +- Each example directory is `cordis.yml` (+ the acp `cordis.snapshot.yml`) + `README.md` + tests only — no `start.ts`, no infra preamble; `base.yml`/`base-core.yml`/`acp-tail.yml` are gone. +- `demo:echo` / `demo:coding` / `demo:acp` run via the app-package `bin`s. +- The new packages carry the per-file 100% coverage gate and a README like every `@deepseek-ai/dsh-*`. Each app package has a keyless **real-load-path** smoke that boots it through its `bin` + the cordis Loader (not a hand-built `ctx.plugin({...})` mount), guarding the `unwrapExports` export-shape bug class ([postmortem 0001](../../../postmortem/0001-acp-default-export-drops-inject.md)). +- The ACP snapshot **replay** transcript is unchanged: the boot restructuring preserved the plugin set + load order, so `pnpm run test:snapshot` stays green against the committed goldens with no re-record. + +## What we give up + +- **The bare-plugin-tree pedagogy.** echo-agent's inlined `cordis.yml` showed every plugin at once; the spine now lives behind a bundle, so seeing the whole tree means opening `dsh-agent-core`. The app package's README carries that teaching weight. +- **A layer of indirection.** "What does this demo load?" becomes a package read, not a single YAML scan. + +## Related + +- Supersedes [Make the shared example base providerless](../../rejected/architecture/2026-06-20-providerless-example-base.md): renaming `base.yml` to the providerless core is moot once the spine moves into `dsh-agent-core` and the `base*.yml` files are deleted. +- Builds on the [capability-seams](2026-06-13-capability-seams.md) interface/implementation/consumer split — backends and presentation stay leaf choices; the spine is the shared bundle. +- Complements [Reorganize packages into a modular hierarchy](2026-06-20-package-hierarchy.md): the new app/core packages slot into existing groups under that hierarchy (`core` for the reusable spine bundle, `ui` for the app-specific front doors). diff --git a/docs/rfc/implemented/architecture/2026-06-20-package-hierarchy.md b/docs/rfc/implemented/architecture/2026-06-20-package-hierarchy.md new file mode 100644 index 0000000000..8b198ed9c6 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-20-package-hierarchy.md @@ -0,0 +1,67 @@ +# RFC: Reorganize packages into a modular hierarchy + +Status: implemented + +## Problem + +`packages/` was flat: 18 packages all sat at `packages//`, so a package's location said nothing about whether it was core product API, a swappable capability seam, a provider adapter, a product integration, or example/test support. The package README carried a `FIXME(package-hierarchy)` and `scripts/publint-all.ts` a `TODO(package-inventory)` flagging exactly this. Core packages, provider integrations, capability seams, example UI support, and snapshot-only replay support all looked equally foundational. + +This was not just cosmetic. Because every top-level package looked like part of the same public surface, future removal was harder, and publish/lint/doc scripts had to encode intent through comments or hand-maintained static lists rather than reading it off the layout. + +## What landed + +Packages are grouped by modular role at a uniform `packages///` depth. Group directories are pure containers (no `package.json`); every package keeps its `@deepseek-ai/dsh-` name — this is repo structure and maintenance policy, not package renaming. + +```text +packages/ + core/ (product API spine) + session/ + system-prompt/ + tools/ + agent/ + agent-loop/ + llm/ (product — capability family) + llm/ + llm-deepseek/ + llm-pi-ai/ + bash/ (product — capability family) + bash/ + bash-local/ + tool-bash/ + session-persistence/ (product — capability family) + session-persistence/ + session-persistence-jsonl/ + session-persistence-sqlite/ + ui/ (product integration) + acp/ + support/ (dev/test/example infrastructure) + invariants/ + ui-stdio/ + llm-replay/ +``` + +### Placement decisions + +- **Same-name nesting for capability families.** A family's interface package sits at `packages///` (`llm/llm`, `bash/bash`, `session-persistence/session-persistence`), with implementations and consumers as flat siblings. There is no extra `adapters/`/`impls/` sub-tier — every package is exactly depth 2, which keeps the workspace glob a clean `packages/*/*` and lets one `@deepseek-ai/dsh-*` tsconfig wildcard resolve every package (unique dir names make first-on-disk-wins unambiguous). +- **`session` stays in `core/`; persistence is its own family.** The session log is core product API. Its storage backends form a parallel capability family (`session-persistence/`) mirroring `llm/` and `bash/`, rather than nesting under `core/session/`. +- **`agent-loop` is in `core/`.** It is the one concrete implementation of the `agent` seam, but it ships as the harness's default product loop, so it lives with the core spine. Plugins still depend on the `agent` vocabulary, never on `agent-loop`, so the loop stays swappable. +- **`invariants` and `ui-stdio` are `support/`, not product.** `invariants` is dev-mode contract checking. `ui-stdio` was extracted from the examples for reuse and the coverage gate — it is example-coupled, so it sits in `support/` alongside `llm-replay` (the snapshot-test replay adapter). `acp` is the only `ui/` member because it is a real product surface (the ACP bridge an editor drives), structurally distinct from the readline demo helper. + +### Deduplicating the package lists + +The package list had been enumerated in five places. The uniform depth-2 layout lets most of them be derived instead: + +- `tsconfig.base.json` maps every package through a single `@deepseek-ai/dsh-*` `paths` wildcard listing one candidate per group, in place of per-package entries. Root `tsconfig.json` reuses that source map and carries the explicit project references that keep package/vendor typecheck boundaries intact. (One subtlety this introduced: a path candidate contains `/*/`, which a naive regex comment-stripper mistakes for a block comment — `scripts/doc-typecheck.ts` reads the JSONC config through TypeScript's parser rather than stripping comments by hand for exactly this reason.) +- `scripts/publint-all.ts` derives its list by reading the hierarchy (`packages//`), resolving the `TODO(package-inventory)`. +- `tsconfig.build.json`'s project `references` stay an explicit list — TypeScript project references have no wildcard form. Generating these from a manifest is left to a follow-up (see [discover package inventories](../../proposed/process/2026-06-20-discover-package-inventory.md)). + +### Guardrails added + +Two doc-sync/hygiene gates keep the structure and its references honest, so the manual checks this restructure required do not have to be repeated by hand: + +- `scripts/verify-package-paths.ts` flags a `packages/` reference (in Markdown or a `.ts` comment/string) that does not resolve **and** names a real package in a segment — i.e. a stale path to a moved package. A path naming a package that exists nowhere (a forward-looking proposal) is left alone, so the gate applies uniformly across proposed/implemented/rejected. +- `scripts/check-workspace-constraints.ts` asserts the `packages//` shape: group dirs carry no `package.json`, and no package sits flat at the root or nests deeper. Group names stay open — a new group may be added without editing the gate; only the depth-2 shape is fixed. + +## What we gave up + +The restructure churned imports, workspace globs, doc links, build references, and package paths in one coordinated move. That churn is acceptable pre-release (per the AGENTS.md foundation-over-blast-radius stance) because it stops the flat layout from fossilizing support packages as product contracts, and it is a one-time cost: the wildcard `paths`, the glob-derived publint list, and the shape gate mean a new package needs no further structural edits. diff --git a/docs/rfc/implemented/2026-06-18-acp-terminal-and-tool-rendering.md b/docs/rfc/implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md similarity index 95% rename from docs/rfc/implemented/2026-06-18-acp-terminal-and-tool-rendering.md rename to docs/rfc/implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md index 3ff014f55f..6e108dc4c1 100644 --- a/docs/rfc/implemented/2026-06-18-acp-terminal-and-tool-rendering.md +++ b/docs/rfc/implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md @@ -4,7 +4,7 @@ Status: implemented ## Problem -The ACP bridge lets each tool own its call rendering via `presentCall`/`presentResult` (see [tool-call UI presentation](../proposed/2026-06-14-acp-agent-client-protocol.md) and `packages/tools`). For `bash` we surface the exact command as the `tool_call` title, the model's `description` as a content text block, `kind: 'execute'`, and the completed output wrapped in a fenced ` ```console ` text block. +The ACP bridge lets each tool own its call rendering via `presentCall`/`presentResult` (see [tool-call UI presentation](../../proposed/feature/2026-06-14-acp-agent-client-protocol.md) and `packages/core/tools`). For `bash` we surface the exact command as the `tool_call` title, the model's `description` as a content text block, `kind: 'execute'`, and the completed output wrapped in a fenced ` ```console ` text block. That is a correct, capability-free baseline, but not how the reference editors render a *terminal* tool at its best. An editor like Zed has a dedicated terminal tool-call card — a header showing the working directory, the command as the label, the command output rendered as a terminal, and an exit-status pill — but it only builds that card when the `tool_call` carries terminal metadata (below). With a plain text block the output appears as static markdown and there is no cwd header. (Zed also HIDES `rawInput` for `kind: 'execute'`, which is why the command IS the title — both reference adapters do the same. The human-readable description rides as a separate content block above the card; note this is a DELIBERATE divergence — claude-agent-acp DROPS the description in terminal mode and renders only the card — we keep the summary visible alongside.) diff --git a/docs/rfc/implemented/2026-06-11-doc-sync-enforcement.md b/docs/rfc/implemented/process/2026-06-11-doc-sync-enforcement.md similarity index 69% rename from docs/rfc/implemented/2026-06-11-doc-sync-enforcement.md rename to docs/rfc/implemented/process/2026-06-11-doc-sync-enforcement.md index 68e6f0a03d..69fca053f3 100644 --- a/docs/rfc/implemented/2026-06-11-doc-sync-enforcement.md +++ b/docs/rfc/implemented/process/2026-06-11-doc-sync-enforcement.md @@ -12,10 +12,10 @@ AGENTS.md promises that docs and code stay strictly in sync, but the promise was Two gates, mirroring the existing `scripts/` style (tsx ESM, one job each): -1. **`doc-typecheck`** extracts every fenced ` ```ts ` block from `README.md`, `docs/**`, and `packages/*/README.md`, writes them to a temp project, and compiles with `tsc --noEmit`. The temp tsconfig copies only resolution-relevant options and the workspace `paths` map from `tsconfig.typecheck.json` (vendor → built `lib`, harness → `src`) — resolving vendor to `lib` is essential, or tsc type-checks raw vendor source and floods the run. A block that is a deliberate sketch opts out with an explicit ` ```ts ignore-check ` info string; the script reports the opt-out ratio and fails if it exceeds half, so the escape hatch can't quietly become the norm. -2. **`verify-event-taxonomy`** extracts the event names from the `interface Events` blocks across `packages/*/src` and from the taxonomy table in `docs/architecture.md`, and asserts the two sets match exactly. Verify, don't generate: the table keeps its hand-written Mode/Purpose columns; only the set of names is checked. (Landing this surfaced three events the table had been missing — `tools/change`, `llm/adapter-change`, `system-prompt/change`.) +1. **`doc-typecheck`** extracts every fenced ` ```ts ` block from `README.md`, `docs/**`, and `packages/*/README.md`, writes them to a temp project extending the root `tsconfig.json`, and compiles it with `tsc -b`. The temp project reuses the source `paths` map and the root project references, so documentation examples see source while vendored code remains checked under its own tsconfig settings. A block that is a deliberate sketch opts out with an explicit ` ```ts ignore-check ` info string; the script reports the opt-out ratio and fails if it exceeds half, so the escape hatch can't quietly become the norm. +2. **`verify-event-taxonomy`** extracts the event names from the `interface Events` blocks across `packages/*/src` and from the taxonomy table in `docs/architecture.md`, and asserts the two sets match exactly. Verify, don't generate: the table keeps its hand-written Mode/Purpose columns; only the set of names is checked. (Landing this surfaced three events the table had been missing — `tools/change`, `llm/adapter-change`, `system-prompt/change`.) **Superseded** by [the generated cordis catalog](2026-06-20-generated-cordis-catalog.md): this gate and its `architecture.md` table are retired in favor of a fully-generated `docs/cordis-catalog/events-and-services.md` and its `verify-cordis-catalog` freshness gate. The other gates here (`doc-typecheck`, and the `verify-md-wrap` amendment below) are unaffected. -Both run via a shared `doc-sync` package.json script that the lefthook pre-push hook and CI both invoke ([mechanical quality gates](2026-06-11-quality-gates.md): hooks and CI call the same scripts, so the gate fires locally before a push — not only after it). They run after `pnpm run typecheck` (which emits the vendor `lib/` that doc-typecheck resolves against). API-extractor golden reports ([the deferred API-extractor-reports proposal](../proposed/2026-06-11-api-extractor-reports.md)) were deliberately **deferred** — low value for an internal monorepo where reviewers already see the source diff, and a heavy, finicky dependency. +Both run via a shared `doc-sync` package.json script that the lefthook pre-push hook and CI both invoke ([mechanical quality gates](2026-06-11-quality-gates.md): hooks and CI call the same scripts, so the gate fires locally before a push — not only after it). They run after `pnpm run typecheck`, which validates the package/vendor build graph that doc-typecheck references. API-extractor golden reports ([the deferred API-extractor-reports proposal](../../proposed/process/2026-06-11-api-extractor-reports.md)) were deliberately **deferred** — low value for an internal monorepo where reviewers already see the source diff, and a heavy, finicky dependency. **Amendment (2026-06-17):** a third gate, **`verify-md-wrap`**, was later folded into `doc-sync`. It parses each in-scope Markdown file (`README.md`, `docs/**`, `packages/*/README.md`, plus `AGENTS.md` / `packages/AGENTS.md`) with `mdast-util-from-markdown` + GFM and fails on any `paragraph` node spanning more than one source line, enforcing the AGENTS.md "Markdown is not hard-wrapped" convention. Same verify-don't-generate principle: it reports hard-wraps and never rewrites, so it adds no formatting churn. `doc-sync` is now three gates. diff --git a/docs/rfc/implemented/2026-06-11-quality-gates.md b/docs/rfc/implemented/process/2026-06-11-quality-gates.md similarity index 82% rename from docs/rfc/implemented/2026-06-11-quality-gates.md rename to docs/rfc/implemented/process/2026-06-11-quality-gates.md index 1f3dd3c520..277a56fa2b 100644 --- a/docs/rfc/implemented/2026-06-11-quality-gates.md +++ b/docs/rfc/implemented/process/2026-06-11-quality-gates.md @@ -12,14 +12,14 @@ This codebase is developed primarily by coding agents. Agents follow enforced ga Every AGENTS.md promise gets a command that exits non-zero, wired into git hooks and CI both calling the same package.json scripts: -- Max-strict TypeScript (`noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, …); tests and examples typecheck in CI via `tsconfig.typecheck.json` (vendored packages resolve as built declarations). +- Max-strict TypeScript (`noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, …); examples, tests, and scripts typecheck in CI via the root no-emit `tsconfig.json` while package/vendor code stays behind its own project-reference boundary. - ESLint strict-type-checked + @stylistic (the house style, enforced); vendored code excluded. - Per-file 100% coverage on `packages/*/src` (v8); unreachable defensive guards carry `/* v8 ignore */ ` with stated reasons instead of deletion. -- knip (dead code/deps), publint (package correctness), workspace constraints (workspace rules: private, cordis peer+dev, uniform version, ESM). +- knip (dead code/deps), publint (package correctness), workspace constraints (workspace rules: private, cordis peer+dev, uniform version, ESM), and a NodeNext consumer typecheck for built package declarations. - lefthook pre-commit (lint staged, typecheck, vendor-manifest guard) and pre-push (tests, hygiene); CI runs the full matrix on node 24/26 plus a demo smoke test driving the echo-agent end to end. ## Consequences - Conventions survive agent turnover; violations fail fast and locally. - The gates themselves are code to maintain; config changes are reviewed like any change. -- 100%-coverage pressure can produce assertion-free tests — mutation testing is the planned counterweight (see [the mutation-testing proposal](../proposed/2026-06-11-mutation-testing.md)). +- 100%-coverage pressure can produce assertion-free tests — mutation testing is the planned counterweight (see [the mutation-testing proposal](../../proposed/testing/2026-06-11-mutation-testing.md)). diff --git a/docs/rfc/implemented/2026-06-11-tsdown-over-dumble.md b/docs/rfc/implemented/process/2026-06-11-tsdown-over-dumble.md similarity index 60% rename from docs/rfc/implemented/2026-06-11-tsdown-over-dumble.md rename to docs/rfc/implemented/process/2026-06-11-tsdown-over-dumble.md index 283b236397..dc028d7e9b 100644 --- a/docs/rfc/implemented/2026-06-11-tsdown-over-dumble.md +++ b/docs/rfc/implemented/process/2026-06-11-tsdown-over-dumble.md @@ -15,12 +15,12 @@ Build output currently matters only for `pnpm run build` + publint (nothing publ Replace dumble with **tsdown** (rolldown-based, ~2.5M downloads/week, VoidZero-backed, actively released): - Root `tsdown.config.ts` with `workspace: ['vendor/*', 'packages/*']` (explicit globs, not `workspace: true`, which would also pick up `examples/*` — they have package.json files but are not pnpm workspaces). -- Shared shape: entry `src/index.ts`, `outDir: 'lib'`, ESM, `platform: node`, `target: es2024`, `fixedExtension: false` (keeps `.js` for `"type": "module"` packages), `dts: false` (tsc -b owns declarations), `clean: false` (lib/ holds tsc's .d.ts output). +- Shared shape: entry `lib/types/index.js`, `outDir: 'lib'`, ESM, `platform: node`, `target: es2024`, `fixedExtension: false` (keeps `.js` for `"type": "module"` packages), `dts: false` (tsc -b owns declarations), `clean: false` (lib/ also holds TSC's `lib/types` intermediate tree). The entry was originally `src/index.ts`; the [TSC-first build RFC](2026-06-17-ts-build-config.md) later moved tsdown to bundling TSC-emitted JS so TypeScript transform behavior comes from one compiler. - Two per-package overrides in vendor/ (ours, like the regenerated tsconfigs; logged in vendor/README.md): schemastery (dual `.mjs`/`.cjs` via `outExtensions`), logger-console (two single-entry passes so the shared base class is inlined into each entry instead of a hash-named chunk, matching upstream's published shape). -- `scripts/build.ts` deleted; `pnpm run build` = `tsc -b && tsdown`. +- `scripts/build.ts` deleted; `pnpm run build` = `tsc -b tsconfig.build.json && tsdown`. Alternatives considered: **direct esbuild script** (most established engine, zero wrapper risk, but hand-maintains the per-package spec table tsdown's workspace mode gives us); **pkgroll** (closest drop-in philosophically, but 78k dl/wk and Rollup-based — strictly weaker maintenance story than tsdown); **keep dumble** (perfect upstream alignment, unacceptable bus factor). ## Consequences -Output file lists are byte-for-byte-list identical to dumble's (verified by snapshot diff at migration time); externals still come from each package's dependencies/peerDependencies. We give up dumble's exports-field inference — new packages with non-default shapes need a per-package `tsdown.config.ts` instead of just package.json fields. Future option: tsdown could also absorb declaration bundling (isolatedDeclarations) if `tsc -b` ever becomes the bottleneck; that would be a new RFC. +Runtime bundle outputs still follow the dumble-era public entry shape (`lib/index.js`, plus package-specific variants such as `schemastery`'s `lib/index.mjs`/`lib/index.cjs` and `logger-console`'s `lib/browser.js`); declarations now live under `lib/types` per the [TSC-first build RFC](2026-06-17-ts-build-config.md). Externals still come from each package's dependencies/peerDependencies. We give up dumble's exports-field inference — new packages with non-default shapes need a per-package `tsdown.config.ts` instead of just package.json fields. Future option: tsdown could also absorb declaration bundling (isolatedDeclarations) if `tsc -b` ever becomes the bottleneck; that would be a new RFC. diff --git a/docs/rfc/implemented/2026-06-11-vendor-cordis-as-source.md b/docs/rfc/implemented/process/2026-06-11-vendor-cordis-as-source.md similarity index 100% rename from docs/rfc/implemented/2026-06-11-vendor-cordis-as-source.md rename to docs/rfc/implemented/process/2026-06-11-vendor-cordis-as-source.md diff --git a/docs/rfc/implemented/2026-06-16-pnpm-over-yarn.md b/docs/rfc/implemented/process/2026-06-16-pnpm-over-yarn.md similarity index 100% rename from docs/rfc/implemented/2026-06-16-pnpm-over-yarn.md rename to docs/rfc/implemented/process/2026-06-16-pnpm-over-yarn.md diff --git a/docs/rfc/implemented/process/2026-06-17-ts-build-config.md b/docs/rfc/implemented/process/2026-06-17-ts-build-config.md new file mode 100644 index 0000000000..a45962c70e --- /dev/null +++ b/docs/rfc/implemented/process/2026-06-17-ts-build-config.md @@ -0,0 +1,73 @@ +# RFC: TSC-first build and one tsconfig + +Status: implemented (accepted 2026-06-20) + + + +## Context + +The current TypeScript build and typecheck setup had these issues: + +- `build` used `tsc` to transform `.ts` to `.d.ts` files for packages under `packages//` and `vendor/*`, and then used `tsdown` to transform `.ts` to bundled `.js` files. This made two tools do TypeScript transform. +- `typecheck` tended to validate packages, vendor source, examples, tests, and scripts through one root typecheck config. + +The goal is to make build and typecheck use matching tsconfig boundaries and TypeScript resolution/transform behavior. Build should generate `.js`, `.d.ts`, `.js.map`, and `.d.ts.map` through one compiler and config, so publish output and type validation stay consistent. + +Validation found several concrete technical issues and possible routes: + +- `tsdown` uses `oxc` to transform TypeScript, which is not the same behavior as `tsc`. + - Bundled `.d.ts` emitted by `tsdown` conflicts with Cordis' internal relative module augmentation shape. + - The tsc output is affected by `allowImportingTsExtensions`, so we need to ensure that generated `.js` files do not import `.ts` files and generated `.d.ts` files keep explicit relative specifiers that NodeNext/Node16 accepts. Therefore, in-package relative imports use explicit `.ts` specifiers in TypeScript source and `rewriteRelativeImportExtensions` rewrites those specifiers to `.js` in emitted JS. + - Bundled `.js` emitted by `tsdown` is not the same behavior as per-file `.js` emitted by `tsc -b`, such as decorator transform behavior. +- `vendor/*/src`, examples, tests, and scripts cannot all be plain-included in one root strict program. + - Directly typechecking `vendor/*/src` under the root strict config triggers many type errors outside this project's ownership. + - Package dependencies under `packages/*/*` on `vendor` are resolved to the `vendor/*/lib` for different tsconfig strictness. + + +## Decision + +In-package relative imports use explicit `.ts` specifiers. + +`pnpm run build` is a two-stage build: + +- Stage 1: `tsc -b tsconfig.build.json` emits per-module `.js`, declarations `.d.ts`, JS sourcemaps `.js.map`, and declaration sourcemaps `.d.ts.map` into each package's `lib/types`. This is the authoritative TypeScript compilation result. For publish we keep `.d.ts` / `.d.ts.map` and ignore `.js` / `.js.map`. + - The build project uses the project-reference graph that `tsc -b` compiles. For example, root `tsconfig.build.json` references package and vendor tsconfigs. It validates and emits package/vendor build results. +- Stage 2: a bundler reads the emitted JS under `lib/types` and writes the bundled runtime entry as `lib/index.js` or `lib/index.mjs` (follow current behavior). This stage is bundling only. It must not read TypeScript source or emit declarations. + +`tsdown` is no longer the owner of TypeScript compilation or declaration output. + +`pnpm run typecheck` runs build mode over the root `tsconfig.json`. +- The root `tsconfig.json` is the single development/typecheck project. It typechecks examples, tests, and scripts with `noEmit`, and validates package/vendor source through references. +- Referenced package/vendor projects keep the same emit behavior as build, so typecheck can refresh their `lib/types` outputs instead of using a separate no-emit graph. Project-specific strictness changes live in the owning `packages/*/*/tsconfig.json` or `vendor/*/tsconfig.json`. +- The root no-emit project disables `rewriteRelativeImportExtensions`; it emits nothing and includes tests that import helpers across project-reference boundaries. Package/vendor emit projects keep the rewrite enabled. + +The command orchestration shape is: + +```sh +pnpm run build: +tsc -b tsconfig.build.json +tsdown + +pnpm run verify-node-next-types: +tsx scripts/verify-node-next-types.ts + +pnpm run typecheck: +tsc -b tsconfig.json +``` + +`pnpm run demo:*` still runs `src` directly through tsx and root paths, without a compile step. + +## Consequences + +Build responsibilities are clearer: + +- Each module under `packages//` and `vendor/*` has one local tsconfig for build, typecheck, and tools that run source directly, such as `tsx` and `vitest`. +- The `build` command uses `tsconfig.build.json`. `tsc -b` owns the publishable per-module `.js` and `.d.ts` output, and the bundler owns only `lib/index.*`. + - `lib/types/*.d.ts` and `.d.ts.map` are the publish declaration output. + - `lib/types/*.d.ts` uses explicit `.ts` relative specifiers, which TypeScript's NodeNext/Node16 resolver maps to sibling `.d.ts` files. + - `lib/types/*.js` is only a bundler input and must not be used as a runtime entry or public import target. + - `lib/index.*` is the publish runtime output and is generated by the bundler, currently `tsdown`. +- `pnpm run verify-node-next-types` scans built declarations for relative specifiers without file extensions, then typechecks a temporary external ESM consumer with `moduleResolution: "NodeNext"` against the built `types`/`exports` surface, so declaration specifier regressions fail before publish. +- The `typecheck` command uses `tsconfig.json`. Examples, tests, and scripts are checked by the root no-emit project, while packages and vendor modules keep the same emit behavior as `build`. Package and vendor source stays behind project-reference boundaries. + +The Cordis vendor copy now has one more type-structure divergence from upstream. During upstream sync, that divergence must be reapplied or explicitly retired. diff --git a/docs/rfc/implemented/2026-06-18-markdown-cross-link-lint.md b/docs/rfc/implemented/process/2026-06-18-markdown-cross-link-lint.md similarity index 97% rename from docs/rfc/implemented/2026-06-18-markdown-cross-link-lint.md rename to docs/rfc/implemented/process/2026-06-18-markdown-cross-link-lint.md index 8bbef405f1..c4c0ca5afb 100644 --- a/docs/rfc/implemented/2026-06-18-markdown-cross-link-lint.md +++ b/docs/rfc/implemented/process/2026-06-18-markdown-cross-link-lint.md @@ -25,4 +25,4 @@ This gate checks *existence*, not anchor validity: a link to a real file with a - Renames and moves that orphan a cross-link now fail the pre-push hook and CI instead of waiting for a reader to click a dead link. This made the RFC reorganization that introduced the gate self-verifying: the same PR that rewrote forty links also added the check that proves none dangle. - One more fast tsx script in the `doc-sync` chain; no new dependency (the mdast/GFM stack is already in devDependencies for `verify-md-wrap`). - Fragment/anchor validity remains unchecked — a known, deliberate scope cut. -- The convention this enforces — cross-reference docs by machine-checkable relative link, never by bare prose or a number — is documented in [docs/AGENTS.md](../../AGENTS.md) so authors know the gate exists and why. +- The convention this enforces — cross-reference docs by machine-checkable relative link, never by bare prose or a number — is documented in [docs/AGENTS.md](../../../AGENTS.md) so authors know the gate exists and why. diff --git a/docs/rfc/implemented/process/2026-06-20-core-data-structures-catalog.md b/docs/rfc/implemented/process/2026-06-20-core-data-structures-catalog.md new file mode 100644 index 0000000000..d589f6136b --- /dev/null +++ b/docs/rfc/implemented/process/2026-06-20-core-data-structures-catalog.md @@ -0,0 +1,56 @@ +# RFC: Core-data-structures catalog and the `ts type-equiv` drift gate + +Status: implemented (accepted 2026-06-20) + + + +## Context + +A reader trying to understand the harness could find its *behavior* in [architecture.md](../../../architecture.md) (the service map, the session/turn/step lifecycle, the event taxonomy) but had no single place describing its *vocabulary* — the data structures that behavior moves around. The type shapes lived only in source, scattered across `packages/*/src/types.ts`, so understanding "what is a `Message`, a `SessionEvent`, a `StreamChunk`" meant reading the declarations directly. A prose catalog would help, but a catalog that paraphrases or paste-copies type definitions rots the instant a field changes — and an out-of-sync type doc is worse than none, because a reader trusts it. + +So the work had two intertwined questions: **what belongs in such a catalog** (the scoping problem — a harness has dozens of cross-package types and dumping all of them helps no one), and **how to keep pasted type definitions from drifting** (the durability problem). This RFC records both decisions. Its sibling, [the generated cordis events + services catalog](2026-06-20-generated-cordis-catalog.md), is the *wiring*-axis complement: this one catalogs the data structures, that one the events and services that move them. + +## Decision + +A new `docs/core-data-structures/` folder catalogs the vocabulary, with a new `verify-type-equiv` doc-sync gate that keeps every pasted type definition byte-identical to its source. + +### What counts as "core" — the spine-vs-seam line + +The scoping line was not picked top-down; it was discovered by testing candidate definitions against concrete borderline types until one rule survived every case. The decisive test was `BashExecRequest`/`BashExecSpec`/`BashRunResult`: bash is a capability *seam*, not part of the agent-loop spine, so if those are "core" then "core" means *all cross-package vocabulary* and the catalog is a flat dump; if they are not, "core" means *the central spine* and bash vocabulary belongs on a sub-page. The latter won, which set the whole structure: a **tiered folder**, not a flat document. + +The rule that settled the remaining cases: ***the type you write, hold, or receive is core; the machinery that types it, renders it, or persists it is a sub-page detail.*** Worked through: + +- A data structure is **core** if it flows through the agent-loop spine — the loop holds, derives, streams, or logs it on every turn regardless of which plugins load (`Message`, `StreamChunk`, `SessionEvent`, the `Agent` handle) — **or** it is the single headline type a plugin author writes against a pipeline (`ToolDefinition`). +- `ToolDefinition` is core (it is what every tool author writes) **even though the loop never holds one** — authoring-importance overrides the strict flows-through-spine rule for this one headline type. But its typing machinery — the `SchemaSpec`/`InferArgs` DSL — is a sub-page detail (you write a `ToolDefinition`; the type-level machinery that types it you do not). That is the spine-vs-seam line made sharp. +- `ToolSchema` is core (it is a field of `GenerateOptions`, the model request that flows through every step) even though it is conceptually part of the tool pipeline — *flows through the spine* wins over *conceptual home* when they conflict. +- The tool-presentation vocabulary (`ToolCallPresentation`, …, carrying a `FIXME(tool-presentation)` redesign marker), the `SessionPersistence` durability seam, and bash vocabulary are sub-pages. + +`core.md` is a **self-contained spine doc**: it states the exact type definition of each spine structure with minimal prose and links to sub-pages for the per-seam detail. The sub-pages are `llm-streaming.md`, `session.md`, `persistence.md` (split from session along the in-memory-model vs. durability-seam line), `tools.md`, and `bash.md`. + +### The `ts type-equiv` mechanism — literal AND drift-proof + +The durability requirement was specific: the doc should show the **literal** current type definition (so a reader sees the real shape, not a paraphrase) **and** be mechanically guaranteed to match source. The repo already compiles fenced ` ```ts ` blocks (`doc-typecheck`), but a real typechecked block needs import noise and proves only *assignability*, not *byte-equality* — a renamed field with the same type would pass. So: + +- Type definitions are pasted verbatim into a dedicated ` ```ts type-equiv ` fence. `doc-typecheck` recognizes the fence and skips it (a bare definition is not standalone-compilable), and **excludes it from the opt-out ratio** — it is a separately-checked category, not an unchecked sketch. +- A new `scripts/verify-type-equiv.ts` extracts each block via the TypeScript parser and asserts a **verbatim source match** against the declared symbol — chosen over a compiled `_Check` assertion precisely because byte-equality, not assignability, is the property we want. +- Provenance lives in a central `scripts/type-equiv.manifest.json` (`{ doc, symbol, source }` entries), **not** in directive comments in the prose. The script enforces a **1:1 correspondence**: every type-equiv block has exactly one manifest entry and vice versa, so a block can never be silently unchecked and an entry can never rot. +- Wired into `doc-sync`, so it runs in the same lefthook pre-push and CI paths as the other doc gates. + +### Maintenance is the author's job, with a gate backstop + +`verify-type-equiv` catches a *drifted paste* of an already-documented type, but it cannot tell you a brand-new core type went undocumented. So AGENTS.md and the `dsh-code-review` skill were updated to require keeping the catalog in sync when a change adds or reshapes a documented type — the gate handles drift, the human handles new surface. + +## Process + +The design was driven entirely by a one-question-at-a-time grilling that walked the scoping decision tree through concrete examples (`BashExecRequest`, `ToolSchema`, `ToolDefinition`, the schema DSL, the presentation types, the session/persistence split) before committing to the spine-vs-seam rule — the rule was the *output* of the examples, not an a-priori axiom. The implementation landed as four commits mirroring the structure of the work: the gate (`e97f94b`), the catalog (`7e33c7b`), the maintenance-guard updates (`53e01a0`), and a review-fix commit (`6da7a0f`). + +That last commit is why the process is worth recording: an independent Codex review (gpt-5.5:xhigh) found a real **scan-gap bug** — `verify-type-equiv` only scanned the docs the manifest named, so a type-equiv block added to an *unmanifested* doc was silently skipped, defeating the 1:1 guarantee in one direction. The fix scans every doc in the markdown scope and reports an unmanifested block as an orphan. The same review corrected a `SessionPersistence` surface-listing prose error (`has`/`delete`) and the `doc-sync` command summary. The bug is the point: a drift gate that silently skips part of its input is worse than no gate, and only an adversarial reader caught it. + +This decision shipped in #71 **without** an RFC at the time — the judgment was that the `ts type-equiv` convention was small enough to document in `development.md`. This RFC is the retroactive record: the spine-vs-seam scoping rule and the verbatim-match-over-assignability choice are exactly the kind of "why was it done this way?" decisions a future maintainer would otherwise re-litigate, and its sibling catalog ([generated cordis events + services](2026-06-20-generated-cordis-catalog.md)) does carry an RFC, so the pair should be documented symmetrically. + +## Consequences + +- The vocabulary now has a single home that **cannot silently drift**: a field rename in source fails `verify-type-equiv` in the pre-push hook and CI until the paste is refreshed. +- The spine-vs-seam line is a reusable scoping tool, not a one-off: the same "the thing you write/hold/receive is core; the machinery that types/renders/persists it is a detail" rule is what later scoped the events/services catalog's harness-vs-inherited tiering. +- The `ts type-equiv` fence is a third doc-block category alongside ` ```ts ` (compiled) and ` ```ts ignore-check ` (sketch). A later sibling added a fourth, ` ```ts cordis-catalog ` (generated signature), reusing the same skip-and-exclude treatment. +- Adding or reshaping a core type now carries a documentation obligation the author must honor (the gate cannot detect a missing *new* type), backstopped by the `dsh-code-review` checklist. diff --git a/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md b/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md new file mode 100644 index 0000000000..2801ae209c --- /dev/null +++ b/docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md @@ -0,0 +1,35 @@ +# RFC: Generated cordis events + services catalog + +Status: implemented (accepted 2026-06-20) + + + +## Context + +A plugin author needs two reference surfaces that no single document gave them: every cordis **event** they can listen to (with its exact signature and dispatch mode) and every `ctx.` **service** they can call (with its exact interface). The pieces existed but were scattered — a hand-maintained event-taxonomy *table* in `docs/architecture.md` (names + prose Mode/Purpose, name-set-checked by `verify-event-taxonomy`), a Service-map table (8 rows of role prose), and the `interface Events` / `interface Context` declarations themselves. The taxonomy table also could not catch a brand-new *undocumented* event: a name-set verifier only checks the names that are already in the table on both sides. + +This is the wiring-axis complement to the [core-data-structures catalog](../../../core-data-structures/core.md) ([its RFC](2026-06-20-core-data-structures-catalog.md)): that one catalogs the *data structures* the loop moves around (verified hand-pastes); this one catalogs the *events and services* that move them. + +## Decision + +Generate the catalog from source instead of hand-maintaining a table and verifying a subset. + +`scripts/gen-cordis-catalog.ts` walks the `interface Events` and `interface Context` declarations (plus the service classes) with the TypeScript compiler API and emits `docs/cordis-catalog/events-and-services.md` — one `## Events` section (grouped by scope, each event rendered as signature + mode badge + its source JSDoc) and one `## Services` section (each `ctx.` with its public method signatures + class JSDoc). It mirrors the `gen-module-graph` pattern exactly: `--write` regenerates, `--check` fails if the committed file is stale, output is deterministic (sorted), and the file is a build artifact that is never hand-edited. `verify-cordis-catalog` (the `--check`) runs inside `doc-sync`, so the freshness gate fires in the same lefthook pre-push and CI paths as every other doc gate. + +Pure generation is correct here because the codebase is disciplined enough that the AST is the whole truth: every event/service name is a string literal that round-trips to a static declaration — there are no dynamically-named events and no runtime-only services. So a generated doc cannot be wrong, and it closes the undocumented-event gap structurally (generation enumerates source rather than checking a hand-written subset). + +Specific choices: + +- **`@mode` tag, cross-checked.** Each harness event's JSDoc carries an explicit `@mode emit|waterfall|parallel` tag; the generator hard-errors on a missing tag. Where the signature shape is conclusive — a trailing `next: () => …` parameter is structurally a waterfall — it asserts the tag agrees and hard-errors on a contradiction. The emit-vs-parallel distinction is not structurally visible (`session/flush` returns `Promise | void` with no `next`), so it is trusted from the tag. The authoring rule lives in [AGENTS.md](../../../../AGENTS.md). +- **Tiered scope.** The harness tier (the 8 `@deepseek-ai/dsh-*` services + their events) is rendered in full from source. The inherited tier (cordis-core `ctx.on/emit/effect/provide/…` + the `internal/*` events + loader/hmr/timer) is pinned vendor source a plugin also sees; it is rendered tersely (name + one-line + source pointer) from a curated table in the generator, NOT walked from the vendor AST — the cordis-core `Context` mixes true ctx members with non-service fields (`root`, `baseUrl`, `logger`), and the vendor surface changes only on a deliberate vendor sync. +- **Cross-links to the data-structure catalog.** A type name in a signature (`GenerateOptions`, `StreamChunk`, `ToolDefinition`, …) links to the core-data-structures page that documents it. The map is a small hand-curated const in the generator — NOT `type-equiv.manifest.json`, which documents the `…Map` symbols while signatures reference the derived union names, and lists a few symbols on two pages. +- **A dedicated fence.** Signature blocks use a ` ```ts cordis-catalog ` info string that `doc-typecheck` recognizes and skips (a bare signature fragment is not standalone-compilable), excluded from the opt-out ratio — the same treatment `type-equiv` blocks get. + +This **supersedes the event-taxonomy half** of [doc-sync enforcement](2026-06-11-doc-sync-enforcement.md): `verify-event-taxonomy` and its `docs/architecture.md` table are retired (the architecture.md heading stays, its body now points at the catalog; the Service-map role table stays as curated prose). The verify-don't-generate principle that RFC chose for the taxonomy is reversed *for this surface only* — the data here is mechanically complete, so generation is strictly stronger (full signatures, cannot drift, catches undocumented events) than a name-set check of a hand-table. doc-typecheck, verify-md-wrap, verify-md-links, and verify-type-equiv are unchanged. + +## Consequences + +- The catalog cannot drift: a source change that the committed file doesn't reflect fails `verify-cordis-catalog` in the pre-push hook and CI. A new event with no `@mode` tag, or a tag that contradicts its signature, fails the generator outright. +- Event prose now has a single home — the JSDoc at the declaration. Thin JSDoc yields a thin catalog entry, which pressures authors to document at the source (the generator is a forcing function for the AGENTS.md "every export has a semantic JSDoc" rule). +- The inherited tier is hand-summarized, so a vendor sync that adds/renames a cordis-core event or `ctx` member needs a matching edit to the curated table in `gen-cordis-catalog.ts`. This is the deliberate cost of not walking pinned vendor source; it changes rarely and is called out in the generator. +- `verify-event-taxonomy.ts` is deleted and the `docs/architecture.md` event table is gone; anyone who linked to a specific table row now lands on the generated catalog instead. diff --git a/docs/rfc/implemented/process/2026-06-20-rfc-classification.md b/docs/rfc/implemented/process/2026-06-20-rfc-classification.md new file mode 100644 index 0000000000..d3fc95399b --- /dev/null +++ b/docs/rfc/implemented/process/2026-06-20-rfc-classification.md @@ -0,0 +1,46 @@ +# RFC: Classify RFCs by kind via path-encoded subdirectories + +Status: implemented (proposed 2026-06-20, accepted 2026-06-20) + +## Context + +`docs/rfc/` grouped RFCs by **lifecycle** only — `proposed/` / `implemented/` / `rejected/`. Nothing recorded what *kind* of decision each RFC was. The index was one flat list per lifecycle, with no way to scan "show me every simplification" or "every testing-strategy decision." A wave of simplification RFCs landing on the same day made the gap concrete: a reader skimming `proposed/` could not tell a new capability from a removal from a tooling-policy change without opening each file. + +The repo's standing bias is [mechanical quality gates over prose guidelines](2026-06-11-quality-gates.md): a convention that isn't machine-checked rots. So a classification scheme here had to be enforceable, not an honor-system header. + +## Decision + +Add a second axis — the RFC's **class** — and encode it in the path: `{lifecycle}/{class}/yyyy-mm-dd-topic.md`. The folder *is* the label. A file's location declares its class, the closed set is "these folders and no others," and the existing [verify-md-links](2026-06-18-markdown-cross-link-lint.md) gate already protects the path rewrites the move required. + +### The closed set of six classes + +| Class | Covers | +|---|---| +| `feature` | A new user- or model-facing capability. | +| `bug-fix` | Corrects a defect or closes a gap a postmortem surfaced. | +| `simplification` | Removes code, behavior, or surface area without adding a capability. | +| `architecture` | A structural decision about the **shipped source** — how packages relate, what the runtime vocabulary is. | +| `process` | Tooling, policy, or workflow **around** the code, not runtime behavior. | +| `testing` | Test infrastructure and strategy. | + +The `architecture` / `process` line: **architecture** is about the source we ship; **process** is the surrounding tooling and workflow. This RFC is itself a `process` decision — it changes how the repo is organized and gated, not what the harness does at runtime — so it lives under `implemented/process/`. + +### Two gates + +Both are `doc-sync` members, in the `verify-md-wrap` style (tsx ESM, verify-don't-generate, exit non-zero on the first violation): + +- **`scripts/verify-rfc-classification.ts`** — the closed set and index completeness. It asserts every file under a lifecycle folder lives in a class folder from the canonical set (a loose `.md` at a lifecycle root, or an unknown class folder, fails), and that `README.md` lists every RFC exactly once under the `###` heading matching its `{lifecycle}/{class}` path. The canonical class set lives as a `const` in this script — the machine source of truth — and [the index](../../README.md) documents it in prose; the two are kept in sync by hand (the README's completeness is gated, its class *descriptions* are not). This mirrors `verify-event-taxonomy`, which checks a doc table against source. +- **`scripts/verify-doc-refs.ts`** — source comments that cite docs. RFC paths are referenced not only from Markdown but from TypeScript doc comments (root-relative prose like `docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md`). `verify-md-links` never saw those, so the reorg could have silently orphaned them. This gate scans repo-authored `.ts` under `packages/**` and `examples/**` (excluding built `lib/` and `vendor/`) for `docs/….md` tokens, resolves each root-relative, and asserts it exists. It requires the `.md` extension so extensionless prose (`docs/postmortem/0001`, `docs/architecture.md § plugin checklist`) is left alone. + +### Rejected alternatives + +- **A `Classification:` prose line** in each file (next to `Status:`), parsed by the gate. Workable, but it duplicates into the file a fact the path can already carry, and a line can disagree with its folder. Path-encoding makes the label and its storage the same thing — there is nothing to keep in sync. +- **A `refactor` class.** It overlaps `simplification` almost entirely; the only discriminator anyone reached for was "does observable behavior change?", which `simplification` already encodes (it does not). One class, not two. +- **Auto-generating the README index** from the filesystem. Rejected to keep the index hand-written like every other doc here; the completeness gate gives the same drift-protection without generated Markdown in a curated file. + +## Consequences + +- Every RFC now sits under a class folder, and the index groups by class within each lifecycle. A reader scans one heading to see all simplifications, or all testing decisions. +- Two more fast tsx scripts in the `doc-sync` chain; no new dependency (the mdast/GFM stack was already present for `verify-md-wrap`/`verify-md-links`). +- Adding a class is a deliberate act: amend the `const` in `verify-rfc-classification.ts` and the [Classification section](../../README.md#classification), not just `mkdir` a folder. The gate rejects an unknown folder, so an ad-hoc class can't slip in. +- Source-comment doc references are now gated too — a moved or renamed doc that a `.ts` comment cites fails the pre-push hook, closing a drift class `verify-md-links` structurally could not see. diff --git a/docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md b/docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md new file mode 100644 index 0000000000..24f693e6b6 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md @@ -0,0 +1,31 @@ +# RFC: Drop the mutable session summary + +Status: implemented (proposed and accepted 2026-06-19) + +## Context + +The [session-persistence seam](../architecture/2026-06-14-session-persistence.md) split a session's out-of-log metadata into two types owned by `dsh-session`: an immutable `SessionHeader` (`version`, `id`, `createdAt`, `cwd?`, `parentSession?`) written once at creation, and a mutable `SessionSummary` (`updatedAt`, `title?`, `firstPrompt?`) "updateable without touching the append-only log". Their union was `SessionMeta = SessionHeader & SessionSummary`, and the abstract `SessionPersistence` service carried a seventh method — `update(id, summary)` — for rewriting the summary. Each backend implemented the mutable store its own way: JSONL wrote a separate atomic `.summary.json` **sidecar** beside the log (temp-write + rename, best-effort), SQLite kept `updated_at`/`title`/`first_prompt` **columns** bumped inside the append transaction. + +The summary was designed for a future session picker (recency ordering via `updatedAt`, a `title`/`firstPrompt` preview). That picker was never built. An audit of the whole repo found the entire `SessionSummary` surface is **dead state**: + +- `SessionPersistence.update()` has **zero production callers** (every `.update(` hit is `createHash().update()` or a test). +- `firstPrompt` is **never read** anywhere in production. +- `title` *is* read in the ACP bridge — but from a tool-call **presenter** (`present.title`), never from stored session metadata. +- `updatedAt` has **no consumer**: the only production caller of `list()` reads `meta.cwd` (a `SessionHeader` field) to validate a workspace on `session/load`; resume reads `createdAt`/`cwd`/`parentSession` — all header fields. +- Decisively: the live `Session.header` was already typed `SessionHeader`, not `SessionMeta` — the summary never existed on the live session object; it lived only in the persistence layer, written and read by nothing but its own contract test. + +## Decision + +Delete the mutable session summary entirely. `SessionSummary` and the `SessionMeta` name are removed; the metadata a backend stores and returns is just `SessionHeader`. `SessionPersistence.update()` is removed from the abstract service and every backend. JSONL loses the whole sidecar machinery (`writeSidecar`/`readSidecar`/`touchSummary`/`removeSidecars`/`sidecarPath` and the load/list overlays); SQLite drops the `updated_at`/`title`/`first_prompt` columns and the per-append `updated_at` bump, and its `SCHEMA_VERSION` goes `1 → 2`. + +Anything the summary was meant to provide is **derivable from the append-only log** when a consumer actually needs it (`firstPrompt` = first `user/message`; recency = the last event's `time` or the file mtime) or already lives in the immutable header (`createdAt`, `cwd`). The one thing *not* derivable — a user-*edited* title — had no implementation and is pure YAGNI; it can return as its own log event or header field if a real feature ever needs it. + +This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](../architecture/2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge. + +## No migration + +This is unreleased software (see [root AGENTS.md](../../../../AGENTS.md) § "Pre-release stance: foundation over blast radius"), so there are no on-disk databases or logs to preserve. SQLite does not migrate a v1 database: the `openDatabase` guard now rejects any non-current on-disk `user_version` (`onDisk !== 0 && onDisk !== SCHEMA_VERSION`) — older *or* newer — so a stale v1 DB is cleanly rejected rather than half-read against the new column set. A fresh database stamps the current version; that is the only path that needs to work. + +## What we gave up + +A future session picker now has to derive its preview/ordering from the log (or reintroduce a typed field) rather than reading a ready-made summary row. That is the correct cost: a cache for a feature that does not exist is dead weight that every backend pays to maintain and every contract test pays to assert. The principle — **a passing test pins current behavior, not necessarily correct behavior; behavior can be an artifact of a past compromise** — is now recorded as a standalone convention in [root AGENTS.md](../../../../AGENTS.md), with this change as its worked example. diff --git a/docs/rfc/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md b/docs/rfc/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md new file mode 100644 index 0000000000..4ec39bcce4 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md @@ -0,0 +1,43 @@ +# RFC: Fold trace-only session facts into load-bearing events + +Status: implemented (proposed and accepted 2026-06-20) + +## Problem + +The session event vocabulary includes first-class events that are not part of replayable conversation history and have little or no production consumption. `usage` is already present as a model stream chunk before the loop also appends a separate `usage` event. `error` duplicates the `turn/end { kind: 'error', message, code }` reason for loop failures; ACP settlement reads the turn-end reason, ACP rendering ignores the `error` event, and `deriveMessages()` skips it. + +These events make the canonical transcript look more useful as telemetry than it currently is. They add event variants, invariants, tests, snapshots, and persistence cases, but they are not load-bearing as separate records. The facts they carry can still be useful: token usage should remain available for accounting, and an error's step number should not silently disappear. The simplification is to fold those facts into nearby events consumers already must understand, not to record less information. + +## Proposal + +Remove standalone trace-only events only where their information can be preserved without a parallel record: + +- Fold successful-step usage into the matching `assistant/message`, e.g. `assistant/message { turn, step, content, usage? }`, so the assembled model output and its accounting travel together. +- For a failed or aborted step that has usage but no `assistant/message`, carry the usage on the terminal turn reason or another load-bearing failure record in the same turn. The implementing design must prove no usage chunk that is currently persisted becomes unrepresented. +- Fold the step number from the standalone `error` event into `turn/end.reason` for `kind: 'error'`, e.g. `{ kind: 'error', step, message, code? }`. `turn/end` is the durable turn outcome ACP and resume already consume. +- Keep `agent/error` and logging for live diagnostics; do not add a second session-log error record after `turn/end`. + +If analytics become real, add a projection helper or a dedicated telemetry store with its own retention policy. The user conversation log should contain what is needed to render, resume, audit, and account for the interaction without requiring consumers to reconcile duplicate trace rows. + +## Acceptance criteria + +- `SessionEventMap` drops standalone `usage` and `error` only after their fields are represented on load-bearing session events. +- The loop no longer appends a separate `usage` event for a usage chunk. +- The loop records durable failures through `turn/end { kind: 'error', step, message, code? }` or an equivalent no-information-loss shape and reports live diagnostics through `agent/error`. +- ACP snapshots and persistence tests stop asserting trace-only lines. +- Documentation explains exactly where token usage and operational errors are observed. +- Recorded fixtures are refreshed for the new event shape; the session format version stays pinned at `0` (unstable/pre-release) and backends reject any non-`0` stored log per the pre-release format policy. + +## What we give up + +A consumer can no longer filter the canonical log for standalone `usage` or step-level `error` rows. It must read those facts from the assistant/failure events that carry them. That is a reasonable simplification only if the implementing PR proves the same facts remain present; otherwise the standalone events should stay. + +## Implementation note + +Shipped as proposed, with one scope refinement (per AGENTS.md "RFCs are proposals, not golden truth"): + +- **Empty-content `assistant/message` hosts usage with no data loss.** The proof the proposal demanded (no persisted usage chunk becomes unrepresented) lands on the max-tokens path: a step cut off with usage but empty content (e.g. only a dropped tool call) previously emitted a standalone `usage`. It now records an empty-content `assistant/message { content: [], usage }`. To keep that from injecting a spurious content-less assistant turn into the provider transcript, `deriveMessages()` skips empty-content `assistant/message` events. A regression test asserts usage stays represented AND derived history is uncorrupted. + +**Format version.** The persisted `SessionEventMap` shape changed (usage folded onto `assistant/message`, standalone `usage`/`error` removed, `step` on `turn/end.reason.error`). The session log uses the **pinned-`0` "unstable / pre-release"** format stance (one of the two stances AGENTS.md § pre-release sanctions): `SESSION_FORMAT_VERSION` stays `0` and absorbs this and every other pre-release shape change without a monotonic bump — bumping on each tweak would dress up an unstable format as a sequence of stable boundaries that mean nothing yet. The constant is centralized in `dsh-session` and read by both write sites and the coordinator's load-time check, which rejects any non-`0` log (no migration — there is no persisted user data to preserve; a real monotonic policy begins at the first tagged release). `turn/end.reason.error.step` is required for newly-written logs. + +Usage is now observed on `assistant/message.usage`; an operational error's step on `turn/end.reason` for `kind: 'error'`. `agent/error` + logging are unchanged for live diagnostics. diff --git a/docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md b/docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md new file mode 100644 index 0000000000..e35e090dbf --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md @@ -0,0 +1,43 @@ +# RFC: Drop the unconsumed `llm/adapter-change` event + +Status: implemented (proposed and accepted 2026-06-20) + +## Problem + +`LlmService.registerAdapter()` emits `llm/adapter-change` on registration and disposal ([packages/llm/llm/src/index.ts](../../../../packages/llm/llm/src/index.ts)). Grepping `llm/adapter-change` across `packages/*/src` and `examples/*/src` finds only the declaration, emit sites, docs, and tests; no production listener subscribes to it. + +This differs from `tools/change` and `system-prompt/change`. Those two events are also unconsumed today, but they are plausible registry-change signals for future live tool/prompt UIs. LLM adapter registration is more of a boot-time implementation detail: adapters are not a user-visible palette and the real model-call interception seam is `llm/stream`. Keeping an adapter-change event with no listener repeats the [drop-the-dead-summary](../../implemented/simplification/2026-06-19-drop-mutable-session-summary.md) pattern at a smaller scale. + +The event is not free. `registerAdapter()` yields its rollback disposer before emitting `llm/adapter-change` so a throwing listener unwinds the mutation instead of leaking an adapter entry, and the package carries tests for that listener-throw path. That defensive ordering protects a failure mode only tests can trigger. + +## Proposal + +Remove only `llm/adapter-change`: + +- Delete the `llm/adapter-change` declaration from `dsh-llm`'s `interface Events`. +- Delete the `ctx.emit('llm/adapter-change')` calls. +- Simplify `registerAdapter()`'s effect generator: keep the mutation and rollback disposer for HMR/disposal, but drop the listener-throw rollback ordering that exists only for the removed event. +- Remove the "Emits `llm/adapter-change` on registration and disposal" sentence from `LlmService.registerAdapter`'s JSDoc. +- Rewrite the adapter-disposer test to assert the returned disposer removes the adapter without subscribing to `llm/adapter-change`; delete the listener-throw rollback test that exists solely for the removed event. +- Update the event taxonomy table in [docs/architecture.md](../../../architecture.md) and [packages/llm/llm/README.md](../../../../packages/llm/llm/README.md). The [doc-sync-enforcement RFC](../../implemented/process/2026-06-11-doc-sync-enforcement.md) should avoid using `llm/adapter-change` as an example once the event is gone. + +## Why not remove every registry change event? + +A microkernel where registries announce mutations is a coherent convention. `tools/change` and `system-prompt/change` may become useful when a UI can live-refresh available tools or prompt sections. This RFC leaves that convention intact where it has a plausible user-facing consumer and cuts only the adapter-change event whose current and likely future consumer is unclear. + +If an LLM adapter browser or dynamic model-picker needs this signal later, reintroduce it with that consumer and a clearer payload than "something changed." + +## Acceptance criteria + +- `llm/adapter-change` and its emits are gone; `pnpm run verify-cordis-catalog` passes against the regenerated catalog. +- HMR-safety tests still pass: disposing a contributing fiber still removes the adapter. +- `tools/change` and `system-prompt/change` remain documented and tested. +- `pnpm run test:coverage` stays 100% per-file. +- No production code path changes observable behavior (verified by unchanged ACP snapshot goldens and the echo-agent smoke test). + +## Risks + +- **Removing a documented emit event is a public-surface change.** It is in the taxonomy table, so it reads as deliberate API. But "declared and emitted" is not "consumed" — the same distinction that justified dropping the mutable summary. The taxonomy table is updated in the same change, so the docs do not drift. +- **The registry-change convention becomes uneven.** That is acceptable because LLM adapter registration is not the same user-facing concept as tools or prompt sections. Uneven but honest beats uniform but dead. + +This is a small cut, but it retires a standing correctness invariant that guards a consumer that does not exist. diff --git a/docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md b/docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md new file mode 100644 index 0000000000..74d37bf75a --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md @@ -0,0 +1,45 @@ +# RFC: Drop unconsumed assembled LLM convenience surfaces + +Status: implemented (proposed and accepted 2026-06-20) + +## Problem + +`LlmService` ([packages/llm/llm/src/index.ts](../../../../packages/llm/llm/src/index.ts)) exposes three call surfaces over a model: + +- `stream()` — raw `StreamChunk`s, dispatched through the `llm/stream` waterfall. +- `streamBlocks()` — a "convenience view" that runs the chunks through a `BlockAssembler` and yields completed `ContentBlock`s in stream order ([index.ts:137-144](../../../../packages/llm/llm/src/index.ts)). +- `generate()` — one fully-assembled `GenerateResult`, dispatched through a second `llm/generate` waterfall ([index.ts:151-157](../../../../packages/llm/llm/src/index.ts)). + +The only production consumer of the LLM service is the agent loop, and it uses `stream()` exclusively — feeding raw chunks through its own `BlockAssembler` so it can log chunks for replay fidelity while assembling in parallel ([packages/core/agent-loop/src/loop.ts](../../../../packages/core/agent-loop/src/loop.ts), the `ctx.llm.stream(req)` step). Grepping `streamBlocks` and `ctx.llm.generate` across `packages/*/src` and `examples/*/src` finds no production callers. The references are the service methods, docs, and tests; adapter tests use `generate()` as a convenient driver, but they can hand-drain `stream()` through the same assembler helper without preserving a public production API. + +This is the [drop-mutable-session-summary](../../implemented/simplification/2026-06-19-drop-mutable-session-summary.md) pattern: assembled-view APIs with tested contracts, consumed by tests rather than production. They were built speculatively for consumers that do not care about token-level deltas, but the one real consumer cares about deltas precisely so it can persist high-fidelity replay data. + +`streamBlocks()` drags a dedicated slice of `BlockAssembler` behind it: `flushReady()` and `flushRemaining()` ([packages/llm/llm/src/assembler.ts:138-168](../../../../packages/llm/llm/src/assembler.ts)) plus the `flushed` cursor field exist only to support incremental in-order yield. `generate()` drags `GenerateResult`, `BlockAssembler.result()`, and the `llm/generate` waterfall as a second interception surface over the same underlying stream. The loop's assembler usage is `push()` / `message()` / `usage` / `finish` — not streaming flush or one-shot service assembly. + +## Proposal + +Make `stream()` the only public LLM call surface: + +- Remove `LlmService.streamBlocks()` and its JSDoc. +- Remove `LlmService.generate()`, the `llm/generate` waterfall event, and `GenerateResult` if no surviving API needs that named result shape. +- Remove `BlockAssembler.flushReady()`, `BlockAssembler.flushRemaining()`, and the `flushed` cursor field. +- Remove `BlockAssembler.result()` if it is only a helper for the deleted `generate()` service path and tests. +- Replace adapter-test use of `ctx.llm.generate()` with a small test helper that calls `ctx.llm.stream()`, pushes chunks into `BlockAssembler`, and returns the assembled message, usage, and finish reason needed by that test. That keeps the [twin-adapter design](../../implemented/architecture/2026-06-13-twin-llm-adapters.md) intact while avoiding a public method whose only callers are tests. +- Remove or rework the `flushReady`/`flushRemaining`-dependent tests. Keep assembler invariants that still apply to `push()` / `blocks()` / `message()`; delete behavior that only pins the removed flush API. +- Update every doc/comment reference to `streamBlocks`, `generate`, `GenerateResult`, and `llm/generate` across `docs/`, package READMEs, and source comments. The `ctx.llm` service-map row in [docs/architecture.md](../../../architecture.md) becomes `stream()` only, the event taxonomy drops `llm/generate`, and the [property-based-testing RFC](../../implemented/testing/2026-06-11-property-based-testing.md) names block-assembly invariants without referring to removed convenience methods. + +## Acceptance criteria + +- `streamBlocks`, `generate`, `llm/generate`, and the assembler helpers they alone require are gone; `pnpm run knip` reports no new dead exports. +- `pnpm run test:coverage` stays at 100% per-file (the deleted methods take their dedicated tests with them; no remaining line goes uncovered). +- Adapter tests still exercise both real adapters through `stream()` and the shared assembler, not through a test-only public shortcut. +- The loop behaves identically — verified by unchanged ACP snapshot goldens. +- `packages/llm/llm/README.md`, [docs/architecture.md](../../../architecture.md), and module docs no longer mention the removed convenience surfaces. + +## Risks + +- **It removes public methods from a core vocabulary package.** A future plugin that wants assembled blocks without deltas would need to call `stream()` and use `BlockAssembler` directly or reintroduce a focused helper with a real consumer. Given the pre-release "foundation over speculative future" stance ([AGENTS.md](../../../../AGENTS.md)), this is the right time to cut test-only public shape. +- **Adapter tests get a little more explicit.** They lose the ergonomic `generate()` wrapper, but that is useful pressure: tests exercise the same streaming path production uses. +- **Waterfall users lose `llm/generate`.** No production listener exists. Any future caching/retry/logging plugin should wrap `llm/stream`, which remains the single provider call path. + +The size is modest, but it is a clean removal of speculative surface area from the LLM package, leaving one model-call contract for both production and tests. diff --git a/docs/rfc/implemented/simplification/2026-06-20-prune-dead-seam-methods.md b/docs/rfc/implemented/simplification/2026-06-20-prune-dead-seam-methods.md new file mode 100644 index 0000000000..9a385a01a9 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-20-prune-dead-seam-methods.md @@ -0,0 +1,42 @@ +# RFC: Prune dead methods from the persistence seam + +Status: implemented (proposed and accepted 2026-06-20) + +> **Implementation note (scope narrowed from the original proposal).** This RFC proposed pruning dead methods from BOTH the persistence seam (`SessionPersistence.has()`/`.delete()`) and the bash seam (`BashExecutor.get()`/`.list()`). Only the **persistence** removal shipped. The bash `get()`/`.list()` removal was reverted before merge: each is a one-line accessor over the executor's already-tracked `tasks` map, and removing them forced `dsh-tool-bash`'s tests onto a ~35-line `onTaskDone`-based completion-tracking harness to replace the one-line `ctx.bash.get(id)` lookup — the migration cost dwarfed the surface removed. Per the [AGENTS.md "RFCs are proposals, not golden truth"](../../../../AGENTS.md) principle, that friction is evidence the method earns its keep (a test harness IS a consumer that programs against the seam), so `get()`/`list()` stay. The bash-seam analysis below is retained for the record but was NOT acted on; `BashTaskId`-branding those methods lands in the [branded-ids RFC](../architecture/2026-06-20-branded-ids.md) instead. The persistence removal stands: `has()`/`delete()` had only contract-test callers and no test-ergonomics cost to remove. + +## Problem + +A capability seam ([interface / implementation / consumer](../../implemented/architecture/2026-06-13-capability-seams.md)) carries abstract methods that no consumer calls. The seam exists to let implementations and consumers evolve independently — but a method no consumer programs against is not a seam, it is speculative surface every implementation must still implement and test. + +### `SessionPersistence.has()` and `.delete()` + +The abstract service declared its operations beyond create/append: `load`, `list`, `has`, `delete`. Production consumers of `ctx.sessionPersistence` use only two: the agent-loop resume path calls `load()` ([packages/core/agent-loop/src/index.ts:176](../../../../packages/core/agent-loop/src/index.ts)), and the ACP bridge calls `list()` for `session/list` ([packages/ui/acp/src/index.ts:494](../../../../packages/ui/acp/src/index.ts)). Grepping every `sessionPersistence.*` / `persistence.*` use across `packages/*/src` and `examples/` finds no `has(` and no `delete(` on the service. The `.has(`/`.delete(` calls in `packages/ui/acp/src/index.ts` are on the in-memory `SessionStore` and a local `Set` of loading ids, not persistence. The only callers of `has`/`delete` were the contract suites and per-backend specs. + +`has()` was not just unused — it was the most intricate branch in the shared coordinator: a tracked-vs-untracked dual-probe (`loadLive(id, cwd)` for a live-tracked session vs `loadStored(id)` for an untracked one) with a multi-line rationale. `delete()` dragged the `deleteStored` backend hook that every backend had to implement. This is the [drop-mutable-session-summary](../../implemented/simplification/2026-06-19-drop-mutable-session-summary.md) pattern: a contract test exercised both, but no shipping code asks "is this session persisted?" or removes one. + +## Proposal + +Remove the methods nothing consumes, from the abstract seam, the implementation, and the contract/spec suites that exist only to exercise them: + +- `SessionPersistence.has()` / `.delete()`: delete the abstract declarations, the coordinator's `has`/`delete`/`deleteCore`, and the `PersistenceBackend.deleteStored` hook. Remove the `has`/`delete` rows from the contract suite and the per-backend specs (jsonl + sqlite each implemented `deleteStored` only to satisfy the hook — that implementation goes too). The backends are the [dual-backend](../../implemented/architecture/2026-06-14-session-persistence.md) design and otherwise out of scope, but removing a hook they implement for no consumer is part of removing the hook, not a backend redesign. +- Update every doc and source-comment reference to the removed methods — not only literal `has(`/`delete(`/`deleteStored` call spellings, but also `{@link has}`/`{@link delete}` JSDoc links and prose that counts the methods (removing 2 of the persistence service's 6 public methods makes any "six public methods" phrasing wrong). The implementing PR greps `has`/`delete`/`deleteStored`/`{@link `/`six ` across `docs/`, `packages/*/README.md`, and source comments, and fixes each. The known doc sites: the seam README ([packages/session-persistence/session-persistence/README.md](../../../../packages/session-persistence/session-persistence/README.md)'s `has(id)`/`delete(id)` API row and its "delegates its six public service methods" prose → four), the backend READMEs that describe `has`/`list` semantics ([packages/session-persistence/session-persistence-sqlite/README.md](../../../../packages/session-persistence/session-persistence-sqlite/README.md), [packages/session-persistence/session-persistence-jsonl/README.md](../../../../packages/session-persistence/session-persistence-jsonl/README.md) — reword "absent from `has()`/`list()`" to just `list()`), the service-map / seam docs in [docs/architecture.md](../../../architecture.md), and the persistence prose in the [session-persistence RFC](../../implemented/architecture/2026-06-14-session-persistence.md) and [shared write-coordinator RFC](../../implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md). The known source-comment sites: the abstract `create()` JSDoc's `{@link has}/{@link list}` link ([packages/session-persistence/session-persistence/src/index.ts](../../../../packages/session-persistence/session-persistence/src/index.ts) — drop the `has` link), the coordinator's "six public methods"/"six public service methods" module + class JSDoc and its lazy-materialization JSDoc justifying the `materialized` flag by "the signal `has`/`list` rely on" ([packages/session-persistence/session-persistence/src/coordinator.ts](../../../../packages/session-persistence/session-persistence/src/coordinator.ts)), the JSONL backend's `loadStored`/`deleteStored` comment, and the SQLite backend's `schema.ts` and `index.ts` comments that mention "absent from `has`/`list`" — all reworded to the surviving four-method, `list()`-only contract. + +## Why not keep them as "the seam should be complete"? + +The instinct that a persistence seam "should" offer delete is real — and it is exactly the speculative-completeness the pre-release stance warns against ([AGENTS.md](../../../../AGENTS.md): optimize for the correct foundation, not for hypothetical callers you do not have). `delete()` is one method to re-add the day a consumer needs it: a session-management UI that deletes old sessions will want it — add it then, designed against that UI's real needs (soft-delete? cascade? confirmation?), not guessed now. + +Re-adding a seam method with a live consumer is cheap and better-designed than the speculative version, because the consumer pins the contract. Carrying it unused means every implementation (and every future backend) must implement and test a method that does nothing. + +## Acceptance criteria + +- `has`/`delete`/`deleteStored` are gone from the persistence seam, impl, and contract suites; `pnpm run knip` reports no new dead exports. +- The remaining persistence operations (`create`/`append`/`load`/`list`) are untouched; ACP `session/list` and crash-recovery behave identically. +- `pnpm run test:coverage` stays 100% per-file (the contract/spec rows for the removed persistence methods are deleted with them). +- The persistence seam README and `docs/architecture.md` no longer list the removed `has`/`delete` methods. + +## Risks + +- **`delete()` is the kind of operation a product eventually wants.** True — but "eventually" is the point. Deleting it now and re-adding it against a real consumer is strictly better than shipping a guessed contract. The dual backends each shed a `deleteStored` impl, which is a bounded edit in otherwise-out-of-scope packages. +- **Low coupling.** The removal is confined to the persistence seam + impl + tests; no cross-package consumer references the removed methods, so there is no ripple beyond the docs. + +Modest size, but it converts the seam from "what an implementation must provide for nobody" back to "exactly what a consumer uses." diff --git a/docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md b/docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md new file mode 100644 index 0000000000..67fe9b07fc --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md @@ -0,0 +1,36 @@ +# RFC: Keep one public stop primitive + +Status: implemented (proposed 2026-06-20; accepted in amended form — `whenIdle()` retained) + +> **Implementation note (scope narrowed from the original proposal).** This RFC proposed removing BOTH `abort()` and `whenIdle()` from the public `Agent` handle. Only `abort()` was removed. Validating the premise against the code ([AGENTS.md "RFCs are proposals, not golden truth"](../../../../AGENTS.md)) found `whenIdle()` to be a **load-bearing quiescence primitive**, not dead surface: it is the settle signal in several ACP tests (`packages/ui/acp/tests/{edges,turns,dispose}.spec.ts`) and is backed by a deliberate loop contract (settle waiters without a status transition; handle the replacement-turn race). The RFC's suggested migration — have consumers observe the `running`→`idle` transition by hand — is exactly the brittle hand-rolled path [AGENTS.md § Defensive patterns](../../../../AGENTS.md) warns against ("Async state is not synchronous state"). Deleting a clean primitive to push every consumer onto that is a net loss, so `whenIdle()` stays. `abort()` was genuinely dead public surface (no production caller; the loop aborts its own `AbortController` directly), so it was removed as proposed. The text below is amended to describe what shipped. + +## Problem + +The public `Agent` handle exposed two overlapping ways to stop in-flight work: `abort(reason?)` and `cancel(reason?)`. `abort()` killed only the in-flight step and left queued work alone; `cancel()` clears queued and steering work, aborts the running step, and handles the pre-step race. In production, ACP uses `cancel()` for `session/cancel`, while lifecycle owners tear down agents through `AgentHandle.dispose()`. No production caller needed bare `abort()`. + +The `abort()`/`cancel()` distinction is real — `abort()` preserves queued prompts and steering while `cancel()` drops them — but no shipping code called the public `abort()` verb. The loop's own stop paths (`cancel()` and disposal) abort the current `AbortController` directly rather than routing through `Agent.abort()`. Most tests that called `abort()` interrupt an empty queue and switch to `cancel(reason)`; the steering re-delivery test that deliberately depends on queue preservation drives the in-flight `AbortController` directly, because `cancel()` would drop the queued steering it is trying to prove survives a step abort. The no-argument `abort()` default reason (`'aborted'`) is deleted with the verb rather than preserved by accident; `cancel()` keeps its own `'cancelled'` default. + +The extra surface area made the loop carry a public verb that is mostly a teardown internal: `abort()` had to be documented as distinct from queue-aware cancellation even though a UI cancellation almost always wants the broader operation. + +## Proposal + +Keep `cancel()` as the only public *stop* primitive on `Agent`. Lifecycle owners use `AgentHandle.dispose()` to stop and unregister an agent; non-owners use `cancel()` to abandon current and queued work. The implementation keeps a private abort controller, but it is not part of the plugin-facing `Agent` contract. + +`whenIdle()` is **retained** as the public quiescence-observation primitive (resolve once the agent settles out of `running`, resolve immediately when already idle, await the loop exit when disposed). It is not a stop verb; it is how a non-owner observes the stop *completing* without disposing the agent. Its live consumers are ACP and agent tests that await settlement through this public seam (`packages/ui/acp/tests`, `packages/core/agent-loop/tests`); the production ACP bridge owns its agents and tears them down through `AgentHandle.dispose()`, so `packages/ui/acp/src` itself has no `whenIdle()` call. + +Delete public `abort()`, the tests that exercise it as standalone API, and the docs that describe step-only abort as an embedding feature. Empty-queue abort tests migrate to `cancel(reason)` where they still prove cancellation behavior; tests whose subject is the loop's internal `AbortController` behavior drive that controller directly via an in-package typed cast to the private field; tests that only pin the removed no-arg `abort()` default go away with the method. The disposer remains async and still waits for the loop to stop. + +## Acceptance criteria + +- `Agent` exposes no public `abort()`; `cancel()`, `whenIdle()`, and `steer()` remain part of the surface. +- ACP cancellation continues to call `cancel()`. +- Agent teardown continues to await quiescence through handle disposal, and `whenIdle()` still resolves on quiescence for non-owner observers. +- Tests cover cancellation and disposal as the two supported stop paths. + +## What we give up + +A future plugin cannot abort only the current model/tool step while preserving queued prompts through the public interface. If that use case becomes real, it should return with a named consumer and a narrower contract. Today it is latent generality that keeps a private loop mechanic public. + +## Related + +This RFC only removes the redundant stop verb. Mid-turn steering remains an intentional message path; quiescence observation remains via `whenIdle()`. The resulting public surface is `send()`, `steer()`, `inject()`, `cancel()`, `whenIdle()`, status, options, session, and identity. diff --git a/docs/rfc/implemented/2026-06-11-property-based-testing.md b/docs/rfc/implemented/testing/2026-06-11-property-based-testing.md similarity index 82% rename from docs/rfc/implemented/2026-06-11-property-based-testing.md rename to docs/rfc/implemented/testing/2026-06-11-property-based-testing.md index 9e4300cf65..c3f1d1a46b 100644 --- a/docs/rfc/implemented/2026-06-11-property-based-testing.md +++ b/docs/rfc/implemented/testing/2026-06-11-property-based-testing.md @@ -8,15 +8,15 @@ Status: implemented (proposed 2026-06-11, accepted 2026-06-14) ## Context -Example-based tests pin the cases we thought of. The harness's core is protocol-shaped — chunk streams, event logs, schema conversion, inbox scheduling — where the input space is combinatorial and the interesting bugs live in interleavings nobody wrote an example for. The motivating evidence: a `streamBlocks` ordering bug once survived 100% line coverage of the happy paths. Per-file 100% coverage proves every line ran, not that every interleaving is correct. +Example-based tests pin the cases we thought of. The harness's core is protocol-shaped — chunk streams, event logs, schema conversion, inbox scheduling — where the input space is combinatorial and the interesting bugs live in interleavings nobody wrote an example for. The motivating evidence: a block-assembly ordering bug once survived 100% line coverage of the happy paths. Per-file 100% coverage proves every line ran, not that every interleaving is correct. ## Decision Adopt `fast-check` (a root devDependency) with one `tests/properties.spec.ts` per protocol-shaped package, generators tuned for *realistic-but-adversarial* inputs (not uniform noise) and `numRuns` kept so the suite stays well under ~10s locally. Failures print a reproducible seed. (The original proposal also sketched a nightly CI job running 100× the iterations; that was not shipped — the property suite runs only in the normal `push`/`pull_request` CI, and a scheduled high-iteration job remains possible future work.) -- **dsh-llm / BlockAssembler:** arbitrary chunk streams (valid + malformed: duplicate indices, stragglers, missing block-start). Invariants: `flushReady()+flushRemaining() ≡ blocks()` in order; the streamed prefix is always a prefix of final `blocks()`; partial count ≤ distinct indices; re-assembly idempotent. +- **dsh-llm / BlockAssembler:** arbitrary chunk streams (valid + malformed: duplicate indices, stragglers, missing block-start). Invariants: `blocks()` count ≤ distinct indices seen; re-assembly idempotent (`blocks()` is stable across repeated calls and `message().content` mirrors it); `blocks()` never throws and yields only valid content-block tags; `finish` reflects the last `finish` chunk, defaulting to `{kind:'stop'}` when none arrives. - **dsh-session:** arbitrary event logs. Invariants: `deriveMessages` deterministic; replay-from-seed identical; seq strictly monotonic; non-message events never affect derived history; derived content is decoupled from the log. -- **dsh-tools:** arbitrary `SchemaSpec`. Invariants: JSON Schema `required` equals the `required:true` keys at every level; conversion total; **and the composition with [runtime arg validation](2026-06-11-runtime-arg-validation.md)** — generated args satisfying a spec pass `validateArgs`, and targeted corruptions (dropped required key, non-object top level) are rejected. This closes the validator/`InferArgs` drift risk. +- **dsh-tools:** arbitrary `SchemaSpec`. Invariants: JSON Schema `required` equals the `required:true` keys at every level; conversion total; **and the composition with [runtime arg validation](../architecture/2026-06-11-runtime-arg-validation.md)** — generated args satisfying a spec pass `validateArgs`, and targeted corruptions (dropped required key, non-object top level) are rejected. This closes the validator/`InferArgs` drift risk. - **dsh-agent-loop:** arbitrary send schedules against a never-exhausting adapter, driven through the `agent/status` settle signal (no wall-clock sleeps). Invariants: no message lost; turn numbers strictly increase; status transitions stay on the legal machine. ## Consequences diff --git a/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md b/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md new file mode 100644 index 0000000000..7f6cd47b48 --- /dev/null +++ b/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md @@ -0,0 +1,79 @@ +# RFC: ACP snapshot tests — record-once / replay-deterministic + +Status: implemented (accepted 2026-06-19) + + + +## Context + +The harness has two test tiers: keyless unit `.spec.ts` (the 100%-per-file coverage gate) and real-API `.e2e.ts` (key-gated, self-skipping in CI). Neither continuously verifies the **complete output transcript** an ACP editor (Zed) sees on its stdin/stdout. The existing ACP e2e ([examples/acp-agent/tests/acp.e2e.ts](../../../../examples/acp-agent/tests/acp.e2e.ts)) is the closest end-to-end check, but it is key-gated and asserts on a handful of *structured fields* (`stopReason`, a `tool_call` title), not the byte-for-byte stream of `session/update` frames. That leaves the "green units, broken product" gap: every unit test can pass while the actual editor-facing protocol output regresses — the same class of failure that shipped the inject bug ([docs/postmortem/0001](../../../postmortem/0001-acp-default-export-drops-inject.md)), where 178 hand-mounted tests stayed green while a real Zed session crashed instantly. + +The blocker for a full-transcript test is the model: the agent's output is driven by a non-deterministic LLM, and a key-gated test that hits the real API on every run is neither deterministic nor CI-runnable. We want the fidelity of a real run with the determinism of a fixture. + +This RFC records the decision to add a third test tier — **snapshot tests** — and the design choices that make it deterministic, keyless-in-CI, and cheap to maintain. + +## Decision + +A snapshot test boots the **real** `examples/acp-agent` subprocess, drives it over real ACP stdio with a deterministic input script, and diffs its (normalized) output against committed golden files. The model is made deterministic by **recording a real run's session log once** against the real API and **replaying it** on every subsequent run. The committed fixture IS the persisted session JSONL — the same append-only log the harness writes for any session. + +### The fixture is the persisted session JSONL + +The per-scenario fixture is `/session.jsonl`: the exact log produced by running the scenario once against the real API (the snapshot harness harvests the file the JSONL persistence backend writes). This log already contains everything needed to reproduce the run deterministically: its `assistant/chunk` events carry every parsed `StreamChunk` (the LLM's behavior), and its `tool/call`/`tool/result`/`turn/*`/`assistant/message` events carry the harness's behavior (token usage rides on `assistant/message.usage`). One artifact captures both, and it is the format the codebase already treats as the authoritative replay record ([packages/core/session/src/types.ts](../../../../packages/core/session/src/types.ts): "raw chunks are the replay record"). + +An earlier draft used a hand-authored `llm.json` of model chunks; reusing the real session log instead means the fixture is a genuine product of the system (not a hand-built mock), and it doubles as a behavioral golden (see below). A byte-level HTTP-record library (Polly/nock/MSW) was rejected: adapter-specific, awkward with streaming SSE, and lower-level than the thing under test. + +### Replay derives the model script from the log + +The replay seam is the provider-agnostic `llm/stream` waterfall ([packages/llm/llm/src/index.ts](../../../../packages/llm/llm/src/index.ts)) — a single listener intercepts every model call regardless of adapter (deepseek, pi-ai), because the loop routes all model calls through `ctx.llm.stream()`. The `llm-replay` plugin short-circuits that waterfall (never calls `next()`) and serves back streams reconstructed from the log: `deriveReplayScript(events)` groups `assistant/chunk` events by `(turn, step)` in log order, yielding one model stream per group. This grouping is exact because the agent loop makes **exactly one `ctx.llm.stream()` call per step** and tags every chunk with the current `(turn, step)` ([packages/core/agent-loop/src/loop.ts](../../../../packages/core/agent-loop/src/loop.ts)): `step` increments once per loop iteration, so `(turn, step)` is unique per model call. A `finish {kind:'error'}` chunk is part of its group and replays naturally — no special-casing. + +### The in-memory replay entry honors the full LLM contract + +`deriveReplayScript` produces a list of `ReplayEntry`, the in-memory unit the replay listener serves positionally: + +``` +{ kind: 'chunks', chunks: StreamChunk[] } +| { kind: 'throw', chunks: StreamChunk[], message: string, code: string, status?: number } +| { kind: 'hang' } +``` + +`chunks` is what the log derives. The other two cover the LLM contract's failure branches the log **cannot** reconstruct from `assistant/chunk` alone: a *pure throw before any chunk* (e.g. an HTTP 401 — the log holds only a `turn/end {error}`, no chunks) and a *cancel/hang* (a timing behavior, not chunk content). A scenario needing those supplies an optional `/replay.override.json` (a `ReplayEntry[]`) that **replaces** the derived script. The `throw` entry carries any prefix chunks so a mid-stream failure replays its partial output before throwing — the "honor cross-seam contracts on BOTH sides" defensive pattern. Synthesizing throw/cancel from the log's `turn/end {kind:error|aborted}` was rejected: it would couple `llm-replay` to loop-internal turn-closing semantics and the `turn/end` reason is lossy (it can't distinguish a thrown 401 from a finish-error). An explicit sidecar is the cleaner seam. + +### Positional replay, one in-flight stream + +Replay is positional: the Nth `stream()` call serves the Nth `ReplayEntry`. This is deterministic **only when at most one model stream is in flight at a time**. The first cut runs one ACP session per scenario, which guarantees that. Multi-session concurrency (the bridge multiplexes N sessions, which can prompt concurrently) would let scheduling decide which model call consumes which entry — so concurrent-session snapshots are out of scope until entries are keyed by request rather than position. A scenario whose control flow changes the number/order of model calls must be re-recorded; the cursor **fails loud on overrun** rather than silently reusing or skipping an entry. A missing `session.jsonl` in replay fails loud too ("record first") — never a silent skip. + +### Recording harvests the log; keyless replay needs a providerless config + +Recording runs the scenario with the real `llm-deepseek` adapter and the JSONL persistence backend, then copies the produced `.jsonl` into the scenario dir. Per-event appends are durable, but the harness shuts the subprocess down gracefully (close stdin → `await ctx.dispose()`) before harvesting so the final events are flushed. `llm-replay` itself does no recording — it is replay-only. + +The ACP server app loads `@deepseek-ai/dsh-llm-deepseek`, whose `apply` throws when no API key is present ([packages/llm/llm-deepseek/src/index.ts](../../../../packages/llm/llm-deepseek/src/index.ts)). So replay cannot reuse the normal config — it uses a dedicated `examples/acp-agent/cordis.snapshot.yml` that installs `llm-replay` in place of the adapter. The rest of the tree is not duplicated: both the normal `examples/acp-agent/cordis.yml` and the replay config load the same `@deepseek-ai/dsh-acp-agent` app entry (which bundles the agent-core spine + JSONL persistence + the ACP bridge), differing only in the LLM backend (`llm-deepseek` vs `llm-replay`) and the bash executor line. Recording reuses the normal `cordis.yml` (real adapter) — its persistence root reads `$DSH_SNAPSHOT_SESSIONS_ROOT` when the harness sets it — so there is no separate record config. The `dsh-acp-agent` bin selects `cordis.snapshot.yml` for `DSH_SNAPSHOT=replay` and skips `.env` loading in that mode so a stray key cannot trigger a live call. + +### Two surfaces: normalize, then compare + +A snapshot run asserts **two** normalized surfaces, because the harness's external surfaces are distinct: + +1. The **stdout transcript** — the framed `session/update` JSON-RPC the editor sees. Catches regressions in the ACP bridge's event→update translation (`streamSessionEventUpdate`). Compared against a committed `stdout.golden.jsonl`. +2. The **re-persisted session JSONL** — the log the replay run itself persists, compared against the scenario's `session.jsonl`. Catches regressions in the loop, tool dispatch, and turn/step structure that never surface on stdout. There is no separate session golden: `session.jsonl` is BOTH the replay source (recorded scenarios) and the expected produced log. Both sides pass through `normalizeSessionLog` before comparing — the fixture is raw-harvested (its own real session id / cwd / timestamps) and the replay output has fresh ones, so each is scrubbed against ITS OWN volatile values (the fixture's read from its header line) and the comparison is on normalized form. For an authored override scenario the same `session.jsonl` holds the expected produced log; `replay.override.json` drives the model, and `llm-replay` ignores the fixture for model chunks when an override exists, so committing the expected log there does not affect replay. + +The two are genuinely additive: stdout is the bridge's *lossy projection* of the log (it drops `assistant/message.usage`, `step/*`, exact `seq`/`time`, and renders tool I/O differently), so a loop/tool/turn-structure regression can change the JSONL while leaving the stdout projection identical, and a bridge-translation regression can change stdout while the JSONL is untouched. Asserting the JSONL equality also echoes the proposed [universal replay fixture](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md) idea. + +Both surfaces contain non-deterministic values that a pure normalization function scrubs **before** the compare: `randomUUID()` session ids → `{{sessionId}}`, the temp `mkdtemp` cwd → `{{cwd}}` (it appears in terminal-card `_meta` and the log header), JSON-RPC ids → a stable sequence, and the log's per-event `time` (epoch ms) + header `createdAt` dropped or zeroed (the log's `seq` is left intact — it is deterministic by contract, `seq = log.length`). Real bash runs during replay, so the JSONL normalizer additionally stabilizes tool-output volatility (any embedded paths/pids/timestamps) — scenarios keep bash commands tightly constrained (`echo`, file writes; no `date`/`env`/background/large-output) so this surface is small. The committed `stdout.golden.jsonl` is itself **JSONL** — one compact, normalized record per line, in the same shape as the wire (NDJSON on the wire, JSONL on disk), so it stays `grep`/`jq`-able and faithful to what the agent actually emits. A separate raw-purity assertion keeps the guarantee that every stdout line parses as JSON (no logger leak onto the protocol channel). Vitest's `toMatchFileSnapshot` provides the stdout golden store and the `-u`/`--update` "accept the diff" workflow; the session log is checked with a plain normalized-string equality against `session.jsonl`, NOT `toMatchFileSnapshot` (which would overwrite the fixture). + +### Isolation: normalization now, sandbox later + + +Determinism of the tool environment comes from a per-test `mkdtemp` cwd, the executor's existing secret-scrubbing env (`/KEY|SECRET|TOKEN/i`), the fresh non-login `bash -c` per call, and the normalization pass — **not** from an OS sandbox. A real rootless sandbox (bwrap on Linux, sandbox-exec/Seatbelt on macOS) is the established cross-platform pattern (Claude Code, Codex), but it is per-OS, fragile on newer kernels (Ubuntu 24.04+ AppArmor blocks unprivileged user namespaces), and unnecessary for transcript determinism. It is reserved as a future tier via the documented `BashExecutor` capability seam ([a sandboxing executor replaces dsh-bash-local without touching a tool schema](../architecture/2026-06-13-capability-seams.md)) — a new `bash-*` package, not a change here. Scenarios keep bash commands tightly constrained (no `date`/`env`/background/large-output) so the temp-dir tier suffices. + +### The replay plugin is its own package + +The replay plugin lives in its own package, `@deepseek-ai/dsh-llm-replay` (`packages/support/llm-replay/`), and the snapshot config references it by package name. It is the keyless replacement for the real LLM adapter: it installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded session JSONL. Its sole consumer is the ACP snapshot harness here, but it is a package (not example-local glue like echo-agent's [mock-llm.ts](../../../../examples/echo-agent/src/mock-llm.ts)) so that its derive/parse/replay branches fall under the per-file 100% coverage gate on package `src` trees — logic under `examples/` is not measured by that gate, which would leave those branches unguarded. + +### Two subcommands, replay in the default gate + +`pnpm run test:snapshot` runs replay (keyless) and is composed into the default `pnpm run test` gate so every PR gets the regression check (the main `vitest.config.ts` include stays narrow; the gate is `test && test:snapshot`). `pnpm run test:snapshot:record` requires `DEEPSEEK_API_KEY` (loaded from repo `.env` first), hits the real API, harvests the produced `session.jsonl` (the replay source AND the expected-log artifact), and `--update`s the stdout golden in one pass. Both forward a scenario filter. A missing fixture in replay **fails loud** with a "record first" message rather than self-skipping (the e2e self-skip rule is a CI-secret accommodation, not appropriate here — a committed-fixture test that silently vanishes is a coverage hole). A no-model scenario's `session.jsonl` simply has no `assistant/chunk` events (empty derived script); fail-loud still applies if a model call happens with no entry. An orphan-fixture guard test fails on a golden/fixture not referenced by any scenario (Vitest does not prune orphaned raw goldens), and a per-kind required-fixture guard asserts each scenario ships exactly the files its kind needs (`input.json` + `stdout.golden.jsonl` + `session.jsonl` for ALL scenarios — the harness passes `/session.jsonl` to `llm-replay` unconditionally, so even a no-model scenario needs its header-only fixture or `loadReplayScript()` fails; `replay.override.json` additionally for authored model scenarios). + +## Consequences + +A new test tier and its fixtures to maintain: each scenario is a directory of `input.json` (the client stdin script) + `session.jsonl` (the recorded log, which doubles as the expected re-persisted log) + an optional `replay.override.json` + an optional `workspace/` seed dir + the `stdout.golden.jsonl`, committed and reviewed. A scenario that needs the agent to operate on existing files (read, edit, grep) ships a `/workspace/` directory; the harness copies its contents into the temp cwd before the run, so the seeded files are present for both record and replay (the cwd is normalized in the goldens, so the seeded paths stay stable). Re-recording when the model's phrasing changes churns the fixture and the stdout golden — visible in review, which is the point of committing them. Bought: deterministic, keyless, full-transcript regression coverage that boots the real Loader (so it still guards the export-shape bug class), exercises the real bash executor, and gives a one-command accept-the-diff loop. The tier is ACP-first but the harness (subprocess + tee + input-DSL + workspace seeding + normalization + JSONL-derived replay) is example-agnostic and extends to other examples. + +This RFC relates to but does not supersede the [proposed determinism RFC](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md): that proposal's "universal replay fixture" re-derives session *message history* after every test (an internal-consistency invariant), whereas snapshot tests pin the *external protocol output*. They are complementary — one guards the event-sourcing invariant, the other guards the editor-facing contract. diff --git a/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md b/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md new file mode 100644 index 0000000000..2001147be8 --- /dev/null +++ b/docs/rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md @@ -0,0 +1,95 @@ +# RFC: Real-API e2e in CI against the external DeepSeek API + +Status: implemented (accepted 2026-06-19) + + + +## Context + +The harness leans hard on real-API tests by policy: AGENTS.md § Secrets argues that a no-key suite proves the plumbing but not the product, and the [ACP inject postmortem](../../../postmortem/0001-acp-default-export-drops-inject.md) is the standing proof — 178 keyless tests stayed green while a real editor session crashed instantly. The real-API e2e suite (`pnpm run test:e2e`, the `*.e2e.ts` files) exists precisely to close that gap: it drives the agent against the live DeepSeek API — real model calls, real bash tools, multi-turn, resume, ACP-over-stdio. + +But until this change **nothing in CI ran it**. The default gate ([.github/workflows/ci.yml](../../../../.github/workflows/ci.yml)) is deliberately keyless — it carries no secret, runs on every push and PR including from forks, and stays green for any contributor. `test:e2e` self-skips without a key (`describe.skipIf(!process.env.DEEPSEEK_API_KEY)`), so even if ci.yml invoked it, a keyless runner would skip it green. The real-API safety net therefore only fired when a developer happened to run it locally with a key in their environment — i.e. unreliably, and never as a merge gate. + +This RFC records the decision to add a **second, secret-consuming workflow** that runs the real-API suite in CI, and — because introducing the first CI secret into a repo that may later go public is a security/isolation decision — the threat model it relies on and what changes when the repo becomes public. + +## Decision + +Add a dedicated workflow, [.github/workflows/e2e.yml](../../../../.github/workflows/e2e.yml), separate from ci.yml. It runs only `pnpm run test:e2e` against the external API using a repo secret, on trusted events, with a preflight that converts a missing secret into a loud failure instead of a false green. ci.yml is left untouched. + +### A separate workflow, not a job in ci.yml + +ci.yml's value is that it is keyless, forkable, and always-green: any contributor (including an outside fork) gets a complete keyless signal with no secret in the blast radius. Adding a secret-consuming job there would couple that always-green gate to credential availability and a different trigger policy. Keeping the secret-bearing work in its own file isolates the secret, trigger, and concurrency policy, and preserves ci.yml's property for forks. Different lifecycles → different files. + +### Cost is not the constraint; reliability is + +The usual reason to ration real-API CI — token cost — does not apply here: we are DeepSeek and internal inference is effectively free. So the design optimizes for *coverage and signal*, not for minimizing calls. The suite runs in full (all six `*.e2e.ts` files), on multiple triggers, on every trusted PR. This is the CI embodiment of the AGENTS.md "lean on with-key e2e tests" policy. + +### Triggers: trusted events only + +`workflow_dispatch` + `push` to `main`/`master` + nightly `schedule` (`17 0 * * *`, 08:17 Asia/Shanghai) + `pull_request`. Push gives a post-merge signal; schedule catches drift in the external API itself even with no commits; dispatch is the manual escape hatch; `pull_request` gives a pre-merge gate. The user explicitly chose to include PR runs for the pre-merge signal, accepting the larger key-exposure surface that implies (see § Security). + +### The untrusted-PR gate + +GitHub withholds repo secrets from two kinds of PR: those from **forks**, and **Dependabot** PRs (same-repo branch, so `head.repo.fork == false`, but secrets are still withheld). A job-level `if:` skips the whole job for both: + +``` +github.event_name != 'pull_request' + || !(github.event.pull_request.head.repo.fork || github.event.pull_request.user.login == 'dependabot[bot]') +``` + +The Dependabot clause keys on the PR **author** (`pull_request.user.login`), not `github.actor` (the run trigger): a maintainer who reopens or re-runs a Dependabot PR would make `github.actor` a human while the PR is still keyless, and an author-based test stays correct across that. A job skipped by a **job-level** `if:` reports as a *successful* check (unlike a workflow/trigger-level skip, which stays pending), so this workflow is safe to mark as a required status check if desired — a fork/Dependabot PR's skipped-but-green check does not block the merge. + +The gate is a *clean-skip nicety*, not the secret's security boundary (see § Security — the boundary is GitHub's own fork-secret withholding under `pull_request`). Without the gate, forks still could not read the key; they would just hit a confusing preflight hard-fail and waste compute. + +### Preflight: fail loud, never false-green + +Because the job only runs on trusted events where the secret is expected, the preflight is an unconditional presence check: empty key → `exit 1` with a `::error::` annotation naming the secret to configure. This is the crux that makes a self-skipping suite safe to gate on. Without it, a deleted/renamed/misconfigured secret would make `test:e2e` skip every real suite and report all-green — a silent regression of the entire safety net. The guard turns "secret missing" from an invisible false pass into a visible failure. (Its correctness was verified live: the run before the secret existed failed at exactly this step.) + +### Secret mapping and hygiene + +The repo secret is named `DEEPSEEK_API_KEY_EXTERNAL`; it is mapped to the `DEEPSEEK_API_KEY` env var the adapters and tests read (`process.env.DEEPSEEK_API_KEY`). The distinct secret name documents intent (this is the *external* public-API key, not an internal-endpoint key) and lets an internal-endpoint key coexist later without collision. Hygiene choices, each defensive: + +- **Step-scoped secret.** `DEEPSEEK_API_KEY` is set in the `env:` of only the preflight and e2e steps, never job-level — so checkout/setup-node/install never see it. A compromised install-time lifecycle script in a dependency cannot read a secret that isn't in its environment. +- **`permissions: contents: read`.** The job only reads the repo to run tests; it needs no write scopes (no PR comments, no status writes), so the `GITHUB_TOKEN` is dropped to least privilege. +- **`DEEPSEEK_BASE_URL` pinned** to `https://api.deepseek.com` on the e2e step. The adapter would default to this when unset ([packages/llm/llm-deepseek/src/index.ts](../../../../packages/llm/llm-deepseek/src/index.ts) `PUBLIC_BASE_URL`), but pinning is self-documenting and hermetic — a stray repo-root `.env` (which `vitest.e2e.config.ts` loads if present) cannot silently redirect the run to another endpoint. +- **No secret echoed.** The preflight prints only `DEEPSEEK_API_KEY present.` — not the value, not its length. (An earlier draft echoed `${#KEY}`; dropped as needless metadata.) + +### Scope, runtime shape + +Run **only** `test:e2e`. The keyless gates (typecheck/lint/coverage/snapshot/build/hygiene) already run in ci.yml on every push and PR; repeating them here would duplicate signal and slow the real-API job. No build step — e2e tests run unbuilt via tsx + the tsconfig paths map. Single Node 24 (the `engines` floor): these tests exercise *API integration*, not node-version compat, which ci.yml's `[24, 26]` matrix already owns; a second Node version would double real-API calls for no added signal. `timeout-minutes: 45` bounds a wedged run given serial files (`fileParallelism: false`), 120s/test, and `retry: 2`. `cancel-in-progress` is enabled only for `pull_request` runs — a superseded PR run is on a stale commit and worth cancelling, whereas a push/schedule run is already producing the post-merge/nightly signal and is never cancelled. + +## Security + +Introducing the first CI secret is the part of this change that warrants a recorded threat model, because the natural question — *"can anyone who opens a PR steal the key?"* — has a non-obvious answer, and the answer shifts when the repo goes public. + +### Who can reach the secret today (private repo) + +- **No write access (fork PRs): cannot.** Two independent facts block it. First, the workflow uses `pull_request`, **not** `pull_request_target` — GitHub does not pass repo secrets to fork-PR runs of `pull_request`, so `secrets.DEEPSEEK_API_KEY_EXTERNAL` resolves to empty on a fork runner. Second, the `if:` gate skips fork PRs entirely. The withholding is the real boundary; the gate is defense-in-depth and UX. +- **Write (push) access: can.** A same-repo branch PR receives secrets, so a write-access author could modify test code (or an install lifecycle script, or the workflow YAML on their branch) to exfiltrate the key. This is **inherent to GitHub Actions, not introduced here**: anyone with push access to any repo can already exfiltrate any of its Actions secrets by authoring a workflow. Write access ⇒ secret access, always. The mitigation lives in who is granted write and in branch protection, not in this file. + +So "everyone who could open a PR can steal it" is false: only the write-access set can, and that set could already steal any secret the repo holds. + +### The residual exposure the `pull_request` trigger adds + +Because PR runs are enabled, the key is handed to **the code on a write-access author's PR branch** — code under review, not yet merged — which is a strictly larger surface than `push`-to-main + `schedule` + `workflow_dispatch` alone (where the key only ever touches already-merged or manually-dispatched code). This was the explicit round-1 tradeoff: the pre-merge real-API gate is worth it for a trusted internal write set and a low-value (internal, free) key. If that calculus changes, the hardening is one line — drop the `pull_request` trigger — keeping post-merge + nightly + on-demand coverage. + +### What changes when the repo goes public + +The secret stays protected from the public **through this workflow**: `pull_request` behaves identically on a public repo — fork PRs (now openable by anyone) still receive no secret, and on public repos GitHub additionally gates fork-PR runs behind maintainer approval, where even an approved run gets no secret (approving the run is not the same as handing over the key). The write-access set is unchanged by visibility, so the insider reality is also unchanged. + +What gets worse is the *surrounding* model, and these are the things to address before flipping visibility: + +- **Logs become world-readable.** A careless secret echo that today leaks to org members would leak to the entire internet and be scraped within minutes. Secret-handling discipline (no value/length echoes — already done) matters far more. +- **The `pull_request_target` footgun becomes catastrophic.** If anyone ever "fixes" PR runs by switching the trigger to `pull_request_target`, the workflow would run untrusted fork code in the base-repo context **with** secrets — a full key-leak vector. This is benign-ish on a private repo and disastrous on a public one. A `SECURITY —` comment on the trigger in e2e.yml forbids the change and points here. +- **Rotate on flip.** The key lived in a private repo's CI; treat going-public as "assume exposed" and rotate `DEEPSEEK_API_KEY_EXTERNAL` at that moment. +- **Settle the secret behind controls.** Confirm Settings → Actions → *"Send secrets to workflows from fork pull requests"* stays **off** (the one setting that would actually break the fork boundary), and consider moving the key into a GitHub **Environment** with required reviewers so even merged code uses it only under controlled conditions and rotation has a single home. + +None of these require changing the workflow to go public; they are operational steps plus the already-added `pull_request_target` guard comment. + +## Consequences + +A second CI workflow and the first repo secret to maintain. The real-API suite now gates merges (pre-merge on trusted PRs, post-merge on the main branch) and runs nightly, so a real break in the agent's interaction with the external API surfaces in CI rather than only in a developer's local run — at the cost of real (but internally free) API calls on every trusted PR and merge. The preflight makes secret misconfiguration self-announcing instead of silently disabling the net. + +The design carries a documented constraint surface: the `pull_request` trigger's key-exposure tradeoff (drop it to harden), the `if:` gate's dependence on the author-based Dependabot test, and the hard prohibition on `pull_request_target`. The going-public checklist above is the operational companion — this RFC is the place a future maintainer should re-read before changing the trigger set or flipping repo visibility, rather than re-deriving the fork/secret model from scratch. + +The scheduled trigger auto-disables after 60 days of repo inactivity (a GitHub behavior); push/PR/dispatch are backstops, and an active monorepo will not hit it. Runner egress to `https://api.deepseek.com` is assumed — GitHub-hosted `ubuntu-latest` has it; an egress-restricted self-hosted runner would need connectivity confirmed before relying on the nightly. diff --git a/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md b/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md new file mode 100644 index 0000000000..af7cd59d06 --- /dev/null +++ b/docs/rfc/implemented/testing/2026-06-20-remove-redundant-snapshot-log-goldens.md @@ -0,0 +1,35 @@ +# RFC: Use `session.jsonl` as the only snapshot session-log artifact + +Status: implemented (proposed and accepted 2026-06-20) + +## Problem + +Model-driving ACP snapshot scenarios ship both `session.jsonl` and `session.golden.jsonl`. For normal recorded scenarios, `session.jsonl` is the replay fixture harvested from a real run, and the replay test normalizes the newly persisted log and compares it to `session.golden.jsonl`. In the current fixtures, the normalized recorded log and normalized golden are identical for the ordinary recorded scenarios. + +Authored override scenarios (`error-finish`, `cancel`) currently use `replay.override.json` to drive model behavior and keep `session.jsonl` as a minimal dummy fixture, while `session.golden.jsonl` holds the expected persisted log. The override file is a JSON array of `ReplayEntry` objects: `{ "kind": "chunks", "chunks": StreamChunk[] }`, `{ "kind": "throw", "chunks": StreamChunk[], "message": string, "code": string, "status"?: number }`, or `{ "kind": "hang" }`. That split is also unnecessary: when an override sidecar exists, `llm-replay` replaces the derived script and does not need `session.jsonl` for model chunks, so `session.jsonl` can still be the expected session-log artifact for the scenario. + +## Proposal + +Remove the `session.golden.jsonl` concept entirely. Every scenario has at most one committed session-log artifact, `session.jsonl`: + +- For recorded scenarios, `session.jsonl` remains the raw harvested log. Replay still derives model chunks from it, and the snapshot test compares the replay run's normalized persisted log against normalized `session.jsonl`. +- For authored override scenarios, `replay.override.json` drives model behavior and `session.jsonl` holds the expected produced session log. The replay adapter ignores the fixture for model chunks when the override exists, so the same file can be the expected log without affecting replay behavior. +- For no-model scenarios, `session.jsonl` can stay as the minimal fixture needed to boot `llm-replay`; no session-log comparison is needed unless the scenario creates a persisted session. + +Stdout goldens remain unchanged; they are the editor-facing projection and are not redundant with the session fixture. + +## Acceptance criteria + +- `session.golden.jsonl` disappears from the snapshot harness, fixtures, orphan guards, and docs. +- The snapshot test derives the expected session log from `session.jsonl` for every model scenario. +- Authored sidecar scenarios commit their expected produced log in `session.jsonl`; `replay.override.json` remains the model-behavior override. +- Orphan-fixture guards understand which files are required by scenario kind. +- The [ACP snapshot tests RFC](../../implemented/testing/2026-06-19-acp-snapshot-tests.md) is updated to describe the reduced fixture set. + +## What we give up + +Reviewers lose one artifact name that made the expected persisted log visually separate from the replay fixture. The stdout golden still protects the editor transcript, and comparing replay output to `session.jsonl` preserves the loop/persistence regression check without duplicating files. + +## Implementation note + +The comparison normalizes BOTH sides, but each against its OWN volatile values, not a shared context. A raw harvested `session.jsonl` bakes in the recording run's session id, cwd, and timestamps; the replay run produces fresh ones. `normalizeSessionLog` scrubs cwd by exact string match, so normalizing the fixture against the *replay* run's cwd would leave the recorded cwd in the header unscrubbed and the compare would fail. The harness therefore derives the fixture's normalize context from its OWN header line (`{ type:'session', id, cwd }`) — `fixtureContext()` in `acp.snapshot.ts` — so both sides scrub to the same `{{sessionId}}`/`{{cwd}}` tokens. An authored fixture copied from the old golden already carries the normalized header (`id:'{{sessionId}}'`, `cwd:'{{cwd}}'`), which yields those tokens as the volatile values and scrubs idempotently. The session-log side uses a plain normalized-string `toEqual`, NOT `toMatchFileSnapshot`, so a run never overwrites the fixture. diff --git a/docs/rfc/proposed/2026-06-18-agent-lifecycle-and-ownership-seams.md b/docs/rfc/proposed/2026-06-18-agent-lifecycle-and-ownership-seams.md deleted file mode 100644 index ea98277d0e..0000000000 --- a/docs/rfc/proposed/2026-06-18-agent-lifecycle-and-ownership-seams.md +++ /dev/null @@ -1,26 +0,0 @@ -# RFC: Agent lifecycle and ownership seams - -Status: proposed - -## Problem - -Several ACP and tool-bash limitations are symptoms of the same missing seam: plugins can create or resume agents through `ctx.agents`, but they cannot own and dispose one agent independently, and long-running bash tasks carry no stable owner in the executor itself. ACP currently aborts and awaits agents on disconnect, but cannot unregister just that session's agent; `session/cancel` cannot cancel queued-but-not-yet-started work; and `tool-bash` keeps task ownership in a plugin-local `Map`, so an HMR reload can make an old task look unowned. - -## Proposal - -Add explicit lifecycle ownership to the agent factory and explicit ownership metadata to background tasks. - -1. `ctx.agents.create/resume` should return an `AgentHandle` (or add an adjacent method) that exposes the `Agent` plus an async disposer. The disposer unregisters the agent, aborts queued/running work, and resolves only when the driver loop reaches quiescence. -2. Add a queue-aware cancel primitive to the `Agent` interface. It must clear queued work that has not started, abort the current step if one exists, and make `whenIdle()` wait for the post-cancel quiescent state. ACP `session/cancel` and bridge teardown then become honest cancellation, not best-effort pre-step cancellation. -3. Move background task ownership into the bash seam. `BashExecSpec` or `BashTask` should carry a stable owner token, preferably the session id rather than the `Agent` object identity. `bash_output`/`bash_kill` then ask the executor for ownership rather than relying on a `tool-bash` instance-local map. - -## Acceptance Criteria - -- ACP disconnect/session close leaves no registered agent for that session, even when `session/load` races teardown. -- `session/cancel` before a queued prompt starts prevents that prompt from running and cannot batch the next prompt into the cancelled turn. -- A `tool-bash` HMR reload does not make an existing background task readable or killable by a different session. -- Existing non-ACP demos still work without managing handles explicitly; config-created agents remain owned by the `AgentLoop` plugin fiber. - -## Risks - -This touches public interfaces (`Agent`, `AgentFactory`, and the bash seam), so it should not be smuggled into a local ACP patch. The compatibility trap is preserving the simple synchronous `Agent.send()` ergonomics while adding a robust async lifecycle path for owners that need it. diff --git a/docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md b/docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md deleted file mode 100644 index 06bbe1a93d..0000000000 --- a/docs/rfc/proposed/2026-06-18-shared-persistence-write-coordinator.md +++ /dev/null @@ -1,24 +0,0 @@ -# RFC: Shared persistence write coordinator - -Status: proposed - -## Problem - -`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration is now duplicated: per-session state, `session/created` adoption, backend-specific prefix reads, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. The pure seed-prefix collision and serializability guards have already moved into the seam package; the remaining orchestration is still correctness-heavy and already receives the same fixes twice. - -## Proposal - -Extract a backend-agnostic coordinator into `dsh-session-persistence`. The coordinator owns live-session adoption, buffering, cursor filtering, per-id serialization, and disposal quiescence. Concrete backends provide small hooks for durable operations: create lazy state, find/load stored prefix, append a contiguous batch, update summary, delete, and list. - -The public `SessionPersistence` service shape can stay the same. The coordinator can be an internal exported helper or protected base class used by first-party backends; third-party backends may still implement the abstract service directly if their write path is different. - -## Acceptance Criteria - -- JSONL and SQLite keep passing the existing shared `runPersistenceContract`. -- HMR/adoption/collision tests move to a shared coordinator test suite and run once for each backend through hook-driven fixtures. -- Backend-specific tests focus on storage mechanics only: JSONL path safety/fsync/sidecar behavior and SQLite schema/WAL/transaction behavior. -- A future backend does not need to copy the current `session/event` → buffer → flush orchestration. - -## Risks - -The current duplication is verbose but explicit. A coordinator must not hide storage-specific durability semantics or make unusual backends fight an inheritance hierarchy. Prefer narrow hooks and contract tests over a large framework. diff --git a/docs/rfc/proposed/2026-06-16-typed-event-schemas.md b/docs/rfc/proposed/architecture/2026-06-16-typed-event-schemas.md similarity index 93% rename from docs/rfc/proposed/2026-06-16-typed-event-schemas.md rename to docs/rfc/proposed/architecture/2026-06-16-typed-event-schemas.md index 7ee53a6cb4..a48caf11d4 100644 --- a/docs/rfc/proposed/2026-06-16-typed-event-schemas.md +++ b/docs/rfc/proposed/architecture/2026-06-16-typed-event-schemas.md @@ -6,9 +6,9 @@ Status: proposed ## Problem -The harness models its core vocabulary — content blocks, message sources, finish reasons, turn triggers, turn-end reasons, and session events — as **merge-extensible maps**: a TypeScript `interface` (e.g. `SessionEventMap`, `ContentBlockMap`) that plugins augment via declaration merging, with the public union derived as `Map[keyof Map]`. This is the repo's universal extension pattern, documented in [docs/architecture.md](../../architecture.md) ("The same merge-extensible-map pattern is used for `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`") and relied on by the `defineTool` `InferArgs` DSL and the `assertNever` exhaustiveness convention. +The harness models its core vocabulary — content blocks, message sources, finish reasons, turn triggers, turn-end reasons, and session events — as **merge-extensible maps**: a TypeScript `interface` (e.g. `SessionEventMap`, `ContentBlockMap`) that plugins augment via declaration merging, with the public union derived as `Map[keyof Map]`. This is the repo's universal extension pattern, documented in [docs/architecture.md](../../../architecture.md) ("The same merge-extensible-map pattern is used for `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`") and relied on by the `defineTool` `InferArgs` DSL and the `assertNever` exhaustiveness convention. -The pattern is **compile-time only**. The types vanish at runtime: there is no schema object to validate an incoming value against, parse untrusted input with, or enumerate at runtime. Two concrete consequences surfaced in review of [the session-persistence work](../implemented/2026-06-14-session-persistence.md) (#33): +The pattern is **compile-time only**. The types vanish at runtime: there is no schema object to validate an incoming value against, parse untrusted input with, or enumerate at runtime. Two concrete consequences surfaced in review of [the session-persistence work](../../implemented/architecture/2026-06-14-session-persistence.md) (#33): 1. **Persistence treats `event.data` as opaque JSON.** The JSONL/SQLite backends `JSON.stringify`/`JSON.parse` each event verbatim; the only runtime guard is `isJsonValue` (round-trip serializability — rejects BigInt, functions, cycles, non-finite numbers, …), NOT structural validation. A corrupted-but-still-JSON event datum (wrong field types, missing fields) round-trips silently and is only caught later, if at all, by a consumer's `switch`. 2. **No runtime contract for plugin-added variants.** A plugin that declaration-merges a new `SessionEventMap` key gets compile-time typing for its own code, but nothing validates that the values it produces match the shape it declared — at the producer, at the persistence boundary, or on reload. @@ -32,7 +32,7 @@ A migration of the event/vocabulary surface to runtime schemas touches, at minim - **The event producers** — 16 `session.append(...)` call sites in the loop — unchanged in shape but now validated at the boundary. - **~7 switch-consumers** that branch on these unions: `deriveMessages` (`dsh-session`), `BlockAssembler` (`dsh-llm`), the `dsh-invariants` plugin, both LLM adapters (`dsh-llm-deepseek`, `dsh-llm-pi-ai`), and the tool schema layer (`dsh-tools`). The `assertNever`-on-closed-unions vs fall-through-on-extensible-unions convention (a documented lint rule) would need rethinking — runtime variants are not statically exhaustive. - **The `defineTool` `InferArgs` DSL** (`dsh-tools`), which derives zero-cast `execute` arg types from a compile-time schema spec — the showcase of the current approach. -- **Docs**: architecture.md (the pattern is described as foundational), [dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md), and any RFC that references the pattern. +- **Docs**: architecture.md (the pattern is described as foundational), [dev-mode invariants](../../implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md), and any RFC that references the pattern. This is a HUGE change. It is not in scope for the RFC-009 session-persistence work and must not be smuggled in through it. diff --git a/docs/rfc/proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md b/docs/rfc/proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md new file mode 100644 index 0000000000..4f034e3020 --- /dev/null +++ b/docs/rfc/proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md @@ -0,0 +1,35 @@ +# RFC: Extract a generic long-running tool runtime + +Status: proposed + +## Problem + +The bash capability seam supports both foreground commands and long-running background tasks. Background support is large: the abstract executor exposes `start`, `get`, `ownerOf`, `list`, `readOutput`, `kill`, and `onTaskDone`; the local executor tracks tasks, incremental reads, owner tokens, process cleanup, and completion listeners; the model sees three tools (`bash`, `bash_output`, `bash_kill`); the tool plugin injects completion notices back into the owning agent's session. The local executor fences task access behind owner tokens because predictable global task ids are a cross-session read/kill hazard. + +The [tool cookbook](../../../cookbook/adding-a-tool.md) already points at the real design smell: background bash is really generic long-running-tool infrastructure living inside one tool. If future tools need background execution, polling, kill, ownership, and completion notices, those semantics should not be hidden in `dsh-bash`. + +## Proposal + +Move long-running task semantics above bash into a tool-agnostic runtime. Bash remains able to run background commands, but it stops owning the general concepts of task ids, ownership tokens, polling, cancellation, completion notifications, and model-facing "read/kill this task" commands. + +The runtime should own: + +- Stable task ids and owner tokens keyed to the calling session/agent. +- Registration of a long-running task with a producer for incremental output and a completion promise. +- Generic read/cancel/list operations with the same cross-session authorization rule for every tool. +- Completion notification injection into the owning session. +- Presentation hooks for pending/running/completed task state, with bash supplying only command-specific labels and output formatting. + +`dsh-bash` then keeps the bash-specific execution contract: resolve a request into a command spec, run a foreground command, or start a process and hand its streams/process handle to the generic runtime. `dsh-tool-bash` keeps the model-facing command tool, but the follow-up operations become generic long-running-tool operations or a shared utility that bash registers with, rather than bespoke `bash_output`/`bash_kill` plumbing. + +## Acceptance criteria + +- The bash-specific packages no longer define the generic task registry, owner-token authorization, polling, cancellation, or completion-notification machinery. +- A shared long-running-task service or tool layer owns those semantics and is documented as the path for any future background-capable tool. +- Bash background behavior remains available through the shared layer, with tests proving cross-session isolation still holds. +- ACP and snapshot fixtures render background bash through the shared task vocabulary, not through bash-only lifecycle semantics. +- The [tool cookbook](../../../cookbook/adding-a-tool.md) points long-running tools at the shared runtime instead of telling each tool to invent its own task protocol. + +## What we give up + +The bash package loses local ownership of an already-working background-task implementation, and the implementing PR may temporarily churn model-facing tool names or transcript presentation. That churn is worthwhile if it leaves one background-task contract instead of making every future long-running tool clone bash's private protocol. diff --git a/docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md b/docs/rfc/proposed/feature/2026-06-14-acp-agent-client-protocol.md similarity index 53% rename from docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md rename to docs/rfc/proposed/feature/2026-06-14-acp-agent-client-protocol.md index 9b9dc8e7b6..675e295c2c 100644 --- a/docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md +++ b/docs/rfc/proposed/feature/2026-06-14-acp-agent-client-protocol.md @@ -3,7 +3,7 @@ Status: proposed -> **Implementation status (MVP landed):** steps 1, 2, 3, 4, 6, 7, 8 are implemented in `packages/acp` + `examples/acp-agent`. **Step 5 (the `session/request_permission` permission gate) is deferred** — the bridge ships a pass-through (tools run with the executor's full authority) marked `TODO(rfc010-permission-gate)`, and lays down only the `WeakMap` ownership seam the gate will build on. Status stays `proposed` until the gate lands. One further best-effort limitation is tracked as `TODO(rfc010-cancel-prestep)`: `session/cancel` aborts a running step and settles the RPC as `cancelled`, but a turn still queued (not yet started) when the cancel arrives may execute before the abort takes effect, pending a loop-level pre-step cancel. **Per-session `cwd` is now honored** (lifting the original "launch the server in the workspace root" restriction — see § Deferred): `session/new` accepts any absolute `cwd`, and `session/load` requires the request `cwd` to match the persisted session `cwd` so the editor and bash executor agree on the workspace. +> **Implementation status (MVP landed):** steps 1, 2, 3, 4, 6, 7, 8 are implemented in `packages/ui/acp` + `examples/acp-agent`. **Step 5 (the `session/request_permission` permission gate) is deferred** — the bridge ships a pass-through (tools run with the executor's full authority) marked `TODO(rfc010-permission-gate)`, and lays down only the `WeakMap` ownership seam the gate will build on. Status stays `proposed` until the gate lands. `session/cancel` is the queue-aware `agent.cancel()`: it aborts a running step, clears queued + steering work, and drops a turn that is about to start, so a queued-but-not-yet-started prompt never runs and a later prompt cannot be batched into the cancelled turn. **Per-session `cwd` is now honored** (lifting the original "launch the server in the workspace root" restriction — see § Deferred): `session/new` accepts any absolute `cwd`, and `session/load` requires the request `cwd` to match the persisted session `cwd` so the editor and bash executor agree on the workspace. ## Problem @@ -11,13 +11,13 @@ The coding agent is reachable only through the readline `stdio-chat` plugin: it Editors are converging on the Agent Client Protocol (ACP), which Zed and others speak: JSON-RPC 2.0 over newline-delimited stdio, modeled on the Language Server Protocol. An editor boots the agent as a subprocess and exchanges `initialize` / `session/new` / `session/prompt`, rendering streamed `session/update` notifications and `session/request_permission` prompts. The goal is for the agent to be a drop-in ACP server — implement the protocol once and run in any ACP client, with no per-editor glue. -This RFC has a hard prerequisite on [session persistence](../implemented/2026-06-14-session-persistence.md): it assumes durable session persistence (the `SessionPersistence` service and the async `AgentLoop.resume` seam) is implemented, so resuming a session via `session/load` is in scope. None of those APIs exist yet — `AgentLoop` currently exposes only the synchronous `create` — so ACP must land after, or in the same change as, [session persistence](../implemented/2026-06-14-session-persistence.md), and pins to its `resume(agentId, resumeSessionId)` contract. Session persistence persists every `SessionEvent` verbatim (including `assistant/chunk`), so a loaded session has the stream chunks needed to replay turns to the client. +This RFC has a hard prerequisite on [session persistence](../../implemented/architecture/2026-06-14-session-persistence.md): it assumes durable session persistence (the `SessionPersistence` service and the async `AgentLoop.resume` seam) is implemented, so resuming a session via `session/load` is in scope. None of those APIs exist yet — `AgentLoop` currently exposes only the synchronous `create` — so ACP must land after, or in the same change as, [session persistence](../../implemented/architecture/2026-06-14-session-persistence.md), and pins to its `resume(agentId, resumeSessionId)` contract. Session persistence persists every `SessionEvent` verbatim (including `assistant/chunk`), so a loaded session has the stream chunks needed to replay turns to the client. ## Proposal -A new plugin package `@deepseek-ai/dsh-acp` — a client-driver / UI plugin, the structured analogue of `stdio-chat`. It is NOT a change to the loop and NOT an [capability seams](../implemented/2026-06-13-capability-seams.md) interface/implementation/consumer capability split; it consumes the existing `agent/*` event taxonomy and the `tools/execute` waterfall. +A new plugin package `@deepseek-ai/dsh-acp` — a client-driver / UI plugin, the structured analogue of `stdio-chat`. It is NOT a change to the loop and NOT an [capability seams](../../implemented/architecture/2026-06-13-capability-seams.md) interface/implementation/consumer capability split; it consumes the existing `agent/*` event taxonomy and the `tools/execute` waterfall. -It depends on the official `@agentclientprotocol/sdk` (the `AgentSideConnection` class) — Apache-2.0, actively versioned. The SDK declares a `zod` peer dependency and imports `zod/v4` at runtime, so `packages/acp` must declare `zod` itself (per the workspace dependency constraints). This is the renamed successor to `@zed-industries/agent-client-protocol`, which is now deprecated on npm. +It depends on the official `@agentclientprotocol/sdk` (the `AgentSideConnection` class) — Apache-2.0, actively versioned. The SDK declares a `zod` peer dependency and imports `zod/v4` at runtime, so `packages/ui/acp` must declare `zod` itself (per the workspace dependency constraints). This is the renamed successor to `@zed-industries/agent-client-protocol`, which is now deprecated on npm. The mapping between ACP and existing harness seams — each row names the seam and any required extension: @@ -25,7 +25,7 @@ The mapping between ACP and existing harness seams — each row names the seam a |---|---|---| | `initialize` | static handler | negotiate `protocolVersion` (echo the supported version, else error); advertise text-only `promptCapabilities` and `loadSession: true`; report agent name/version | | `session/new {cwd, mcpServers, additionalDirectories}` → `{sessionId}` | the `dsh-agent` create factory (see Dependency note + Plan) | the seam must accept `{ sessionId, meta }` so the ACP-generated `sessionId` becomes the live/persisted session id and the validated `cwd` is attached as the `SessionHeader` (today `AgentLoop.create(id)` hardcodes `${id}-session` and takes no metadata); reject a 2nd session (single-session MVP, see [ACP multi-session](2026-06-14-acp-multi-session.md)); `cwd` validated (require absolute) — any absolute cwd is honored: it becomes the session's `SessionHeader.cwd` and the default bash workdir (per-session cwd, see § Deferred → RESOLVED), so the server need not launch in the workspace; non-empty `mcpServers` and `additionalDirectories` are rejected for the MVP because silently ignoring requested servers/roots would desync the client's tool and filesystem-scope UI | -| `session/load {sessionId, cwd, mcpServers, additionalDirectories}` | the `dsh-agent` resume factory ([session persistence](../implemented/2026-06-14-session-persistence.md) + Dependency note) | load `{ meta, events }`, seed the session, re-derive history via `deriveMessages()`, replay prior turns to the client as `session/update` per the ACP load contract; `mcpServers` and `additionalDirectories` rejected as in `session/new` | +| `session/load {sessionId, cwd, mcpServers, additionalDirectories}` | the `dsh-agent` resume factory ([session persistence](../../implemented/architecture/2026-06-14-session-persistence.md) + Dependency note) | load `{ meta, events }`, seed the session, re-derive history via `deriveMessages()`, replay prior turns to the client as `session/update` per the ACP load contract; `mcpServers` and `additionalDirectories` rejected as in `session/new` | | `session/prompt {prompt}` | `agent.send()` (idle) | text blocks → `TextBlock`; reject image/audio per advertised capabilities; one in-flight prompt per session | | resolve `session/prompt` → `{stopReason}` | `agent/turn-end` (extended, see Plan) | map the harness kebab `TurnEndReason` to the ACP snake_case `StopReason` wire enum: `completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`(cancel)→`cancelled`, plus `refusal`/`max_turn_requests` when applicable; honor the batch-into-one-turn and send-not-synchronously-running settle semantics | | `session/update: agent_message_chunk` | `agent/stream-chunk` `text-delta` only | do NOT also emit on `block-end(TextBlock)` — it carries the fully-assembled block and would duplicate the streamed text | @@ -33,41 +33,41 @@ The mapping between ACP and existing harness seams — each row names the seam a | `session/update: tool_call` (pending→in_progress) | `session/event` `tool/call` | demux via a Session→sessionId map; `kind` inferred from the tool name | | `session/update: tool_call_update` (completed/failed) | `session/event` `tool/result` | a throwing `tools/execute` yields NO `tool/result` → fail the pending tool UI from `agent/error`/turn-end | | `session/request_permission {sessionId, toolCall, options}` | prepended `tools/execute` listener | no-op unless `exec.agent` is ACP-owned; await the outcome; `selected/allow_*` → `next()`; `reject_*`/`cancelled` → veto `ToolExecutionResult{isError}` | -| `session/cancel` (notification) | `agent.abort(reason)` | settle the in-flight prompt as `cancelled`; resolve any pending permission as `cancelled` exactly once | +| `session/cancel` (notification) | `agent.cancel(reason)` | the queue-aware cancel (abort running step, clear queued + steering, drop an about-to-start turn); settle the in-flight prompt as `cancelled`; resolve any pending permission as `cancelled` exactly once | -The permission gate is the first real consumer of the `tools/execute` veto seam (the documented "single veto/sandbox/permission seam" plus the deferred "Permission system" TODO in [docs/architecture.md](../../architecture.md)). It is a single global listener registered with `prepend: true` so it runs before any other tool wrapper. `ToolExecution.agent` is optional and the `Agent` interface carries no origin marker, so the bridge tracks ownership itself: it records each agent it creates in a `WeakMap` and the gate no-ops (calls `next()` immediately) for any `exec.agent` it does not own — non-ACP agents and the no-agent case pass straight through. For an owned agent it resolves the session, issues `session/request_permission`, and stores the pending resolver on that session's record so the outcome — or a `session/cancel`/connection-close — settles it exactly once. +The permission gate is the first real consumer of the `tools/execute` veto seam (the documented "single veto/sandbox/permission seam" plus the deferred "Permission system" TODO in [docs/architecture.md](../../../architecture.md)). It is a single global listener registered with `prepend: true` so it runs before any other tool wrapper. `ToolExecution.agent` is optional and the `Agent` interface carries no origin marker, so the bridge tracks ownership itself: it records each agent it creates in a `WeakMap` and the gate no-ops (calls `next()` immediately) for any `exec.agent` it does not own — non-ACP agents and the no-agent case pass straight through. For an owned agent it resolves the session, issues `session/request_permission`, and stores the pending resolver on that session's record so the outcome — or a `session/cancel`/connection-close — settles it exactly once. -Lifecycle and disposal: the connection, listeners, and in-flight permission promises register via `ctx.effect`/`ctx.on`; teardown is async and awaits quiescence — close the connection, settle/reject pending permissions, `agent.abort()`, and wait for the agent to settle. The disposal-settle signal must come from the `dsh-agent` interface, not the loop: `agent.done` exists only on the concrete `LoopAgent`, so the bridge instead observes `agent/status` reaching `idle`/`disposed` (or the RFC lifts a quiescence promise onto the `Agent` interface). Every listener contains its `send()` exceptions (log, never reject the turn) because stream chunks are emitted inside the model step, so a throwing listener would corrupt the turn. +Lifecycle and disposal: the connection, listeners, and in-flight permission promises register via `ctx.effect`/`ctx.on`; teardown is async and must *reach* quiescence, not just request it — close the connection, settle/reject pending permissions, and dispose each owned agent through its `AgentHandle.dispose()` (which stops the loop, `await`s its exit, and unregisters). Owner teardown goes through that handle seam, not the loop's concrete `agent.done` (which exists only on `ReactLoopAgent`); a non-owner that merely wants to *observe* the current work settling without tearing the agent down awaits the interface-level `agent.whenIdle()`. Every listener contains its `send()` exceptions (log, never reject the turn) because stream chunks are emitted inside the model step, so a throwing listener would corrupt the turn. -**Dependency note (architecture rule).** [docs/architecture.md](../../architecture.md) states "plugins depend on interface packages, never on `dsh-agent-loop`." Creating and resuming agents is currently only on the concrete `AgentLoop` (`ctx.agentLoop`), so this RFC proposes adding an **abstract create/resume factory** to the `dsh-agent` interface (registry-level `create({ sessionId, meta })` / `resume(...)`), implemented by the loop, so `dsh-acp` injects only `agents` (the interface) and the dependency rule holds. The alternative — injecting the concrete `agentLoop` and recording a documented exception in the architecture doc — is explicitly the non-preferred fallback. +**Dependency note (architecture rule).** [docs/architecture.md](../../../architecture.md) states "plugins depend on interface packages, never on `dsh-agent-loop`." Creating and resuming agents is currently only on the concrete `AgentLoop` (`ctx.agentLoop`), so this RFC proposes adding an **abstract create/resume factory** to the `dsh-agent` interface (registry-level `create({ sessionId, meta })` / `resume(...)`), implemented by the loop, so `dsh-acp` injects only `agents` (the interface) and the dependency rule holds. The alternative — injecting the concrete `agentLoop` and recording a documented exception in the architecture doc — is explicitly the non-preferred fallback. ## Plan -1. Package scaffold `packages/acp/` per [the cookbook](../../cookbook/adding-a-package.md); add `@agentclientprotocol/sdk` and `zod`. Add the abstract create/resume factory to `dsh-agent` (the interface) so the bridge can `inject: ['agents', 'sessions', 'tools', 'sessionPersistence']` without depending on the concrete loop; `sessionPersistence` is required because `session/load` advertises `loadSession: true`. (Fallback only if the factory is judged not worth it: inject `agentLoop` directly and record the architecture-rule exception in `docs/architecture.md`.) +1. Package scaffold `packages/ui/acp/` per [the cookbook](../../../cookbook/adding-a-package.md); add `@agentclientprotocol/sdk` and `zod`. Add the abstract create/resume factory to `dsh-agent` (the interface) so the bridge can `inject: ['agents', 'sessions', 'tools', 'sessionPersistence']` without depending on the concrete loop; `sessionPersistence` is required because `session/load` advertises `loadSession: true`. (Fallback only if the factory is judged not worth it: inject `agentLoop` directly and record the architecture-rule exception in `docs/architecture.md`.) 2. Connection plus `initialize`/`session/new`: wire `AgentSideConnection` to stdin/stdout; protocolVersion negotiation; the single-session guard; create the live session through the new `{ sessionId, meta }` factory seam (so the ACP `sessionId` and validated `cwd` become the session's id and header); the `sessionId↔agent` and `Session↔sessionId` maps. -3. Internal edit — turn-end reason fidelity (sanctioned: edit internals to fit ACP). Extend `TurnEndReasonMap` in the proper places: (a) declaration-merge a `max-tokens` variant in the owning package (`packages/session/src/types.ts`, alongside `completed|aborted|error|disposed`) — add `max-tokens` because `FinishReasonMap` produces it (DeepSeek maps `length` → `max-tokens`); do not add `refusal`, since no current adapter produces it (unknown DeepSeek finish reasons collapse to `error`), but leave a comment in `TurnEndReasonMap` noting `refusal` should be added when an adapter first emits it (`FinishReasonMap` is merge-extensible); (b) make `agent-loop`'s `loop.ts` populate the reason from the model `finish` chunk — `assembler.finish` lives inside `runStep`, so `runStep` must return it up to `runTurn`, and the rule is "the last step's finish reason wins, but any `max-tokens` in the turn surfaces as `max-tokens`"; (c) no consumer exhaustively switches over `TurnEndReason` today (the invariants plugin switches on `SessionEventType`, and `deriveMessages` ignores `turn/end`), so adding `max-tokens` is a non-breaking extension — but recheck before landing; (d) update [docs/architecture.md](../../architecture.md) (the CI-verified loop-lifecycle/event-taxonomy doc) and the affected package READMEs/JSDoc (`dsh-session`, `dsh-agent`, `dsh-agent-loop`) per the repo doc-sync policy. This replaces a fragile "observe the finish chunk in the bridge" hack with a real, documented contract. +3. Internal edit — turn-end reason fidelity (sanctioned: edit internals to fit ACP). Extend `TurnEndReasonMap` in the proper places: (a) declaration-merge a `max-tokens` variant in the owning package (`packages/core/session/src/types.ts`, alongside `completed|aborted|error|disposed`) — add `max-tokens` because `FinishReasonMap` produces it (DeepSeek maps `length` → `max-tokens`); do not add `refusal`, since no current adapter produces it (unknown DeepSeek finish reasons collapse to `error`), but leave a comment in `TurnEndReasonMap` noting `refusal` should be added when an adapter first emits it (`FinishReasonMap` is merge-extensible); (b) make `agent-loop`'s `loop.ts` populate the reason from the model `finish` chunk — `assembler.finish` lives inside `runStep`, so `runStep` must return it up to `runTurn`, and the rule is "the last step's finish reason wins, but any `max-tokens` in the turn surfaces as `max-tokens`"; (c) no consumer exhaustively switches over `TurnEndReason` today (the invariants plugin switches on `SessionEventType`, and `deriveMessages` ignores `turn/end`), so adding `max-tokens` is a non-breaking extension — but recheck before landing; (d) update [docs/architecture.md](../../../architecture.md) (the CI-verified loop-lifecycle/event-taxonomy doc) and the affected package READMEs/JSDoc (`dsh-session`, `dsh-agent`, `dsh-agent-loop`) per the repo doc-sync policy. This replaces a fragile "observe the finish chunk in the bridge" hack with a real, documented contract. 4. Prompt-turn streaming plus load: translate `agent/stream-chunk` and `session/event` into `session/update`; resolve `session/prompt` on settle, mapping the harness `TurnEndReason` to the ACP `StopReason` wire enum (`completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`→`cancelled`) — a small total function with a test asserting the exact wire strings, since the SDK rejects an unknown `stopReason`. Concrete correlation, since the loop batches queued messages into one turn and `send()` does not synchronously flip to running: install listeners before `send()`; gate on an observed `agent/turn-start` (confirms work was accepted) then resolve on the next `agent/turn-end`; reject an empty/whitespace prompt up front rather than calling `send()` (no turn would ever start, so the RPC would hang). Implement `session/load` on the session-persistence resume seam. 5. Permission gate: a single `tools/execute` listener registered with `prepend: true`, owning a `WeakMap` of bridge-created agents; no-op (`next()`) for unowned/no-agent calls; for owned calls → `session/request_permission` → allow (`next()`) / veto; settle the stored resolver exactly once on outcome, cancel, or connection close. -6. Example wiring (extract a shared base). `@cordisjs/plugin-include` is itself a plugin entry that resets `ctx.baseUrl` and loads a path, so a child `cordis.yml` can nest-include a shared base; the extraction is safe because every dependent plugin declares `inject` (loader groups initialize via `Promise.all`, so YAML order is NOT the dependency mechanism — never rely on it). Extract the provider/tool core (`llm, sessions, system-prompt, tools, agents, invariants, llm-deepseek, bash-local, tool-bash`) into `examples/base.yml`; have both `coding-agent` and a new `examples/acp-agent/` include it and add their own UI plugin plus logger. Keep `agent-loop` per-example (NOT in the base): `AgentLoop` creates its configured agents in its constructor, and the two examples disagree — `coding-agent` needs a pre-created `main` (its `stdio-chat` calls `ctx.agents.get('main')`), while `acp-agent` must pre-create none (ACP `session/new` creates agents). So `coding-agent` declares `agent-loop` with `agents: [{ id: main, … }]` and `acp-agent` with `agents: []`. `acp-agent` loads `dsh-session-persistence-jsonl` (from [session persistence](../implemented/2026-06-14-session-persistence.md) — required for `session/load`), omits the stdout logger (see Risks), and adds `pnpm run demo:acp` plus the Zed `agent_servers` snippet. -7. Tests (the repo cares a lot here): a property-based test for the protocol shape (precedent: [property-based testing](../implemented/2026-06-11-property-based-testing.md)) — fuzz arbitrary harness event sequences and assert ACP-stream invariants (never a `tool_call_update` before its `tool_call`; exactly one `session/prompt` resolution per prompt; monotonic, well-formed ordering; `stopReason` in the legal set); codec unit tests over an in-memory `Duplex` pair (drive `AgentSideConnection` without a subprocess; assert exact frames for `initialize`, `session/new`, a full prompt turn); the mandatory HMR-safety test (dispose the fiber; assert the connection closed, all `ctx.on` listeners gone, any in-flight `request_permission` settled); failure-path tests (connection closes mid-stream; closes with a permission pending; a notification `send()` rejects but the turn survives; `finish{kind:'error'|'aborted'}`; a `tools/execute` throw with no `tool/result`; a second `session/new` rejected; a `session/prompt` while one is in flight; an empty prompt rejected without hanging; a `session/load` re-derives identical history and replays it); and an e2e (`*.e2e.ts`, self-skips without `DEEPSEEK_API_KEY`) that boots `examples/acp-agent`, connects a `ClientSideConnection`, sends a real prompt, owns and disposes the harness in `afterEach`, and verifies the world (files on disk), not the agent's self-report. -8. Docs: module/JSDoc plus a package README; extend [the extension cookbook](../../cookbook/extension-cookbook.md) with the client-driver pattern. Flip Status to `implemented` on landing; record a decision in this RFC only if it proves durable, contested, and surprising (candidates: the `tools/execute` permission-ownership rule, the npm-dependency choice) — not auto-required. +6. Example wiring (extract a shared base). `@cordisjs/plugin-include` is itself a plugin entry that resets `ctx.baseUrl` and loads a path, so a child `cordis.yml` can nest-include a shared base; the extraction is safe because every dependent plugin declares `inject` (loader groups initialize via `Promise.all`, so YAML order is NOT the dependency mechanism — never rely on it). Extract the provider/tool core (`llm, sessions, system-prompt, tools, agents, invariants, llm-deepseek, bash-local, tool-bash`) into `examples/base.yml`; have both `coding-agent` and a new `examples/acp-agent/` include it and add their own UI plugin plus logger. Keep `agent-loop` per-example (NOT in the base): `AgentLoop` creates its configured agents in its constructor, and the two examples disagree — `coding-agent` needs a pre-created `main` (its `stdio-chat` calls `ctx.agents.get('main')`), while `acp-agent` must pre-create none (ACP `session/new` creates agents). So `coding-agent` declares `agent-loop` with `agents: [{ id: main, … }]` and `acp-agent` with `agents: []`. `acp-agent` loads `dsh-session-persistence-jsonl` (from [session persistence](../../implemented/architecture/2026-06-14-session-persistence.md) — required for `session/load`), omits the stdout logger (see Risks), and adds `pnpm run demo:acp` plus the Zed `agent_servers` snippet. +7. Tests (the repo cares a lot here): a property-based test for the protocol shape (precedent: [property-based testing](../../implemented/testing/2026-06-11-property-based-testing.md)) — fuzz arbitrary harness event sequences and assert ACP-stream invariants (never a `tool_call_update` before its `tool_call`; exactly one `session/prompt` resolution per prompt; monotonic, well-formed ordering; `stopReason` in the legal set); codec unit tests over an in-memory `Duplex` pair (drive `AgentSideConnection` without a subprocess; assert exact frames for `initialize`, `session/new`, a full prompt turn); the mandatory HMR-safety test (dispose the fiber; assert the connection closed, all `ctx.on` listeners gone, any in-flight `request_permission` settled); failure-path tests (connection closes mid-stream; closes with a permission pending; a notification `send()` rejects but the turn survives; `finish{kind:'error'|'aborted'}`; a `tools/execute` throw with no `tool/result`; a second `session/new` rejected; a `session/prompt` while one is in flight; an empty prompt rejected without hanging; a `session/load` re-derives identical history and replays it); and an e2e (`*.e2e.ts`, self-skips without `DEEPSEEK_API_KEY`) that boots `examples/acp-agent`, connects a `ClientSideConnection`, sends a real prompt, owns and disposes the harness in `afterEach`, and verifies the world (files on disk), not the agent's self-report. +8. Docs: module/JSDoc plus a package README; extend [the extension cookbook](../../../cookbook/extension-cookbook.md) with the client-driver pattern. Flip Status to `implemented` on landing; record a decision in this RFC only if it proves durable, contested, and surprising (candidates: the `tools/execute` permission-ownership rule, the npm-dependency choice) — not auto-required. Deferred (each names its owning future work): - Multiplexing concurrent sessions → [ACP multi-session](2026-06-14-acp-multi-session.md). - ~~`cwd` honoring.~~ **RESOLVED.** Originally there was no path from `session/new.cwd` to the bash workdir (`tool-bash` forwarded only an explicit `args.workdir`; `LocalBashExecutor.resolve` defaulted to its own config or `process.cwd()`), so the MVP validated `cwd` (require absolute) AND required the server to launch in the workspace root, erroring on a mismatch. This is now lifted: the validated `cwd` is stored as `SessionHeader.cwd`, and `dsh-tool-bash` defaults the bash workdir to the calling agent's `session.header.cwd` (an explicit model `workdir` still wins; a relative one resolves against it). Any absolute `cwd` is honored — the server need not launch in the workspace, and N sessions can each target a different directory. Widening scope beyond the single cwd (`additionalDirectories`) remains deferred. -- Client `terminal/*` proxying (a live editor terminal) and `fs/*` (editor-rendered diffs) — a future `BashExecutor` over the [capability seams](../implemented/2026-06-13-capability-seams.md) bash seam, gated on `clientCapabilities.terminal`. +- Client `terminal/*` proxying (a live editor terminal) and `fs/*` (editor-rendered diffs) — a future `BashExecutor` over the [capability seams](../../implemented/architecture/2026-06-13-capability-seams.md) bash seam, gated on `clientCapabilities.terminal`. - Image/audio prompts (blocked on the DeepSeek adapter, which skips `image` blocks today), modes, auth, `available_commands`/slash-commands, `plan`, and `usage_update`. ## Risks stdout is the protocol — guaranteed by config, not by monkey-patching. The console logger writes through `console.log` to stdout, so any stdout UI/logger plugin corrupts JSON-RPC. The guarantee is config-only: the `acp-agent` example loads no stdout plugin (no console logger, no `stdio-chat`) and, if logging is wanted, uses a stderr exporter. A defensive process-wide `process.stdout.write`/`console.log` hijack inside `dsh-acp` is explicitly rejected — it lives outside Cordis' effect-scoped, HMR-friendly plugin model, races the connection's own stdout handoff, and fights the logger. A test asserts the example emits only framed JSON-RPC on stdout. -New third-party runtime dependency plus protocol drift: `@agentclientprotocol/sdk` is young (0.25.x, recently renamed) and evolving. Pin the version and isolate churn to the one bridge package. This is not a vendoring-policy violation — [vendoring Cordis as source](../implemented/2026-06-11-vendor-cordis-as-source.md) vendors the framework; genuine third-party deps already live on npm (`@earendil-works/pi-ai`). +New third-party runtime dependency plus protocol drift: `@agentclientprotocol/sdk` is young (0.25.x, recently renamed) and evolving. Pin the version and isolate churn to the one bridge package. This is not a vendoring-policy violation — [vendoring Cordis as source](../../implemented/process/2026-06-11-vendor-cordis-as-source.md) vendors the framework; genuine third-party deps already live on npm (`@earendil-works/pi-ai`). -Turn-settle and prompt-correlation hazards: honor "queued messages batch into one turn" and "`send()` does not synchronously flip to running" (see `stdio-chat.ts` and the defensive-patterns section of [docs/architecture.md](../../architecture.md)); gate resolution on an observed running→idle transition and handle the empty-prompt / no-work branch so an RPC can't hang. +Turn-settle and prompt-correlation hazards: honor "queued messages batch into one turn" and "`send()` does not synchronously flip to running" (see `stdio-chat.ts` and the defensive-patterns section of [docs/architecture.md](../../../architecture.md)); gate resolution on an observed running→idle transition and handle the empty-prompt / no-work branch so an RPC can't hang. -Permission-await and disposal hangs: a pending `request_permission` whose connection closes or whose turn aborts must settle exactly once; disposal must reach quiescence (observe the interface-level settle signal — `agent/status` reaching `idle`/`disposed`, since `agent.done` is `LoopAgent`-only), not orphan awaits on a closed pipe. +Permission-await and disposal hangs: a pending `request_permission` whose connection closes or whose turn aborts must settle exactly once; disposal must reach quiescence — tear each owned agent down through `AgentHandle.dispose()` (which stops the loop and awaits its exit), rather than orphaning awaits on a closed pipe. The 100% per-file coverage gate (repo policy) makes a branch-heavy protocol bridge real work. Accepted deliberately, surfaced so it isn't a surprise at PR time. diff --git a/docs/rfc/proposed/2026-06-14-acp-multi-session.md b/docs/rfc/proposed/feature/2026-06-14-acp-multi-session.md similarity index 71% rename from docs/rfc/proposed/2026-06-14-acp-multi-session.md rename to docs/rfc/proposed/feature/2026-06-14-acp-multi-session.md index 388ec04880..ed51a2ded8 100644 --- a/docs/rfc/proposed/2026-06-14-acp-multi-session.md +++ b/docs/rfc/proposed/feature/2026-06-14-acp-multi-session.md @@ -3,22 +3,24 @@ Status: proposed -> **Implementation status:** the multi-session bridge (steps 1, 3, 4) and the bash task-ownership isolation are implemented in `packages/acp` + `packages/tool-bash`. **Per-session *permission* ownership is deferred** — it depends on [the ACP support permission gate](2026-06-14-acp-agent-client-protocol.md) (`TODO(rfc010-permission-gate)`), which is itself deferred; the `agent→sessionId` reverse map the gate will route through is in place. Step 2's "real per-session disposer scope" is also deferred (`TODO(rfc010-agent-disposal)`): the bridge demuxes via id-keyed maps and global `ctx.on` listeners (correct and leak-free — disposal drains every session in parallel to quiescence), and a per-agent disposer seam is the follow-up. Status stays `proposed` until per-session permission ownership lands. +> **Implementation status:** the multi-session bridge (steps 1, 3, 4) and the bash task-ownership isolation are implemented in `packages/ui/acp` + `packages/bash/tool-bash`. **Per-session *permission* ownership is deferred** — it depends on [the ACP support permission gate](2026-06-14-acp-agent-client-protocol.md) (`TODO(rfc010-permission-gate)`), which is itself deferred; the `agent→sessionId` reverse map the gate will route through is in place. Step 2's per-session disposer scope is now implemented (see [agent lifecycle & ownership seams](../../implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md)): the factory returns a per-agent `AgentHandle` whose `dispose()` stops the loop, awaits quiescence, unregisters the agent, and removes its session, so a bare client disconnect leaves no registered agent or session-store entry. Status stays `proposed` until per-session permission ownership lands. + +> **Target-client note:** Zed is the current target ACP client, and its ACP client maintains a `HashMap` plus `pending_sessions` for concurrent `session/load` calls. The competing simplification to return to one live session per connection was rejected after checking that target-client shape; this RFC remains the path for finishing multiplexing and per-session permission ownership. See [the rejected simplification](../../rejected/simplification/2026-06-20-single-session-acp-bridge.md). ## Problem [ACP support](2026-06-14-acp-agent-client-protocol.md) ships with a single active session per connection: a second `session/new` is rejected. Editors expect to run several conversations over one agent subprocess — a user opens multiple threads, or a client pre-warms sessions. The single-session guard is a deliberate MVP scope cut, not an architectural limit; this RFC lifts it. -This paragraph is historical: the multi-session bridge has landed. The remaining proposed work is per-session permission ownership plus the lifecycle seams now tracked in [agent lifecycle and ownership seams](2026-06-18-agent-lifecycle-and-ownership-seams.md). +This paragraph is historical: the multi-session bridge has landed. The remaining proposed work is per-session permission ownership plus the lifecycle seams now tracked in [agent lifecycle and ownership seams](../../implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md). ## Proposal The harness core already supports many agents (`AgentRegistry.list()` and `AgentLoop.create` impose no count limit), so multiplexing is a bridge-layer change in `@deepseek-ai/dsh-acp`, not a loop or core change. -- Lift the single-session guard in `session/new`; allow N live sessions, each mapped to its own `LoopAgent`. +- Lift the single-session guard in `session/new`; allow N live sessions, each mapped to its own `ReactLoopAgent`. - The bridge's `sessionId→agent` and `Session→sessionId` maps (introduced single-entry by [the ACP support RFC](2026-06-14-acp-agent-client-protocol.md)) become true multi-entry, plus a third `agent→sessionId` reverse map: the `tools/execute` permission gate receives only `exec.agent` (no sessionId), so it needs an O(1) reverse lookup to find the owning session. Every `agent/*` event and every `session/event` is demuxed strictly by id, so two sessions streaming at once never interleave their `session/update` notifications. - Per-session prompt queues: [the ACP support RFC](2026-06-14-acp-agent-client-protocol.md)'s single-entry in-flight-prompt state becomes multi-entry — one in-flight prompt *per session*, tracked per `sessionId`. -- Per-session cancel routing: `session/cancel` aborts only its own session's agent and settles only that session's in-flight prompt. `agent.abort()` drives a per-agent `AbortController`, so the per-session `exec.signal` is the natural isolation fence. +- Per-session cancel routing: `session/cancel` cancels only its own session's agent (via the queue-aware `agent.cancel()`) and settles only that session's in-flight prompt. The cancel is scoped to that one agent — a per-agent `AbortController` for the running step plus the agent's own queued/steering FIFOs — so it never touches another session's stream or pending prompt. - Per-session permission ownership: a `session/request_permission` and its outcome are bound to the originating session via the reverse map, so a permission prompt or a cancel in one session can never resolve another session's pending permission. ## Plan diff --git a/docs/rfc/proposed/2026-06-15-optional-code-mode.md b/docs/rfc/proposed/feature/2026-06-15-optional-code-mode.md similarity index 85% rename from docs/rfc/proposed/2026-06-15-optional-code-mode.md rename to docs/rfc/proposed/feature/2026-06-15-optional-code-mode.md index 598986a521..eb41750d1d 100644 --- a/docs/rfc/proposed/2026-06-15-optional-code-mode.md +++ b/docs/rfc/proposed/feature/2026-06-15-optional-code-mode.md @@ -6,7 +6,7 @@ Status: proposed ## Problem -Today the agent loop advertises every registered tool to the model as a native JSON-schema function definition. `ToolRegistry` feeds its schemas into `ctx.systemPrompt`, the loop puts them on `GenerateOptions.tools`, and the adapter serializes them to the provider's function-calling wire format. The model then invokes one `tool-call` block per step, the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../architecture.md)), and **every** intermediate `tool-result` re-enters the model's context on the next request. +Today the agent loop advertises every registered tool to the model as a native JSON-schema function definition. `ToolRegistry` feeds its schemas into `ctx.systemPrompt`, the loop puts them on `GenerateOptions.tools`, and the adapter serializes them to the provider's function-calling wire format. The model then invokes one `tool-call` block per step, the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../architecture.md)), and **every** intermediate `tool-result` re-enters the model's context on the next request. For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each of those round-trips drags the entire intermediate result back into context whether the model needs it or not. @@ -16,9 +16,9 @@ This RFC proposes an **optional** Code Mode for the DeepSeek Harness, covering * ## Proposal -The design follows the codebase's capability-seam pattern ([capability seams](../implemented/2026-06-13-capability-seams.md), the `bash` template) as a three-package split, plus one consumer plugin. Nothing in `dsh-session`, `dsh-agent`, `dsh-agent-loop`, `dsh-llm`, `dsh-tools`, or `dsh-system-prompt` changes. +The design follows the codebase's capability-seam pattern ([capability seams](../../implemented/architecture/2026-06-13-capability-seams.md), the `bash` template) as a three-package split, plus one consumer plugin. Nothing in `dsh-session`, `dsh-agent`, `dsh-agent-loop`, `dsh-llm`, `dsh-tools`, or `dsh-system-prompt` changes. -**Prior art.** `@cloudflare/codemode` validates this shape directly and several of its decisions are adopted below. Its `Executor` interface is deliberately tiny — `execute(code, fns) → { result, error?, logs? }` — with a production `DynamicWorkerExecutor` (isolated Workers) and a six-line `NodeVMExecutor` example as two implementations behind it: exactly the interface/implementation split [the capability-seam pattern](../implemented/2026-06-13-capability-seams.md) prescribes. It generates TypeScript type definitions from tools for the model's context and runs the generated JavaScript in a sandbox, capturing console output alongside the return value. It normalizes model output into an async arrow function via AST parsing (acorn) and sanitizes tool names into valid JS identifiers (`my-tool` → `my_tool`, `delete` → `delete_`). It blocks outbound network by default. The transferable lessons — minimal executor contract, host-side type derivation, capture-output-and-return-value, name sanitization, AST-normalize the code, isolate by default — are folded into the design below. What does **not** transfer is the substrate: Cloudflare's isolation is Workers-specific; our equivalent hardened substrate is the deferred follow-up. +**Prior art.** `@cloudflare/codemode` validates this shape directly and several of its decisions are adopted below. Its `Executor` interface is deliberately tiny — `execute(code, fns) → { result, error?, logs? }` — with a production `DynamicWorkerExecutor` (isolated Workers) and a six-line `NodeVMExecutor` example as two implementations behind it: exactly the interface/implementation split [the capability-seam pattern](../../implemented/architecture/2026-06-13-capability-seams.md) prescribes. It generates TypeScript type definitions from tools for the model's context and runs the generated JavaScript in a sandbox, capturing console output alongside the return value. It normalizes model output into an async arrow function via AST parsing (acorn) and sanitizes tool names into valid JS identifiers (`my-tool` → `my_tool`, `delete` → `delete_`). It blocks outbound network by default. The transferable lessons — minimal executor contract, host-side type derivation, capture-output-and-return-value, name sanitization, AST-normalize the code, isolate by default — are folded into the design below. What does **not** transfer is the substrate: Cloudflare's isolation is Workers-specific; our equivalent hardened substrate is the deferred follow-up. **Prompt-budget tradeoff (Code Mode is not unconditionally cheaper).** Deriving the SDK types host-side costs no extra *discovery* round-trip, but the generated `.d.ts` is injected into the system prompt (§3a), so the type definitions themselves **do** consume context — and for an all-tools SDK that cost scales with every registered tool and can be comparable to, or larger than, the native JSON schemas it replaces. Code Mode's saving is on the **output/result** side (the model curates what comes back; intermediate results never re-enter context) and on **round-trips** (compose many calls in one program), not on the input-side tool description. The net win is workload-dependent: it pays off for multi-call, large-intermediate-result workflows and can cost *more* for a single call against a large tool surface. The `.d.ts` section is a prefix-stable prompt prefix, so prompt caching amortizes its per-turn cost across a session; the RFC notes that caching is what keeps the injected SDK affordable, and that a deployment with a very large tool surface should weigh the SDK size against native schemas rather than assume Code Mode is strictly cheaper. @@ -29,7 +29,7 @@ The design follows the codebase's capability-seam pattern ([capability seams](.. - a readonly `safe: boolean` on the `CodeRuntime` service — `false` for an unsandboxed stub, `true` only for a real isolating substrate; consumers gate on it (§2). - `SdkBinding = { namespace: string; fns: Record Promise> }` -Per the "explicit > implicit at seams" convention, the request spells out every field the runtime acts on; defaulting (e.g. an output cap, a timeout derived from `signal`) is the implementation's explicit job, not a hidden `?? default` inside `run()`. The split into interface + implementation is justified under [the capability-seam pattern](../implemented/2026-06-13-capability-seams.md) because there is **genuinely more than one planned implementation** — the node:vm stub *and* the hardened substrate (a real isolate, or the generated program run as a sandboxed process through the existing `ctx.bash` seam) that is scheduled follow-up work, not speculative optionality. The capability-seam pattern warns against splitting preemptively when only one implementation is conceivable; here a second is not just conceivable but required before any untrusted use, so the seam earns its keep. +Per the "explicit > implicit at seams" convention, the request spells out every field the runtime acts on; defaulting (e.g. an output cap, a timeout derived from `signal`) is the implementation's explicit job, not a hidden `?? default` inside `run()`. The split into interface + implementation is justified under [the capability-seam pattern](../../implemented/architecture/2026-06-13-capability-seams.md) because there is **genuinely more than one planned implementation** — the node:vm stub *and* the hardened substrate (a real isolate, or the generated program run as a sandboxed process through the existing `ctx.bash` seam) that is scheduled follow-up work, not speculative optionality. The capability-seam pattern warns against splitting preemptively when only one implementation is conceivable; here a second is not just conceivable but required before any untrusted use, so the seam earns its keep. **Backends can differ by language/runtime, not only by trust level.** The two implementations above (unsafe stub vs. hardened substrate) differ along the *trust* axis while staying TypeScript/JS, but nothing in the `CodeRuntime` contract — a program string plus a set of named async SDK bindings in, and a `{ result, logs, error? }` out — is bound to one source language. The same seam can host backends that differ along the *language* axis, executing a program written in something other than TypeScript. Two illustrative directions: @@ -38,7 +38,7 @@ Per the "explicit > implicit at seams" convention, the request spells out every These are illustrations of the seam's reach, **not commitments** — the MVP ships only the TypeScript path. The honest caveat is that the *execution* contract is language-agnostic but the *presentation* is not: the SDK-generation pipeline below (§3a and the `jsonSchemaToTs` codegen, which emits a TypeScript `.d.ts`) is TypeScript-specific, so a non-TS backend pairs the shared `CodeRuntime` contract with its own language-appropriate SDK generator and system-prompt section (a `.pyi` stub and Python usage instructions for the Python backend, AssemblyScript-flavored types for that one). The runtime seam is reused as-is; only the codegen/prompt half is per-language. -**2. Implementation package `packages/code-runtime-vm/`** — a new package `@deepseek-ai/dsh-code-runtime-vm`, the `node:vm` reference stub. It type-erases the model's TypeScript via the compiler's `transpileModule` (or sucrase) — the types exist only to guide the model; the runtime is plain JS — then wraps the body in an async IIFE for top-level `await` (Cloudflare's `NodeVMExecutor` does literally `new AsyncFunction("codemode", "return await (${code})()")`), runs it in a `vm.Context` whose globals are a capturing `console` and the SDK namespace objects, awaits the IIFE, and captures the return value, the buffered logs, and any thrown error (as `error: string`). It applies an **output cap** (truncate captured logs) and a **timeout tied to `request.signal`**. These caps limit blast radius; **they are not a security boundary**. node:vm is **not** isolation: withholding `require`/`process` does not contain anything (code escapes via `constructor`/prototype reflection), and per [AGENTS.md](../../../AGENTS.md) the harness must never hand model output the ambient environment. +**2. Implementation package `packages/code-runtime-vm/`** — a new package `@deepseek-ai/dsh-code-runtime-vm`, the `node:vm` reference stub. It type-erases the model's TypeScript via the compiler's `transpileModule` (or sucrase) — the types exist only to guide the model; the runtime is plain JS — then wraps the body in an async IIFE for top-level `await` (Cloudflare's `NodeVMExecutor` does literally `new AsyncFunction("codemode", "return await (${code})()")`), runs it in a `vm.Context` whose globals are a capturing `console` and the SDK namespace objects, awaits the IIFE, and captures the return value, the buffered logs, and any thrown error (as `error: string`). It applies an **output cap** (truncate captured logs) and a **timeout tied to `request.signal`**. These caps limit blast radius; **they are not a security boundary**. node:vm is **not** isolation: withholding `require`/`process` does not contain anything (code escapes via `constructor`/prototype reflection), and per [AGENTS.md](../../../../AGENTS.md) the harness must never hand model output the ambient environment. **The unsafe-runtime guard is enforceable, not a README warning.** Because a README caveat is not a control — and AGENTS.md's "never hand model output ambient authority" is a hard rule, not advice — the design makes the danger refuse to run by construction. Two layers: @@ -63,11 +63,11 @@ These are illustrations of the seam's reach, **not commitments** — the MVP shi **Sub-call CallIds.** Real tool calls dispatched from inside `run_code` need ids, but `CallId` is normally provider-issued (a branded string for correlating a call with its result — only brand-wrapped via `CallId()`, with no generator and no documented session-global-uniqueness guarantee). The plugin mints deterministic sub-ids scoped to the parent: `` `${exec.callId}:code:${n}` `` with a per-run counter `n`. These are unique within one `run_code` run (assuming the parent `callId` is unique, which the provider guarantees per turn); the `code/dispatch` event additionally carries the session log's `seq` so the UI and persistence can order and disambiguate globally without relying on the id alone. `ToolExecution.agent` is optional; the normal loop always supplies it (and with it `exec.agent.session`, the log `code/dispatch` appends to). A `run_code` execution arriving without `exec.agent` still runs (sub-calls propagate `agent: undefined`, exactly as the loop's own contract allows) but **skips session-log observability** — with no session to append to, those direct runs are simply not logged. -**Observability without context cost.** Each sub-dispatch emits a session event **declared by the `dsh-code-mode` plugin itself** via `SessionEventMap` declaration merging (the map is merge-extensible precisely so plugins can add events without touching `dsh-session`). Shape: `code/dispatch` with `{ parentCallId, subCallId, name, arguments (or redacted), isError, summary }`, ordered by the session log's own `seq`. `deriveMessages()` does **not** translate it into a model message — an unknown event type falls through its `default`, per the merge-extensible-union convention — so the UI and persistence ([session persistence](../implemented/2026-06-14-session-persistence.md)) can render every sub-call while the model's context only ever receives the single `run_code` tool-result. Because the event lives in the plugin, this adds no core change. +**Observability without context cost.** Each sub-dispatch emits a session event **declared by the `dsh-code-mode` plugin itself** via `SessionEventMap` declaration merging (the map is merge-extensible precisely so plugins can add events without touching `dsh-session`). Shape: `code/dispatch` with `{ parentCallId, subCallId, name, arguments (or redacted), isError, summary }`, ordered by the session log's own `seq`. `deriveMessages()` does **not** translate it into a model message — an unknown event type falls through its `default`, per the merge-extensible-union convention — so the UI and persistence ([session persistence](../../implemented/architecture/2026-06-14-session-persistence.md)) can render every sub-call while the model's context only ever receives the single `run_code` tool-result. Because the event lives in the plugin, this adds no core change. **SDK codegen.** A pure `jsonSchemaToTs(schema)` in `code-mode` maps the JSON-schema subset the `defineTool` DSL produces (object/string/number/boolean/array, `properties`, `required[]`, `enum` → string-literal union, nested objects, array `items`) to a TS type literal. It is **total**: any unsupported construct (`$ref`, `oneOf`/`anyOf`, `integer`, `null`, `additionalProperties`, or any raw MCP shape it does not recognize) degrades to `unknown` without throwing — it never crashes codegen. Typing is best-effort, not a guarantee, because MCP tools accept arbitrary JSON Schema and `ToolSchema.parameters` is typed only as `Record`. Because `ToolSchema.name` is an arbitrary string (not necessarily a valid TS identifier), the SDK is generated as a **namespace with quoted access** (e.g. `tools["some-mcp-tool"](args)`) plus safe camelCase aliases where the name is a clean identifier; alias collisions and TS reserved words fall back to quoted-only access (no duplicate alias emitted). This mirrors Cloudflare's `sanitizeToolName`. `run_code` itself is filtered out of the SDK. The MVP surfaces text content only; image and other block types in sub-results are deferred (noted as a limitation). -**Concurrency — serialized by default (the binding must enforce it).** The SDK functions are async, so a model writing `await Promise.all([tools.a(...), tools.b(...)])` would *start both* immediately, and each would call `ctx.tools.execute` right away — i.e. the binding shape makes concurrent dispatch the **default**, not an opt-in. Because the tool contract carries **no concurrency-safety metadata today** (parallel tool execution and a concurrency-safety hint are an open TODO in both `dsh-tools` and [docs/architecture.md](../../architecture.md): "phase 1 executes tool calls sequentially"), concurrent dispatch through a not-yet-hardened tool may race. So a prose "may serialize" is not sufficient. **Decision: the MVP SDK bindings enforce serialization** — each `run_code` invocation owns a per-run dispatch queue, and every `invoke()` chains onto it (`tail = tail.then(() => ctx.tools.execute(...))`), so even `Promise.all` over SDK calls executes them one at a time in submission order. This is a hard acceptance criterion, with a test that issues `Promise.all([...])` from a program and asserts the underlying `ctx.tools.execute` calls did **not** overlap (e.g. a probe tool records enter/exit and the test asserts no interleaving). The `.d.ts` may *describe* the model-visible functions as async (they are), but the implementation guarantees serial execution. Lifting serialization is deferred: only once a tool can declare itself read-only / concurrency-safe does the binding allow those specific tools to overlap. The same per-run queue is where the before/after abort checks (§3c) live, so an aborted run drains no further queued dispatches. +**Concurrency — serialized by default (the binding must enforce it).** The SDK functions are async, so a model writing `await Promise.all([tools.a(...), tools.b(...)])` would *start both* immediately, and each would call `ctx.tools.execute` right away — i.e. the binding shape makes concurrent dispatch the **default**, not an opt-in. Because the tool contract carries **no concurrency-safety metadata today** (parallel tool execution and a concurrency-safety hint are an open TODO in both `dsh-tools` and [docs/architecture.md](../../../architecture.md): "phase 1 executes tool calls sequentially"), concurrent dispatch through a not-yet-hardened tool may race. So a prose "may serialize" is not sufficient. **Decision: the MVP SDK bindings enforce serialization** — each `run_code` invocation owns a per-run dispatch queue, and every `invoke()` chains onto it (`tail = tail.then(() => ctx.tools.execute(...))`), so even `Promise.all` over SDK calls executes them one at a time in submission order. This is a hard acceptance criterion, with a test that issues `Promise.all([...])` from a program and asserts the underlying `ctx.tools.execute` calls did **not** overlap (e.g. a probe tool records enter/exit and the test asserts no interleaving). The `.d.ts` may *describe* the model-visible functions as async (they are), but the implementation guarantees serial execution. Lifting serialization is deferred: only once a tool can declare itself read-only / concurrency-safe does the binding allow those specific tools to overlap. The same per-run queue is where the before/after abort checks (§3c) live, so an aborted run drains no further queued dispatches. **Tool visibility tiers (design intentionally skipped).** A natural extension is to mark each tool with a *visibility tier*: some tools "direct-call eligible" (still offered as native wire tools alongside `run_code`), some "code-mode only" (reachable solely from within a `run_code` program, never on the wire), and the default "both." This would let a deployment keep a few high-frequency or approval-gated tools as direct calls while routing the long tail through Code Mode, or hide composition-only primitives from the native surface entirely. This RFC notes the possibility but **intentionally skips the detailed design** — the per-tool metadata, how it interacts with the `agent/request` enforcement in 3b, and the presentation split in 3a are left to a follow-up. The MVP is the simple two-state model: Code Mode on (everything via `run_code`) or off (everything native). @@ -75,7 +75,7 @@ These are illustrations of the seam's reach, **not commitments** — the MVP shi ## Alternatives -**Result elision / summarization over native tool-calling (the narrower route).** The Problem has two halves — context bloat (every intermediate `tool-result` re-enters context) and serial composition (one tool call per round-trip). The context-bloat half can be addressed *without* any code-execution runtime: keep provider tool-calling exactly as it is, and add a plugin on the `agent/request` waterfall (or a compaction pass akin to [the session-persistence work](../implemented/2026-06-14-session-persistence.md)) that elides or summarizes older `tool-result` blocks before they re-enter the model's context — drop them past a window, replace large payloads with a digest, or keep only the blocks the model still references. This is strictly less invasive than Code Mode: no new runtime seam, no model-written programs, no new safety surface. It is the right tool if context growth is the only pain. +**Result elision / summarization over native tool-calling (the narrower route).** The Problem has two halves — context bloat (every intermediate `tool-result` re-enters context) and serial composition (one tool call per round-trip). The context-bloat half can be addressed *without* any code-execution runtime: keep provider tool-calling exactly as it is, and add a plugin on the `agent/request` waterfall (or a compaction pass akin to [the session-persistence work](../../implemented/architecture/2026-06-14-session-persistence.md)) that elides or summarizes older `tool-result` blocks before they re-enter the model's context — drop them past a window, replace large payloads with a digest, or keep only the blocks the model still references. This is strictly less invasive than Code Mode: no new runtime seam, no model-written programs, no new safety surface. It is the right tool if context growth is the only pain. It is insufficient for the **composition / round-trip** half, which is the decisive reason this RFC does not stop there. Elision still pays one model round-trip per tool call: a loop over N items is N turns, a branch on an intermediate value is a turn to fetch then a turn to act, and post-processing (filter, join, reduce) either happens in the model's head over full payloads or not at all. Code Mode collapses all of that into one program — the loop, the branch, the join run in the runtime, and only the curated result returns. Elision also cannot express fan-out or data-dependent control flow; it only shrinks what comes back. So the two are complementary, not competing: elision could even layer *under* Code Mode for the residual native-tool paths. The RFC chooses Code Mode because the round-trip/composition cost is the larger structural limit, and accepts the new code-execution surface as the price — which is exactly why the execution substrate is gated behind the enforceable safety guard (§2) and the hardened backend is a hard prerequisite for untrusted use. @@ -83,12 +83,12 @@ It is insufficient for the **composition / round-trip** half, which is the decis ## Plan -1. Scaffold the interface package `packages/code-runtime/` per [the cookbook](../../cookbook/adding-a-package.md): abstract `CodeRuntime extends Service` (`super(ctx, 'codeRuntime')`) with a readonly `safe: boolean`, the `declare module 'cordis'` ctx key, the `CodeRunRequest`/`CodeRunResult`/`SdkBinding` vocabulary, method contracts documented in JSDoc (what `run` captures, abort semantics, that an error is a result field not a throw, what `safe` means). HMR-safety test (dispose the contributing fiber, assert `ctx.codeRuntime` is gone). +1. Scaffold the interface package `packages/code-runtime/` per [the cookbook](../../../cookbook/adding-a-package.md): abstract `CodeRuntime extends Service` (`super(ctx, 'codeRuntime')`) with a readonly `safe: boolean`, the `declare module 'cordis'` ctx key, the `CodeRunRequest`/`CodeRunResult`/`SdkBinding` vocabulary, method contracts documented in JSDoc (what `run` captures, abort semantics, that an error is a result field not a throw, what `safe` means). HMR-safety test (dispose the contributing fiber, assert `ctx.codeRuntime` is gone). 2. Scaffold the implementation package `packages/code-runtime-vm/`: the node:vm stub — `safe = false`, a constructor that **throws unless given `{ unsafe: true }`**, transpile/type-erase, async-IIFE wrap, capturing `console`, SDK globals, return-value/logs/error capture, output cap, signal-tied timeout. Tests for output capture, return value, error-as-field, abort, the constructor refusal without `unsafe`, and a README documenting the "not a sandbox, trusted-only" caveat prominently. 3. Scaffold the consumer plugin `packages/code-mode/`: `jsonSchemaToTs` codegen with namespace/quoted-access + alias handling (unit tests, including non-identifier MCP names and unsupported-shape → `unknown`); the registered lazy `ctx.systemPrompt.section()` carrying the SDK `.d.ts`; the `agent/request` listener (`prepend: true`) collapsing `request.tools` to `[run_code]` after `await next()`; the **unsafe-runtime gate** (refuse to register `run_code` when `ctx.codeRuntime.safe === false` unless `allowUnsafeRuntime` is set); the `run_code` tool with the dispatch bridge (per-run serialization queue, deterministic sub-call ids, before/after abort checks, `CodeRunError` on error results); and the `code/dispatch` event declared here via `SessionEventMap` merge. Declare `inject = ['tools', 'systemPrompt', 'codeRuntime']`. 4. Tests: HMR-safety (dispose removes the tool, the section, and the listener); a waterfall test that the wire tool list is exactly `[run_code]` (spy adapter, asserting via `agent/request` and optionally `llm/stream`); an integration test that a program calling two tools returns only its printed/returned output (verify the world, not the self-report); a **serialization test** that `Promise.all([...])` over SDK calls does not overlap the underlying `ctx.tools.execute` invocations (a probe tool records enter/exit; assert no interleaving); `deriveMessages()` ignores `code/dispatch`; abort mid-program stops further dispatches; `CodeRunError` surfaces as `isError: true`; and the **unsafe-runtime refusal test** (§3, the VM-guard): with the unsafe flag unset, a non-mock agent's `run_code` is refused; with it set, the program runs. 5. Wire an example: `examples/coding-agent-code-mode` (or a config flag on the existing example) loading the trio. Running it against the node:vm stub requires both opt-ins (`VmCodeRuntime({ unsafe: true })` and `code-mode`'s `allowUnsafeRuntime`); the example sets them explicitly and comments why, or uses a mock model — a real model never reaches the unsandboxed stub without those deliberate flags. Add a `pnpm run demo:*` entry. -6. Docs: update [docs/architecture.md](../../architecture.md) (a `ctx.codeRuntime` row in the service map, a Code Mode note under the tool pipeline / capability seams sections); add a [cookbook](../../cookbook/) note on writing a `CodeRuntime` backend; and **file the follow-up RFC for the hardened execution substrate** (the isolate/sandboxed-process design, the additional-language backends sketched in §1 — AssemblyScript/WASM, Python — with their per-language SDK generators, plus the tool-visibility-tier design skipped here). On landing, move this file to `implemented/` and update its row in [the RFC index](../README.md). +6. Docs: update [docs/architecture.md](../../../architecture.md) (a `ctx.codeRuntime` row in the service map, a Code Mode note under the tool pipeline / capability seams sections); add a [cookbook](../../../cookbook) note on writing a `CodeRuntime` backend; and **file the follow-up RFC for the hardened execution substrate** (the isolate/sandboxed-process design, the additional-language backends sketched in §1 — AssemblyScript/WASM, Python — with their per-language SDK generators, plus the tool-visibility-tier design skipped here). On landing, move this file to `implemented/` and update its row in [the RFC index](../../README.md). ## Risks diff --git a/docs/rfc/proposed/2026-06-11-api-extractor-reports.md b/docs/rfc/proposed/process/2026-06-11-api-extractor-reports.md similarity index 86% rename from docs/rfc/proposed/2026-06-11-api-extractor-reports.md rename to docs/rfc/proposed/process/2026-06-11-api-extractor-reports.md index db004eb8b9..c1099a6cce 100644 --- a/docs/rfc/proposed/2026-06-11-api-extractor-reports.md +++ b/docs/rfc/proposed/process/2026-06-11-api-extractor-reports.md @@ -4,7 +4,7 @@ Status: proposed -> Split out from the original "Doc-sync and API reports" RFC (2026-06-11). Parts 1-2 (doc-block typechecking, event-taxonomy verification) shipped — see [doc-sync enforcement](../implemented/2026-06-11-doc-sync-enforcement.md). This is the deferred part 3, kept as a standalone proposal. +> Split out from the original "Doc-sync and API reports" RFC (2026-06-11). Parts 1-2 (doc-block typechecking, event-taxonomy verification) shipped — see [doc-sync enforcement](../../implemented/process/2026-06-11-doc-sync-enforcement.md). This is the deferred part 3, kept as a standalone proposal. ## Problem diff --git a/docs/rfc/proposed/2026-06-11-architectural-conformance.md b/docs/rfc/proposed/process/2026-06-11-architectural-conformance.md similarity index 83% rename from docs/rfc/proposed/2026-06-11-architectural-conformance.md rename to docs/rfc/proposed/process/2026-06-11-architectural-conformance.md index 1268e6dc49..6eaf8b03a7 100644 --- a/docs/rfc/proposed/2026-06-11-architectural-conformance.md +++ b/docs/rfc/proposed/process/2026-06-11-architectural-conformance.md @@ -6,7 +6,7 @@ Status: proposed ## Problem -Two architectural guarantees currently live only in prose: (1) nothing depends on the concrete loop package ([the microkernel promise](../implemented/2026-06-11-microkernel-event-taxonomy.md)), and (2) every LlmAdapter speaks the chunk protocol correctly. Both should be mechanical ([the quality-gates principle](../implemented/2026-06-11-quality-gates.md)). +Two architectural guarantees currently live only in prose: (1) nothing depends on the concrete loop package ([the microkernel promise](../../implemented/architecture/2026-06-11-microkernel-event-taxonomy.md)), and (2) every LlmAdapter speaks the chunk protocol correctly. Both should be mechanical ([the quality-gates principle](../../implemented/process/2026-06-11-quality-gates.md)). ## Proposal @@ -18,7 +18,7 @@ Two architectural guarantees currently live only in prose: (1) nothing depends o - `vendor/*` must not import from `packages/*`. - Layering: dsh-llm imports nothing from other dsh packages; dsh-session only dsh-llm; etc. (the dependency table in packages/README.md, enforced). -**Adapter conformance kit** in dsh-llm (`@deepseek-ai/dsh-llm/conformance`): a reusable vitest suite parameterized by an adapter factory, asserting the chunk-protocol contract — index monotonicity per block, no deltas after `block-end` for an index, exactly one `finish`, usage at most once, every `tool-call-delta` carries the call id, abort honored promptly. Run it against the mocks now; the DeepSeek V4 adapter inherits it on day one. Optionally a dev-mode `strictAdapter()` wrapper enforcing the same at runtime behind a debug flag (pairs with [the dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md)). +**Adapter conformance kit** in dsh-llm (`@deepseek-ai/dsh-llm/conformance`): a reusable vitest suite parameterized by an adapter factory, asserting the chunk-protocol contract — index monotonicity per block, no deltas after `block-end` for an index, exactly one `finish`, usage at most once, every `tool-call-delta` carries the call id, abort honored promptly. Run it against the mocks now; the DeepSeek V4 adapter inherits it on day one. Optionally a dev-mode `strictAdapter()` wrapper enforcing the same at runtime behind a debug flag (pairs with [the dev-mode invariants](../../implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md)). ## Plan diff --git a/docs/rfc/proposed/2026-06-11-supply-chain-and-vendor-drift.md b/docs/rfc/proposed/process/2026-06-11-supply-chain-and-vendor-drift.md similarity index 79% rename from docs/rfc/proposed/2026-06-11-supply-chain-and-vendor-drift.md rename to docs/rfc/proposed/process/2026-06-11-supply-chain-and-vendor-drift.md index c187c85355..e35e79f0a5 100644 --- a/docs/rfc/proposed/2026-06-11-supply-chain-and-vendor-drift.md +++ b/docs/rfc/proposed/process/2026-06-11-supply-chain-and-vendor-drift.md @@ -6,7 +6,7 @@ Status: proposed ## Problem -The vendor manifest ([the vendoring decision](../implemented/2026-06-11-vendor-cordis-as-source.md)) is enforced at commit time in the *forward* direction (vendored change ⇒ manifest update) but nothing verifies the manifest's *claims*: that vendor/ actually equals upstream-at-SHA plus exactly the logged modifications. And the handful of true npm dependencies have no advisory monitoring or update cadence. +The vendor manifest ([the vendoring decision](../../implemented/process/2026-06-11-vendor-cordis-as-source.md)) is enforced at commit time in the *forward* direction (vendored change ⇒ manifest update) but nothing verifies the manifest's *claims*: that vendor/ actually equals upstream-at-SHA plus exactly the logged modifications. And the handful of true npm dependencies have no advisory monitoring or update cadence. ## Proposal diff --git a/docs/rfc/proposed/process/2026-06-20-discover-package-inventory.md b/docs/rfc/proposed/process/2026-06-20-discover-package-inventory.md new file mode 100644 index 0000000000..784a36c593 --- /dev/null +++ b/docs/rfc/proposed/process/2026-06-20-discover-package-inventory.md @@ -0,0 +1,28 @@ +# RFC: Discover package inventories instead of maintaining static lists + +Status: proposed + +## Problem + +Package and gate inventories are repeated by hand. The [package cookbook](../../../cookbook/adding-a-package.md) tells authors to update several files. The [package README](../../../../packages/README.md) carries a hand-written dependency graph. [CI](../../../../.github/workflows/ci.yml) and [development docs](../../../development.md) can drift from the actual `doc-sync` subcommands when new gates are added. `tsconfig.build.json` lists all 18 packages as explicit project `references`. These lists are small today, but every new package or gate creates another manual synchronization point. + +The [package hierarchy](../../implemented/architecture/2026-06-20-package-hierarchy.md) already removed several of these by hand: `scripts/publint-all.ts` now derives its list from the `packages//` layout, and the two `tsconfig` `paths` maps collapsed to one `@deepseek-ai/dsh-*` wildcard. What remains is the inventory that cannot be globbed away — chiefly `tsconfig.build.json`'s project `references`, which TypeScript requires as an explicit array (no wildcard form). + +Static lists are appropriate when they encode policy; they are needless friction when they duplicate manifest data or layout facts that already exist in `package.json`, workspace globs, or the package hierarchy. + +## Proposal + +Make the remaining package/gate inventories discoverable. A single canonical source — the `packages//` hierarchy plus package manifests — should drive `tsconfig.build.json`'s `references`, the module graph, and any other full-package list, with a generate-and-verify step (the existing `gen-module-graph` / `gen-cordis-catalog` pattern: a generator writes the artifact, a `--check` mode in `hygiene`/`doc-sync` fails on a stale committed copy). Module graph generation already reads package manifests. `doc-sync` should be the one command that defines and prints its sub-gates, with docs linking to that command rather than restating a second list. + +The hierarchy does not need to encode every fact about a package, but it should encode the broad maintenance policy: core/product packages, integrations, capability seams, and support/test/example packages should not all require a hand-maintained exception list before scripts can tell them apart. + +## Acceptance criteria + +- `tsconfig.build.json` project `references` are generated from the hierarchy (a generator emits them; a `--check` gate fails when the committed copy is stale), rather than hand-maintained. +- Adding a package does not require editing a static package list for any gate. +- Docs describe the source of truth rather than repeating generated inventories. +- CI invokes the aggregate commands and lets those commands own their sub-gate lists. + +## What we give up + +Discovery scripts can become too clever. The implementation should stay boring: read manifests, filter on explicit fields, print the resolved list, and fail loud. The payoff is removing manual inventory drift, not inventing a build system. diff --git a/docs/rfc/proposed/simplification/2026-06-20-remove-agent-boundary-mirror-events.md b/docs/rfc/proposed/simplification/2026-06-20-remove-agent-boundary-mirror-events.md new file mode 100644 index 0000000000..4b1cd75a56 --- /dev/null +++ b/docs/rfc/proposed/simplification/2026-06-20-remove-agent-boundary-mirror-events.md @@ -0,0 +1,31 @@ +# RFC: Stop mirroring durable boundaries as agent events + +Status: proposed + +## Problem + +The loop records the canonical transcript in `SessionEvent` and also emits a parallel set of live `agent/*` mirror events: `agent/turn-start`, `agent/turn-end`, `agent/step-start`, `agent/step-end`, `agent/stream-chunk`, and `agent/steering`. The mirrors make consumers choose between two sources of truth. ACP already chose the session log for the editor-facing transcript because a throwing peer listener can prevent later `agent/*` listeners from observing a boundary, while the session event was already appended. The stdio UI is the only production consumer that still renders turn boundaries and the token stream from the mirror events; it already renders tool calls and results from `session/event`. + +This duplication is not free. Every lifecycle change has to update the session event, the mirror event, docs, invariants, tests, and snapshot expectations. The duplicate boundary events also make failure ordering subtle: a turn can be durably closed before a live `agent/turn-end` listener runs, so a post-boundary listener failure has no valid in-log position left and must be reported out of band. + +## Proposal + +Make `session/event` the live transcript stream. Consumers that render turns, tool calls, tool results, assistant messages, and durable boundaries subscribe to `session/event` and derive their UI from the same event vocabulary persistence uses. Keep agent lifecycle/control events that are not transcript data: `agent/created`, `agent/disposed`, `agent/status`, `agent/error`, and `agent/queued`. `agent/queued` is an inbox acknowledgement rather than a transcript mirror: it fires before any durable event exists, and cancelled queued work may never enter the log. + +Remove the duplicate durable-boundary mirrors from the agent event taxonomy. If a UI wants an agent handle from a session event, it can keep a small map from session id to agent built from `agent/created`/`agent/disposed`, or the registry can offer an explicit lookup. The canonical record remains the event-sourced session log. + +## Acceptance criteria + +- ACP and stdio render transcript content from `session/event`. +- `agent/turn-start`, `agent/turn-end`, `agent/step-start`, `agent/step-end`, and `agent/steering` are removed or reduced to private implementation details. +- `agent/queued` is either retained and documented as live-only inbox/control state, or deleted in a separate proposal that names the queue-acknowledgement capability loss. +- Tests assert the persisted event stream, not a second mirror stream, for turn and step ordering. +- Documentation presents `SessionEvent` as both the durable source and the live transcript feed. + +## What we give up + +A plugin can no longer observe turn/step boundaries from a convenient `Agent`-first event. It must either subscribe to `session/event` or maintain a session-to-agent association. That is an acceptable trade: transcript consumers should not depend on a second event feed that can drift from the durable log. + +## Related + +Because high-fidelity `assistant/chunk` persistence remains load-bearing, `agent/stream-chunk` can be evaluated as another mirror of durable session data rather than as the only token stream. If a future proposal moves chunks out of the canonical log, `agent/stream-chunk` would need a fresh decision as a deliberately live-only UI signal. diff --git a/docs/rfc/proposed/simplification/2026-06-20-unify-agent-and-session-id.md b/docs/rfc/proposed/simplification/2026-06-20-unify-agent-and-session-id.md new file mode 100644 index 0000000000..19bde62d01 --- /dev/null +++ b/docs/rfc/proposed/simplification/2026-06-20-unify-agent-and-session-id.md @@ -0,0 +1,57 @@ +# RFC: Unify the agent id and the session id + +Status: proposed + +## Problem + +The agent factory carries TWO ids for what is, in every live consumer, one thing: + +- `agentId` — the `AgentRegistry` handle (the actor identity; the registry rejects a duplicate). +- `sessionId` — the event-sourced session / persisted-log identity (`session.header.id`). + +`CreateAgentOptions` takes both separately; `ResumeAgentOptions` takes an `agentId` plus a `resumeSessionId`. They diverge in exactly two places: + +- **Config-driven create** (`AgentLoop.create`): a stable `agentId` (e.g. `"echo"`) with a fresh per-run `sessionId` (`${id}-session-`). +- **Resume**: a caller-supplied `agentId` (e.g. `"main"`) on a persisted `resumeSessionId`. + +Everywhere a live consumer actually looks an agent up — the **ACP bridge, the only production path** — the two are already unified: `agentId === sessionId === `. + +The separation is **latent generality no consumer exercises**: nothing reads a *stable* `agentId` back across runs (each process starts fresh, and persistence keys off the session id, never the agent id). The config path's "stable agentId, fresh sessionId" buys nothing concrete — it is cosmetic. And the `agentId !== sessionId` case is precisely what opens the bash owner-token alias hole: the bash completion-notice routes by `session.header.id`, but the registry enforces uniqueness only on `agentId`, so a programmatic caller registering two agents with different agent ids but the SAME session id can mis-route a notice (see [agent lifecycle and ownership seams](../../implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md) § Seam precondition). The current code documents this as a precondition rather than guaranteeing it. + +## Proposal + +Make an agent BE its session: one id. An agent's registry handle IS its `session.header.id`. + +- `CreateAgentOptions` drops the separate `sessionId` — the single `id` is both the registry handle and the live/persisted session id. (ACP already passes the same UUID for both, so its call site simplifies to one field.) +- `ResumeAgentOptions` drops the separate `agentId` — resuming `sessionId` X registers the agent under id X. (ACP already does this.) +- The config path (`AgentLoop.create`) uses its configured `id` directly as the session id, applying whatever resume-or-create policy it adopts (today it appends a per-run uuid to avoid colliding with an on-disk log; that policy moves onto the single id, e.g. the config id IS the session and a durable backend resumes it — to be settled in the implementing PR). +- The registry's existing unique-`agentId` check becomes, by construction, a unique-session-id guarantee — the bash alias hole is closed with NO new defensive invariant: two agents cannot share a session id because the session id is the agent id. + +## Why not just enforce session-id uniqueness in `AgentRegistry.register()`? + +That was the review's first suggestion. It would couple the generic registry to a session-uniqueness assumption (the registry tracks *agents*, not sessions) and entrench the very separation this RFC removes. Unifying the ids closes the hole more cleanly — there is nothing left to enforce. + +## Acceptance criteria + +- `ctx.agents.create`/`resume` take a single id; the ACP bridge passes one id. +- The config-driven agent path has a deliberate, documented session-id policy (no silent per-run id divergence that no consumer reads). +- The bash owner-token alias hole is gone by construction (no two live agents can share a session id). +- All existing behavior the tests pin (ACP create/resume/load, config startup, durability) still holds — or the tests change WITH the behavior where the divergence was an artifact (per AGENTS.md "tests document behavior, not golden truth"). + +## Risks + +This touches public factory interfaces (`CreateAgentOptions`, `ResumeAgentOptions`, `AgentFactory`) and the config-agent id scheme, so it is a deliberate cross-package change, not a local patch — it ships as its own PR (converged with Codex), stacked on the bash owner-token work that surfaced the precondition. + +The genuine risks of collapsing the two ids into one (the case AGAINST this proposal — to be weighed honestly before implementing): + +- **It forecloses a one-agent-resumes-many-sessions / one-session-driven-by-many-agents future.** Today the separate ids leave room for an agent (a stable actor) to detach from one session and attach to another, or for a handoff where a new agent process adopts an existing session under a new actor handle. Unifying makes "agent" and "session" the same lifetime, so any such future needs a NEW seam (e.g. an explicit `actorId` distinct from the session) — re-introducing the very separation we removed. We judge this generality currently unused, but it is a door this change closes. + +- **Sub-agents / fork / spawn (an explicitly deferred seam) may WANT a stable actor id across forked sessions.** `AgentLoop.create`'s `TODO(sub-agents)` envisions a child agent seeded from a parent's event log. If the design wants "the same agent identity across a fork" (parent and child share an actor but have distinct session logs), a unified id blocks it. The implementing PR must check the intended fork/spawn model BEFORE unifying, or accept that fork always mints a fresh combined id. + +- **The config-driven resume-or-create policy becomes load-bearing, not cosmetic.** Today the per-run-uuid session id quietly sidesteps the "a fixed id collides with its own on-disk log on the second run" problem. Once the id is unified and stable, a config agent restarting MUST decide resume-vs-fresh deliberately — there is no longer a throwaway session id to hide behind. Getting this wrong reintroduces the create-collision the uuid was avoiding (a durable backend refuses to re-create an id whose log exists). This is the one real design decision the implementing PR owns, and it is easy to get subtly wrong. + +- **Persisted/on-disk identity becomes the agent identity.** Unifying means the registry handle is now a persisted, externally-meaningful string (a session id a client chose), not an internal label. A caller that previously used a short human label (`"main"`) as the agent id now must use the session id. This is fine for ACP (already a UUID) but is a semantic narrowing for any programmatic embedder that relied on naming its agents independently of session storage. + +- **Migration churn touches every create/resume call site and its tests.** `CreateAgentOptions`/`ResumeAgentOptions` shape changes ripple to ACP, the config path, the agent-loop factory, and ~dozens of test fixtures that currently pass distinct `agentId`/`sessionId` (some deliberately distinct to exercise the divergence — those tests change WITH the behavior, per AGENTS.md "tests document behavior, not golden truth"). The risk is mechanical but broad; a missed call site is a type error, but a missed *test* could silently lose coverage of a path. + +The one real design question the implementing PR must settle first is the config-driven resume-or-create policy once the id is unified (today's per-run-uuid behavior is a demo simplification already flagged `TODO(demo)`). If, on closer look, the fork/spawn or multi-session-actor futures turn out to be wanted, this RFC should be REJECTED in favor of the lighter "enforce session-id uniqueness in the registry" guard — the alias hole is not reachable via ACP, so keeping the ids separate and merely documenting (or mechanically enforcing) the precondition remains a valid alternative. diff --git a/docs/rfc/proposed/2026-06-11-deterministic-and-stress-testing.md b/docs/rfc/proposed/testing/2026-06-11-deterministic-and-stress-testing.md similarity index 100% rename from docs/rfc/proposed/2026-06-11-deterministic-and-stress-testing.md rename to docs/rfc/proposed/testing/2026-06-11-deterministic-and-stress-testing.md diff --git a/docs/rfc/proposed/2026-06-11-mutation-testing.md b/docs/rfc/proposed/testing/2026-06-11-mutation-testing.md similarity index 79% rename from docs/rfc/proposed/2026-06-11-mutation-testing.md rename to docs/rfc/proposed/testing/2026-06-11-mutation-testing.md index 8a1e422f7b..209b0cdbd0 100644 --- a/docs/rfc/proposed/2026-06-11-mutation-testing.md +++ b/docs/rfc/proposed/testing/2026-06-11-mutation-testing.md @@ -6,7 +6,7 @@ Status: proposed ## Problem -The per-file 100% coverage gate ([the quality-gates decision](../implemented/2026-06-11-quality-gates.md)) proves every line *executes* under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite *kills* deliberately injected bugs. +The per-file 100% coverage gate ([the quality-gates decision](../../implemented/process/2026-06-11-quality-gates.md)) proves every line *executes* under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite *kills* deliberately injected bugs. ## Proposal diff --git a/docs/rfc/rejected/2026-06-11-immutable-public-surfaces.md b/docs/rfc/rejected/architecture/2026-06-11-immutable-public-surfaces.md similarity index 72% rename from docs/rfc/rejected/2026-06-11-immutable-public-surfaces.md rename to docs/rfc/rejected/architecture/2026-06-11-immutable-public-surfaces.md index 8433862e1f..0e791ef689 100644 --- a/docs/rfc/rejected/2026-06-11-immutable-public-surfaces.md +++ b/docs/rfc/rejected/architecture/2026-06-11-immutable-public-surfaces.md @@ -1,6 +1,6 @@ # RFC: Deep-readonly public surfaces -Status: rejected — the pervasive `DeepReadonly` type flip was rejected in favor of an always-on `deriveMessages` clone plus dev-mode `Object.freeze` + invariants. The immutability *goal* shipped via that alternative; see [dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md). +Status: rejected — the pervasive `DeepReadonly` type flip was rejected in favor of an always-on `deriveMessages` clone plus dev-mode `Object.freeze` + invariants. The immutability *goal* shipped via that alternative; see [dev-mode invariants](../../implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md). @@ -10,18 +10,18 @@ The session log is append-only by contract, but `session.events` returns `readon ## Proposal -> **Implemented differently — see the Status line and [dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md).** The `DeepReadonly` design below was rejected as written (compile-only, high type-noise, castable). What shipped: an always-on deep clone in `deriveMessages` (closing the request/adapter aliasing path) plus a dev-mode `Object.freeze` + invariants plugin. The proposal text is kept for the record. +> **Implemented differently — see the Status line and [dev-mode invariants](../../implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md).** The `DeepReadonly` design below was rejected as written (compile-only, high type-noise, castable). What shipped: an always-on deep clone in `deriveMessages` (closing the request/adapter aliasing path) plus a dev-mode `Object.freeze` + invariants plugin. The proposal text is kept for the record. Make immutability part of the type where mutation is corruption: - `SessionEvent` data becomes `DeepReadonly` on the way OUT of a session (`events`, `session/event` listeners); `append()` keeps taking plain mutable input. A `DeepReadonly` utility type lands in dsh-llm next to the brand/never helpers. - `deriveMessages()` returns deep-readonly messages; the loop clones before handing a mutable request to the `agent/request` waterfall (mutation there is sanctioned — the clone makes the boundary explicit and cheap, once per step). - `PromptAssembly` stays mutable through its waterfall (sanctioned) but the registry's internal section list is cloned per assembly (already true). -- Optionally, dev-mode `Object.freeze` of event data behind [the dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md) flag, so sanctioned-mutation violations throw in tests rather than corrupting silently. +- Optionally, dev-mode `Object.freeze` of event data behind [the dev-mode invariants](../../implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md) flag, so sanctioned-mutation violations throw in tests rather than corrupting silently. ## Plan -Introduce `DeepReadonly`, flip the session read paths, fix resulting compile errors in consumers (expected: a handful in tests), add the freeze-in-dev option alongside [the dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md) plugin. +Introduce `DeepReadonly`, flip the session read paths, fix resulting compile errors in consumers (expected: a handful in tests), add the freeze-in-dev option alongside [the dev-mode invariants](../../implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md) plugin. ## Risks diff --git a/docs/rfc/rejected/architecture/2026-06-20-providerless-example-base.md b/docs/rfc/rejected/architecture/2026-06-20-providerless-example-base.md new file mode 100644 index 0000000000..7856c04b81 --- /dev/null +++ b/docs/rfc/rejected/architecture/2026-06-20-providerless-example-base.md @@ -0,0 +1,27 @@ +# RFC: Make the shared example base providerless + +Status: rejected — superseded by [Extract example apps into packages](../../implemented/architecture/2026-06-20-extract-example-app-packages.md), which moves the spine into a `dsh-agent-core` bundle and deletes the `base*.yml` files, so there is no shared base YAML left to rename. + +## Problem + +The examples had two shared base files: `examples/base-core.yml` was providerless, while `examples/base.yml` included that core plus the real `llm-deepseek` adapter. Snapshot replay needs the providerless core with `llm-replay`, because loading the real adapter without a key throws. The normal demos need the real adapter. The result was a naming inversion: the file named `base.yml` was not the reusable base for all examples, while the true base was `base-core.yml`. + +The split was understandable, but it made every config explanation longer. It also led to awkward test setup like a keyless smoke test carrying a dummy API key so an adapter could boot even though the model is not called. + +## Proposal + +Rename the providerless core to `examples/base.yml` and make adapter selection explicit in each concrete example. The coding and ACP real configs add a tiny `llm-deepseek` include or local block; snapshot config adds `llm-replay`. Delete `examples/base-core.yml`. + +The shared base should contain only provider-neutral services and tools: `llm`, sessions, system prompt, tools, agents, invariants, bash executor, and bash tool schemas. Anything that chooses a model provider belongs at the leaf config. + +## Acceptance criteria + +- `examples/base.yml` is providerless. +- `examples/base-core.yml` is deleted. +- Real demo configs explicitly add the DeepSeek adapter. +- Snapshot replay config includes the same providerless base and its replay adapter. +- The [examples README](../../../../examples/README.md), example-specific READMEs, and RFC references stop explaining "base = base-core plus adapter". + +## What we give up + +Real demos lose one layer of convenience: each must opt into the adapter. That is the right default for examples, because adapter choice is the variable part and providerless wiring is the shared product core. diff --git a/docs/rfc/rejected/simplification/2026-06-20-assembled-assistant-messages-only.md b/docs/rfc/rejected/simplification/2026-06-20-assembled-assistant-messages-only.md new file mode 100644 index 0000000000..56f20c04ef --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-assembled-assistant-messages-only.md @@ -0,0 +1,32 @@ +# RFC: Persist assembled assistant messages, not stream chunks + +Status: rejected — high-fidelity chunk replay, partial failed streams, and snapshot replay currently depend on persisted `assistant/chunk` events. Dropping chunks is only viable with a no-information-loss replay/artifact replacement. + +## Problem + +The canonical session log currently persists every `assistant/chunk` exactly as streamed by the model. The [session persistence RFC](../../implemented/architecture/2026-06-14-session-persistence.md) chose this for token-level replay fidelity and contiguous `seq`, but the cost has grown: JSONL fixtures are dominated by tiny delta records, snapshot scenarios replay the model by grouping chunk events, ACP load reconstructs prior assistant output from chunks, and any future log reader must distinguish durable message history from token-level trace. + +For successful steps that assemble completed content, the loop already appends an `assistant/message`. That is the event `deriveMessages()` uses for the next model request. In other words, the normal resumable conversation state is already present without the chunks; chunks are a live rendering and deterministic-test artifact, not required conversation history. Failed or aborted streams are different: partial assistant output may exist only as chunks, and empty max-token steps may produce no `assistant/message` at all. + +## Proposal + +Stop storing `assistant/chunk` in the canonical session log. The durable log keeps `assistant/message`, `tool/call`, `tool/result`, `usage` if retained, and turn boundaries. Live UIs can still receive token deltas through a deliberately transient stream event. Snapshot replay should move its model script into an explicit fixture sidecar or derive it from a recorded adapter artifact, rather than treating the canonical user session as a token tape. Scenarios that need partial failed-stream output must record that output in the replay fixture. + +ACP `session/load` can replay prior assistant messages as complete content blocks instead of simulating the original token stream. A loaded transcript need not reproduce every historical delta; it must show the same completed assistant content and resume with a valid provider history. + +## Acceptance criteria + +- `SessionEventMap` drops `assistant/chunk`, or marks it as non-persisted if a transitional live event is needed. +- [Session persistence docs](../../../../packages/session-persistence/session-persistence/README.md) no longer require every stream chunk to be stored verbatim. +- `llm-replay` and ACP snapshots use an explicit replay fixture format or sidecar for model chunks. +- `session/load` renders completed assistant messages from `assistant/message`. +- Stored logs get much smaller and remain `seq`-contiguous without chunk holes. +- The session format version and recorded fixtures are refreshed; non-current stored logs are rejected per the pre-release format policy. + +## What we give up + +The canonical user session no longer reconstructs the exact token stream of an old turn. It also loses partial assistant output from failed or aborted streams unless another event or fixture records it. That is too much information loss for the current resume, load, and snapshot contracts. Tests that need exact deterministic streams should own that fixture directly only if the production session log keeps enough fidelity for user-visible recovery. + +## Related + +This supersedes the chunk-persistence choice in [session persistence](../../implemented/architecture/2026-06-14-session-persistence.md) and affects [ACP snapshot tests](../../implemented/testing/2026-06-19-acp-snapshot-tests.md), whose current replay plugin derives its script from `assistant/chunk` events. diff --git a/docs/rfc/rejected/simplification/2026-06-20-drop-acp-session-load.md b/docs/rfc/rejected/simplification/2026-06-20-drop-acp-session-load.md new file mode 100644 index 0000000000..f6cc3a3236 --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-drop-acp-session-load.md @@ -0,0 +1,25 @@ +# RFC: Drop ACP session/load until resume has a product shape + +Status: rejected — Zed is the current target ACP client, advertises and exercises load-capable sessions, and keeps pending-load state for concurrent `session/load`. The bridge should keep `session/load` and make the resume contract solid. + +## Problem + +ACP advertises `loadSession: true` and implements `session/load` by injecting persistence into the bridge, validating cwd against stored metadata, reconstructing an agent from the persisted log, and replaying prior transcript updates to the client. That path has its own race handling, loading-id guard, replay presenter logic, and tests. It also depends on the canonical log retaining enough UI data to reconstruct old chunks and tool presentations. + +Durable persistence remains foundational, but editor-visible resume is not yet a designed product flow. There is no session picker, no title/preview metadata, and no clear UX for failed or partial loads. The bridge is paying complexity for a feature that is exercised by tests, documentation, and the current target client's session model. + +## Proposal + +For now, ACP starts fresh sessions only. `initialize` advertises `loadSession: false` or omits the capability, and `session/load` is unsupported. Persistence remains available to the agent loop and tests; resume can still exist as a lower-level factory if another consumer needs it. The editor bridge should reintroduce `session/load` alongside a real session-selection UX and a stable load transcript contract. + +## Acceptance criteria + +- ACP no longer injects `sessionPersistence` solely for `session/load`. +- `initialize` does not advertise load support. +- The `session/load` handler, loading-id tracking, cwd preflight for loaded sessions, and load replay tests are removed. +- Snapshot fixtures no longer rely on load replay presentation. +- [ACP docs](../../../../packages/ui/acp/README.md) describe fresh-session support only. + +## What we give up + +An editor cannot reopen a prior persisted session through ACP. That is a real product feature, but the current implementation is ahead of the UX and ties the bridge to token-level log replay. Keeping persistence while dropping editor load narrows the bridge to the workflow it can currently present cleanly. diff --git a/docs/rfc/rejected/simplification/2026-06-20-drop-acp-terminal-meta.md b/docs/rfc/rejected/simplification/2026-06-20-drop-acp-terminal-meta.md new file mode 100644 index 0000000000..4b7ca91a0c --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-drop-acp-terminal-meta.md @@ -0,0 +1,27 @@ +# RFC: Drop ACP terminal `_meta` rendering + +Status: rejected — Zed is the current target client, and the terminal `_meta` convention is intentional Zed UX with a plain ACP fallback for other clients. + +## Problem + +The ACP bridge implements a Zed-specific terminal-card convention through `_meta.terminal_info`, `_meta.terminal_output`, and `_meta.terminal_exit`. The implemented [rich ACP bash rendering RFC](../../implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md) deliberately avoided ACP's client-side `terminal/create` because bash execution belongs in the harness, but still adopted the reference agents' display-only `_meta` convention. That gives a nicer Zed card at the cost of bridge state, capability negotiation, terminal ids, special update mapping, text fallback tests, and exit-pill parsing in `dsh-tool-bash`. + +The fallback path already exists: render the tool call and completed output as normal ACP content blocks. Non-Zed clients rely on that path anyway, but the Zed terminal card is a current target-client feature rather than speculative decoration. + +## Proposal + +Ignore `clientCapabilities._meta.terminal_output` and render bash results through the plain ACP content path. Keep execution agent-side through `dsh-bash`; only the display-specific terminal metadata is removed. A terminal card can return later if ACP standardizes agent-executed terminals or if the product decides Zed-specific display is worth the maintenance cost. + +This proposal is narrower than [collapsing tool-owned UI presentation](2026-06-20-generic-tool-rendering.md): it keeps generic `presentCall`/`presentResult` if those survive, but removes the terminal sub-shape and `_meta` mapping. + +## Acceptance criteria + +- ACP no longer reads or stores `_meta.terminal_output` capability state. +- `TerminalRendering`, terminal ids, terminal cwd resolution, and `_meta.terminal_*` update mapping disappear from `@deepseek-ai/dsh-acp`. +- `ToolTerminal` disappears from `@deepseek-ai/dsh-tools`, or is unused and deleted with the presentation cleanup. +- Bash result presentation no longer parses exit status for terminal pills. +- The implemented [rich ACP bash rendering RFC](../../implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md) stays in `implemented/` as shipped history and is cross-linked from this proposal if superseded. + +## What we give up + +Zed users lose the dedicated terminal card: no cwd header, terminal display, or exit pill. They still see the command and output as plain content. That is a reasonable simplification while the ACP bridge is still unreleased and the `_meta` keys are a convention rather than a standard. diff --git a/docs/rfc/rejected/simplification/2026-06-20-drop-bash-output-spill-files.md b/docs/rfc/rejected/simplification/2026-06-20-drop-bash-output-spill-files.md new file mode 100644 index 0000000000..a6a47c66a0 --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-drop-bash-output-spill-files.md @@ -0,0 +1,27 @@ +# RFC: Drop bash full-output spill files + +Status: rejected — full-output recovery is a real bash behavior. A future artifact/blob service may generalize it, but dropping spill files before that replacement would lose useful command output. + +## Problem + +`dsh-bash-local` keeps bounded in-memory output and spills large stdout/stderr streams into private temp files. That requires a private directory, random owner-only file creation, close-failure handling, byte-offset incremental reads, lossy read reporting, path rendering in model-facing text, and cleanup discipline. The tool then tells the model to read a local spill path when output was truncated. + +This solves a real problem, but in a narrow and leaky way. A spill path is a process-local filesystem artifact exposed to model output, not a durable harness artifact with scoped access, retention, or UI affordances. It also complicates background-task reads because a lossy incremental read has to point at one or two spill files. + +## Proposal + +Keep tail truncation, drop full-output spill files. A bash result contains the bounded tail plus a clear truncation marker; no path is emitted. If users need full-output recovery, add a generic artifact/blob service with explicit ownership, cleanup, and UI rendering, then let bash attach large outputs to that service. + +This proposal can land independently of [a generic long-running tool runtime](../../proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md). If background tasks stay, `bash_output` should still report that output was dropped, but without advertising a spill path. + +## Acceptance criteria + +- `CollectedOutput` no longer carries spill paths. +- `OutputCollector` keeps bounded buffers only and deletes the temp-file machinery. +- `renderResult()` reports truncation without a filesystem path. +- Tests cover tail truncation and no longer assert full-output file contents. +- Security guidance in [root AGENTS.md](../../../../AGENTS.md) stops treating private spill files as a model-visible interface. + +## What we give up + +A model or user cannot recover the omitted prefix of a huge command output from a temp file. That is acceptable until there is a real artifact service. The current spill path is too much bespoke machinery for a feature whose lifecycle and permissions are not designed. diff --git a/docs/rfc/rejected/simplification/2026-06-20-drop-durable-step-boundaries.md b/docs/rfc/rejected/simplification/2026-06-20-drop-durable-step-boundaries.md new file mode 100644 index 0000000000..833fae5bc6 --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-drop-durable-step-boundaries.md @@ -0,0 +1,28 @@ +# RFC: Drop durable step boundary events + +Status: rejected — `step/end` is the durable indication that a model step finished, and keeping the symmetric `step/start` / `step/end` pair makes crash repair, invariants, and transcript inspection clearer than inferring completion from adjacent step-scoped events. + +## Problem + +The session log stores `step/start` and `step/end` events even though every step-scoped event already carries `{ turn, step }`: assistant chunks, assistant messages, tool calls, tool results, usage, and errors. `deriveMessages()` ignores step boundaries, ACP ignores them for UI, and the main consumers are invariants, tests, snapshot goldens, and crash repair. + +The rejected argument was that boundary events make the log more ceremonial than informative. In practice, `step/end` is concrete information: a reader can tell whether a model request finished, crashed, or is being repaired without deriving that state from the next event. A bare `step/start` is likewise useful for a model request that began but produced no chunks before failing. + +## Proposal + +Make the turn the only durable boundary. Remove `step/start` and `step/end` from `SessionEventMap`; keep the numeric `step` field on events that need grouping. The loop increments the step counter and records step-scoped events with that number, but it no longer appends open/close boundary events. Consumers infer step groups from contiguous events sharing `(turn, step)`. + +The invariants plugin should enforce that step-scoped events have valid positive step numbers within an open turn, not that separate boundary records surround them. Crash repair should not synthesize `step/end`; if an interrupted turn is preserved, the repair path can still close the turn without inventing step boundary records. + +## Acceptance criteria + +- `SessionEventMap` no longer includes `step/start` or `step/end`. +- The loop has no `closeStep()` finalization path. +- ACP snapshots and persistence contract fixtures stop expecting step-boundary lines. +- `deriveMessages()` and replay derive the same message history from step-scoped events. +- The [event taxonomy docs](../../../architecture.md) describe turns as the durable boundary and steps as a field on step-scoped records. +- The session format version and recorded fixtures are refreshed; non-current stored logs are rejected per the pre-release format policy. + +## What we give up + +The log no longer records "a model request started but produced no event before the process died" as a durable fact, and no longer has an explicit "this step completed" marker. That loss is not acceptable while the session log is the durable replay and audit surface. diff --git a/docs/rfc/rejected/simplification/2026-06-20-drop-unused-session-lineage.md b/docs/rfc/rejected/simplification/2026-06-20-drop-unused-session-lineage.md new file mode 100644 index 0000000000..f32b0f3461 --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-drop-unused-session-lineage.md @@ -0,0 +1,27 @@ +# RFC: Drop unused session lineage metadata + +Status: rejected — `parentSession` is part of the documented fork/sub-agent seam and is already preserved by the agent/session resume path. The field is future-facing, but it is not accidental dead state. + +## Problem + +`SessionHeader.parentSession` records the session a new session was forked from. It is defined in `dsh-session`, preserved by persistence backends, copied through resume, documented as lineage metadata, and covered by round-trip tests. The repo has no production fork UI or sub-agent flow that reads it. The planned sub-agent/fork seam is still a TODO, so the field is currently stored future shape. + +The cost is small per file but broad across the format: every backend schema and metadata serializer preserves a value that no completed feature reads yet. Because the header is an on-disk contract, even a placeholder field becomes something future refactors must either maintain, migrate, or deliberately break. + +## Proposal + +Remove `parentSession` from `SessionHeader` until a real fork/resume feature needs lineage. Forking can still seed a new session with prior events if such an API exists, but the durable parent pointer should be introduced alongside the feature that reads it and the UX that explains it. + +If lineage returns, decide then whether it belongs in the immutable header, a session graph index, or a first-class event. The current field should not pre-commit that design. + +## Acceptance criteria + +- `SessionHeader` contains version, id, createdAt, and optional cwd only. +- JSONL and SQLite metadata schemas stop storing parent-session ids. +- Resume and list APIs no longer round-trip `parentSession`. +- Docs and tests remove fork-lineage claims that are not backed by a production consumer. +- The session format version, backend schema versions, and recorded fixtures are refreshed as needed; non-current stored data is rejected per the pre-release format policy, with no migration path. + +## What we give up + +The codebase loses a ready-made lineage hook for future fork/sub-agent UX. That is intentional. The field is easy to reintroduce when the feature exists, and the unreleased stance lets the format change without migrations. diff --git a/docs/rfc/rejected/simplification/2026-06-20-fold-session-persistence-interface.md b/docs/rfc/rejected/simplification/2026-06-20-fold-session-persistence-interface.md new file mode 100644 index 0000000000..229e2810fc --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-fold-session-persistence-interface.md @@ -0,0 +1,27 @@ +# RFC: Fold the persistence interface into dsh-session + +Status: rejected — the separate persistence interface package is the intended modular capability seam for durable backends. Folding it into `dsh-session` would reduce package count at the cost of a cleaner backend boundary. + +## Problem + +`dsh-session-persistence` is an interface package whose main concepts are already owned by `dsh-session`: `SessionHeader`, `SessionEvent`, `SessionId`, `session/event`, and `session/flush`. The package adds the abstract `SessionPersistence` service, the shared write coordinator, and contract helpers. Backend packages depend on it, and `agent-loop` has to optionally find a sibling service for resume. + +The capability-seam split made sense when persistence was a new swappable backend design. After the mutable summary was removed, the interface package mostly wraps the session log's own storage concern. Keeping it separate may be more ceremony than clarity. + +## Proposal + +Move the abstract `SessionPersistence` service, the coordinator, and persistence contract helpers into `dsh-session`. Keep JSONL and SQLite as separate backend packages that register the session-owned service. This preserves backend swappability while deleting one support package and one cross-package seam. + +The implementing PR should update the [capability seams](../../implemented/architecture/2026-06-13-capability-seams.md) guidance with the exception: persistence is not like bash or LLM because its vocabulary and lifecycle events are already the session package's core domain. + +## Acceptance criteria + +- `@deepseek-ai/dsh-session-persistence` is removed as a package. +- `dsh-session` exports the persistence service type, coordinator, and contract helpers. +- JSONL and SQLite backend packages depend on `dsh-session` directly. +- `agent-loop` resume uses the session-owned service key. +- [Session persistence](../../implemented/architecture/2026-06-14-session-persistence.md), [shared persistence write coordinator](../../implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md), and [package docs](../../../../packages/session-persistence/session-persistence/README.md) explain why backend implementations remain separate. + +## What we give up + +`dsh-session` becomes heavier: it owns both the in-memory log and the persistence interface. That is the trade. If third-party persistence backends were already a public ecosystem, the separate interface package would be a cleaner SDK boundary; pre-release, the extra package looks like abstraction before there is an external consumer. diff --git a/docs/rfc/rejected/simplification/2026-06-20-generic-tool-rendering.md b/docs/rfc/rejected/simplification/2026-06-20-generic-tool-rendering.md new file mode 100644 index 0000000000..5b80920a6a --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-generic-tool-rendering.md @@ -0,0 +1,31 @@ +# RFC: Collapse tool-owned UI presentation + +Status: rejected — tool-owned presentation should wait for more real tools before being generalized or deleted. Bash and ACP currently need the existing richer presentation path. + +## Problem + +Tools can define `presentCall()` and `presentResult()` callbacks that return `ToolCallPresentation`, `ToolResultPresentation`, and optional `ToolTerminal` fields. The code itself flags the design as muddy: title, kind, raw input, content, terminal cwd, terminal output, exit code, and signal grew incrementally into a bag of optional fields. ACP then maintains pending call state to pair a result with the original args, creates replay-only presenters on `session/load`, and maps terminal subfields into Zed-specific `_meta`. `dsh-tool-bash` even parses exit status back out of rendered text because the pure replay-safe presenter no longer has the structured `BashRunResult`. + +The real first-party use is bash presentation for ACP. That is too little evidence to freeze a cross-package UI presentation API. + +## Proposal + +Remove tool-owned UI presentation callbacks for now. The canonical tool events already carry the tool name, raw argument string, result content, and error state. UIs render a generic tool card from those fields. Tool-specific rich rendering can return later as a tagged render-intent union after there are at least two real tools and two real consumers to validate the vocabulary. + +As a smaller alternative, replace the current optional-field bag with one explicit union in a single PR; but if the goal is simplification, the stronger move is to delete the callbacks and keep the generic path. + +## Acceptance criteria + +- `ToolDefinition` drops `presentCall` and `presentResult`. +- `ToolCallPresentation`, `ToolResultPresentation`, `ToolTerminal`, and `ToolCallKind` disappear unless a minimal generic UI type still needs one. +- ACP no longer keeps presenter pending state or calls tool callbacks during live streaming/load replay. +- `dsh-tool-bash` no longer parses rendered text to recover exit status for a UI pill. +- Snapshot goldens show generic tool cards and text results. + +## What we give up + +Bash loses its custom terminal-looking card and model-written description placement. The fallback remains reasonable: the command appears as tool input, and the output appears as text. Rich rendering should be designed when the product has enough UI/tool variety to justify a stable presentation contract. + +## Related + +This is the broad version of [dropping ACP terminal metadata](2026-06-20-drop-acp-terminal-meta.md). If this RFC is accepted, that narrower RFC becomes unnecessary. diff --git a/docs/rfc/rejected/simplification/2026-06-20-retire-mid-turn-steering.md b/docs/rfc/rejected/simplification/2026-06-20-retire-mid-turn-steering.md new file mode 100644 index 0000000000..97c7d5dc61 --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-retire-mid-turn-steering.md @@ -0,0 +1,33 @@ +# RFC: Retire mid-turn steering + +Status: rejected — mid-turn steering is an intentional agent capability for between-step user/plugin input and future goal/loop workflows. It is complexity with a product direction, not an accidental duplicate of `send()`. + +## Problem + +The agent exposes two user-message paths that look close but have different lifecycle semantics: `send()` queues a normal user turn, while `steer()` injects a message between steps of the currently running turn and falls back to `send()` when idle. That distinction leaks through the whole stack: `Agent.steer()` is public API, the session log has a durable `steering/message` event, the agent event taxonomy has `agent/steering`, the loop maintains a steering FIFO beside the queued-message FIFO, cancellation clears both queues, and `deriveMessages()` has to render steering as a tagged synthetic user message rather than a normal prompt. + +The continuation seam amplifies the cost. `agent/turn-continuation` defaults to `hadToolCalls || steeringInjected`, so a same-turn steering message can force the loop to call the model again even if the model did not ask for tools. The comments name future `/goal`, `/loop`, and budget-guard uses, but the current repo has no production listener; only tests register the waterfall. Separately, the only production UI that calls `steer()` is the stdio demo. ACP already sends prompts through the ordinary queue while a turn is running. + +## Proposal + +Delete mid-turn user steering for now. `Agent.send()` becomes the single public way to submit user content; when the agent is running, the content waits for the next turn. The loop continues within a turn only for tool calls, not because a user typed while a step was running. A caller that wants to interrupt the current turn uses `cancel()` and then `send()`. + +Remove `Agent.steer()`, the steering FIFO, `steering/message`, `agent/steering`, steering-derived continuation, and the cancellation logic that distinguishes queued messages from steering messages. Remove `agent/turn-continuation` in the same change unless the implementing PR discovers a production listener; without steering, the current repo has no concrete continuation consumer left. If a real budget or goal plugin later needs forced continuation, it should reintroduce a narrower seam with that plugin as the concrete consumer. + +## Acceptance criteria + +- `Agent` exposes one user-message entry point, `send()`. +- The durable session event vocabulary no longer contains `steering/message`. +- `deriveMessages()` renders normal user messages and context injections, with no steering tag path. +- The loop has one queued-message FIFO and no same-turn user-message continuation path. +- `agent/turn-continuation` is removed or narrowed to a named production consumer. +- The stdio UI and docs describe input while running as queued next-turn input. +- The session format version and recorded fixtures are refreshed; non-current stored logs are rejected per the pre-release format policy. + +## What we give up + +A user cannot add same-turn steering content while a model is between tool steps. That behavior is useful in theory for "while you are already working, also consider X", but it is not the behavior ACP exposes today and it makes the turn boundary much harder to reason about. The simpler behavior is reasonable: user input becomes the next prompt, and cancellation remains the explicit tool for replacing in-flight work. + +## Related + +This pairs naturally with [dropping durable step boundaries](2026-06-20-drop-durable-step-boundaries.md), because removing same-turn steering and `agent/turn-continuation` leaves tool calls as the only reason a turn contains multiple model steps. diff --git a/docs/rfc/rejected/simplification/2026-06-20-single-session-acp-bridge.md b/docs/rfc/rejected/simplification/2026-06-20-single-session-acp-bridge.md new file mode 100644 index 0000000000..b2a2c3b84f --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-single-session-acp-bridge.md @@ -0,0 +1,27 @@ +# RFC: Return the ACP bridge to one live session per connection + +Status: rejected — Zed is the current target ACP client and its ACP implementation is explicitly multi-session: it stores live sessions in a `HashMap`, tracks `pending_sessions`, joins concurrent loads for the same id, and tests close-during-load behavior. + +## Problem + +The ACP bridge now supports multiple live sessions on one JSON-RPC connection. That capability brings multi-entry session maps, reverse session/agent lookups, per-session prompt state, loading ids, demux for every event, cross-session teardown, and isolation concerns for future permission prompts and background tasks. The older [multi-session ACP proposal](../../proposed/feature/2026-06-14-acp-multi-session.md) still tracks the unfinished permission-ownership piece; this RFC is the competing simplification path. + +The product target has proven it needs concurrent editor conversations over one harness process: Zed's ACP connection owns multiple sessions and load states. The snapshot replay tier still avoids concurrent model streams because its replay entries are positional; that is a test-fixture limitation, not a reason to remove bridge multiplexing. + +## Proposal + +Scope ACP back to one live session per connection. `session/new` or `session/load` creates the only session record; a second live session request is rejected until the existing session is disposed or the connection closes. If editors need multiple chat tabs, they can launch multiple agent subprocesses until the bridge has a concrete multi-session UX and permission model. + +Remove the multi-session maps and demux where a single `SessionRecord | undefined` is enough. The bridge can still keep the agent/session lifecycle seams that make disposal correct; the simplification is only about multiplexing more than one active session through the same transport. + +## Acceptance criteria + +- ACP has one active session record per connection. +- `session/new` and `session/load` reject while that record exists. +- Event handlers no longer demux across a `Map`. +- Multi-session tests are removed or moved under the proposal that continues to defend multiplexing. +- The existing [multi-session ACP proposal](../../proposed/feature/2026-06-14-acp-multi-session.md) is updated to link this RFC and remains the live direction. + +## What we give up + +An ACP client cannot host several concurrent conversations on one server process. That is a meaningful capability cut. The simpler model is still reasonable for an unreleased harness: one editor conversation maps to one agent process, and cross-session permission/background-task isolation stops being a live correctness burden. diff --git a/docs/rfc/rejected/simplification/2026-06-20-truncate-interrupted-turns.md b/docs/rfc/rejected/simplification/2026-06-20-truncate-interrupted-turns.md new file mode 100644 index 0000000000..90e2b3f7a4 --- /dev/null +++ b/docs/rfc/rejected/simplification/2026-06-20-truncate-interrupted-turns.md @@ -0,0 +1,32 @@ +# RFC: Truncate interrupted final turns on load + +Status: rejected — a single turn can contain substantial real work, including many steps and large tool output. Preserving interrupted turns is preferable to silently dropping that tail on load. + +## Problem + +The current persistence contract preserves a final turn that was durably written but never closed. On load, `interruptedTurnClosers()` scans the tail, synthesizes error `tool/result` events for unanswered tool calls, appends a `step/end` when a step is open, appends `turn/end { kind: 'interrupted' }`, and asks the backend to durably commit that repair. The coordinator, JSONL backend, SQLite backend, session event vocabulary, invariants, docs, and tests all model this synthetic close path. + +This is a lot of machinery to preserve partial work from the last crashed turn. It also invents events that never happened. A synthetic tool result is useful because it makes provider history valid, but it also means the resumed log contains model-visible text that no tool produced. The current design optimizes for maximum tail preservation before there is a released product or a real resume UX that proves partial-turn recovery matters. + +## Proposal + +On load, keep only the last completed turn. A backend still tolerates and truncates a torn final record, but if the parsed durable prefix ends after an open `turn/start`, the canonical repair is to drop every event after the previous `turn/end`. No synthetic `tool/result`, no synthetic `step/end`, no `turn/end { interrupted }`, and no `interrupted` turn-end reason. + +This makes the persisted turn boundary simple: a completed `turn/end` is the checkpoint. Anything after the last checkpoint is crash tail. The next prompt resumes from the last known-valid provider transcript, not from a partially reconstructed final turn. + +## Acceptance criteria + +- `TurnEndReasonMap` drops the `interrupted` variant. +- `interruptedTurnClosers()` and its tests disappear. +- The persistence coordinator's repair hook truncates backend-specific torn/open tail state without appending closers. +- [Session persistence docs](../../../../packages/session-persistence/session-persistence/README.md) say load returns the last completed turn, plus no partial final turn. +- Snapshot and contract tests update together with the behavior they pin. +- The session format version and recorded fixtures are refreshed; non-current stored logs are rejected per the pre-release format policy, with no migration path. + +## What we give up + +A crash can lose real work from the final turn: assistant text, tool calls, and tool output appended after the previous `turn/end`. That is the deliberate simplification. The product is unreleased, the final-turn recovery semantics are not user-proven, and a clean completed-turn checkpoint is much easier to explain, test, and implement. A future "recover partial crashed work" feature should be designed as an explicit user-facing recovery view, not as synthetic events silently inserted into the canonical transcript. + +## Related + +This is a direct simplification of [session persistence](../../implemented/architecture/2026-06-14-session-persistence.md) and [turn enclosure](../../implemented/architecture/2026-06-15-turn-enclosure-invariant.md). It also removes much of the motivation for durable step boundary events, making [drop durable step boundary events](2026-06-20-drop-durable-step-boundaries.md) smaller. diff --git a/eslint.config.mjs b/eslint.config.mjs index a4f48af798..52236e5d64 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -30,13 +30,13 @@ export default tseslint.config( // --- our packages: full strictness ------------------------------------- { - files: ['packages/*/src/**/*.ts', 'examples/**/*.ts', 'scripts/**/*.ts'], + files: ['packages/*/*/src/**/*.ts', 'examples/**/*.ts', 'scripts/**/*.ts'], extends: [ ...tseslint.configs.strictTypeChecked, ], languageOptions: { parserOptions: { - project: ['./tsconfig.typecheck.json'], + project: ['./packages/*/*/tsconfig.json', './tsconfig.json'], tsconfigRootDir: import.meta.dirname, }, }, @@ -81,13 +81,13 @@ export default tseslint.config( // --- tests: same rules, minus the friction that fights test ergonomics -- { - files: ['packages/*/tests/**/*.ts'], + files: ['packages/*/*/tests/**/*.ts', 'examples/*/tests/**/*.ts'], extends: [ ...tseslint.configs.strictTypeChecked, ], languageOptions: { parserOptions: { - project: ['./tsconfig.typecheck.json'], + project: ['./tsconfig.json'], tsconfigRootDir: import.meta.dirname, }, }, diff --git a/examples/AGENTS.md b/examples/AGENTS.md new file mode 100644 index 0000000000..67ea68ae6f --- /dev/null +++ b/examples/AGENTS.md @@ -0,0 +1,26 @@ +# AGENTS.md — Examples + +Runnable demos that show how the harness is wired. **Examples are NOT workspaces** — each `examples/*/package.json` is a private, dependency-free stub with no build. They are booted as unbuilt `tsx` subprocesses via the cordis Loader reading a `cordis.yml`; the `@deepseek-ai/dsh-*` plugin names in those YAML files resolve through the root `tsconfig.json` `paths` map, not through `node_modules`. + +Because examples are not under the `packages/*/src` coverage gate, an example that grows real, reusable *logic* should extract it into a `packages/` package (where it gets the per-file 100% gate and a README). Keep only example-specific glue here: the `cordis.yml` wiring, demo-only mocks/teaching artifacts, and the e2e/snapshot scenarios. There is no `start.ts` — the boot glue (Loader tail, `.env` load, snapshot-mode selection, stdin-dispose lifecycle) lives in each app package's `bin` (`@deepseek-ai/dsh-stdio-agent`, `@deepseek-ai/dsh-acp-agent`), which the `demo:*` scripts invoke against the leaf `cordis.yml`. + +## Every example ships e2e smokes (keyless + with-key) + +Each example must have **both** kinds of end-to-end smoke, because they catch different failures: + +- **Keyless smoke** — boot the example through its real `cordis.yml` via the Loader (no API key), drive it, and assert the rendered output and a clean exit. This is the guard a hand-mounted unit test structurally cannot be: it exercises the REAL load path (`unwrapExports`, `inject`, the whole plugin tree), so a broken plugin export shape — e.g. a stray `export default` that collapses a namespace plugin and drops `inject` — fails here even when unit tests stay green (see [docs/postmortem/0001](../docs/postmortem/0001-acp-default-export-drops-inject.md)). It runs in the default e2e gate (CI has no secrets). +- **With-key smoke** — send a real prompt against the live model and verify the WORLD (a file on disk, a non-empty assistant turn), not the agent's self-report. This proves the actual product works, which a mock/keyless run structurally cannot. Key-gated: it self-skips without `DEEPSEEK_API_KEY` (see [the with-key policy](../AGENTS.md#secrets--env) — inference is cheap here, so write many). + +**Exception — keyless-by-nature examples.** An example whose model is itself a mock/deterministic stand-in (no real provider) has no meaningful with-key smoke; the keyless smoke is the complete requirement. State the exception inline in the test. + +A keyless smoke that spawns the example from a temp cwd must set `TSX_TSCONFIG_PATH` to the repo-root tsconfig — the unbuilt `paths` map is found by searching UP from cwd, so a temp cwd outside the repo would otherwise fall back to stale built `lib/`. Pass `--expose-internals` when the example's `cordis.yml` loads the HMR plugin (mirror the `demo:*` script). + +## Current state + +| Example | Keyless smoke | With-key smoke | +|---|---|---| +| `echo-agent` | `tests/echo.e2e.ts` — boots the real `cordis.yml`, drives the echo tool round-trip and the direct canned reply | **N/A — keyless by nature** (the `mock-echo` model has no real provider) | +| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit | `tests/{full-loop,coding-task,resume}.e2e.ts` — real model + real bash, world-verified | +| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless; `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote | + +See [the root AGENTS.md](../AGENTS.md) for repo-wide conventions and [docs/architecture.md](../docs/architecture.md) for the design. diff --git a/examples/CLAUDE.md b/examples/CLAUDE.md new file mode 120000 index 0000000000..47dc3e3d86 --- /dev/null +++ b/examples/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/examples/README.md b/examples/README.md index 59d3f70719..3fd9259e36 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,23 +1,26 @@ # Examples -Runnable demos (not workspaces) that showcase how the harness is wired. +Runnable demos (not workspaces) that showcase how the harness is wired. Each example is now a **thin leaf**: a `cordis.yml` that picks the swappable backends (an LLM adapter, a bash executor) and loads ONE app package, plus any demo-only mocks. The composition — the spine, the front-door cluster, and the boot glue — lives in the app packages ([`@deepseek-ai/dsh-stdio-agent`](../packages/ui/stdio-agent), [`@deepseek-ai/dsh-acp-agent`](../packages/ui/acp-agent)) and the [`@deepseek-ai/dsh-agent-core`](../packages/core/agent-core) bundle they share. There is no `start.ts`; the `demo:*` scripts invoke each app package's `bin`. ## echo-agent -A mock model + echo tool + stdio UI + JSONL persistence demo. Demonstrates: +A mock model + echo tool on the stdio chat app — the all-mock skeleton. The leaf swaps `dsh-stdio-agent`'s LLM backend to a local `mock-echo` adapter and adds a local `echo` tool. Demonstrates: -- Loading plugins from a `cordis.yml` via `@cordisjs/plugin-loader` + `@cordisjs/plugin-include` +- A thin leaf `cordis.yml` loading the `@deepseek-ai/dsh-stdio-agent` app - Registering a mock `LlmAdapter` (streaming scripted responses) - Registering a tool via `ctx.tools.register()` -- Persisting session events to JSONL via the `session/event` + `session/flush` pattern -- A minimal stdio UI consuming `agent/stream-chunk` and session events +- "Swap the backend, keep the app" — the only difference from `coding-agent` is the adapter -Run with: `pnpm run demo:echo` - -When prompted, type "echo " to trigger a tool call round-trip. +Run with: `pnpm run demo:echo`. When prompted, type "echo " to trigger a tool call round-trip. ## coding-agent -The real thing: DeepSeek V4 + the bash tool suite + stdio chat + JSONL persistence, wired from `cordis.yml`. Where echo-agent proves the skeleton with mocks, this is a usable coding assistant. +The real thing: DeepSeek V4 + the bash tool suite on the same `@deepseek-ai/dsh-stdio-agent` app. Where echo-agent proves the skeleton with mocks, this is a usable coding assistant. Run with: `pnpm run demo:coding` (needs `DEEPSEEK_API_KEY` in the environment or a gitignored repo-root `.env`). See [coding-agent/README.md](coding-agent/README.md) for details. + +## acp-agent + +The same coding agent exposed as an **Agent Client Protocol (ACP)** server over JSON-RPC stdio, via the [`@deepseek-ai/dsh-acp-agent`](../packages/ui/acp-agent) app — drive it from Zed or any other ACP client. Also the home of the keyless snapshot tests. + +Run with: `pnpm run demo:acp` (needs `DEEPSEEK_API_KEY`). See [acp-agent/README.md](acp-agent/README.md) for the Zed setup and the snapshot-test design. diff --git a/examples/acp-agent/README.md b/examples/acp-agent/README.md index a75279e3ed..1df36715ec 100644 --- a/examples/acp-agent/README.md +++ b/examples/acp-agent/README.md @@ -6,11 +6,11 @@ The DeepSeek Harness coding agent exposed as an **Agent Client Protocol (ACP)** pnpm run demo:acp # needs DEEPSEEK_API_KEY (repo-root .env or env) ``` -This boots `@deepseek-ai/dsh-acp` over the shared provider/tool core (`../base.yml`), with `agent-loop` configured with **no pre-created agents** (ACP `session/new` creates them on demand) and JSONL session persistence (so `session/load` works). +This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-acp-agent`](../../packages/ui/acp-agent) app (which bundles the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, JSONL session persistence, and the `@deepseek-ai/dsh-acp` bridge — with **no pre-created agents**, since ACP `session/new` creates them on demand) plus the two swappable backends (`llm-deepseek`, `bash-local`). The app package bakes in the no-stdout-logger cluster, so a leaf has no logger entry to get wrong by default — keeping stdout pure for JSON-RPC. ## stdout is the protocol -This example loads **no stdout logger** — `stdout` carries the JSON-RPC frames, and any other write corrupts them. Do not add `@cordisjs/plugin-logger-console` or a stdio UI here. Use a stderr exporter if you need logs. +This example loads **no stdout logger** — `stdout` carries the JSON-RPC frames, and any other write corrupts them. `@deepseek-ai/dsh-acp-agent` includes no logger entry, so this leaf has none to get wrong by default; do not add one (use a stderr exporter if you need logs). ## Zed configuration @@ -28,8 +28,12 @@ Add to your Zed `settings.json` under `agent_servers`: } ``` -The editor sets each session's `cwd` to the project it opens; the agent's bash tools run there (see the per-session `cwd` note in `packages/acp`), so launch the server from the harness repo with `pnpm --dir …` and let ACP carry the workspace path per session. +The editor sets each session's `cwd` to the project it opens; the agent's bash tools run there (see the per-session `cwd` note in `packages/ui/acp`), so launch the server from the harness repo with `pnpm --dir …` and let ACP carry the workspace path per session. + +## Snapshot tests (record-once / replay-deterministic) + +This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `@deepseek-ai/dsh-llm-replay`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`". The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md](../../docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md) for the full design. ## MVP limitations -The bridge supports N concurrent sessions per connection, each in its own workspace `cwd` (RFC 011). Remaining limits: prompts support ACP's baseline `text` and `resource_link` blocks only, `additionalDirectories` and `mcpServers` are rejected, and the tool-permission gate is deferred (`TODO(rfc010-permission-gate)` — tools run with the executor's full authority). See `packages/acp/README.md` for the full contract. +The bridge supports N concurrent sessions per connection, each in its own workspace `cwd` (RFC 011). Remaining limits: prompts support ACP's baseline `text` and `resource_link` blocks only, `additionalDirectories` and `mcpServers` are rejected, and the tool-permission gate is deferred (`TODO(rfc010-permission-gate)` — tools run with the executor's full authority). See `packages/ui/acp/README.md` for the full contract. diff --git a/examples/acp-agent/cordis.snapshot.yml b/examples/acp-agent/cordis.snapshot.yml new file mode 100644 index 0000000000..b57920668a --- /dev/null +++ b/examples/acp-agent/cordis.snapshot.yml @@ -0,0 +1,39 @@ +# Snapshot-test REPLAY config: the acp-agent plugin tree with the model backend +# swapped to llm-replay (serves a recorded session JSONL — no API key, no +# network). The dsh-acp-agent bin selects this file for DSH_SNAPSHOT=replay. +# +# Same app as cordis.yml (@deepseek-ai/dsh-acp-agent: the agent-core spine + +# JSONL persistence + the ACP bridge) — only the LLM backend differs: llm-replay +# here, llm-deepseek there. It can't reuse the real adapter because llm-deepseek's +# apply() throws without DEEPSEEK_API_KEY, killing a keyless replay run at boot. +# +# stdout is reserved for the ACP JSON-RPC protocol — no stdout logger (the app +# package omits it). The replay fixture path comes from $DSH_SNAPSHOT_FILE (and +# an optional $DSH_SNAPSHOT_OVERRIDE sidecar), set by the snapshot harness. + +# The replay adapter: short-circuits llm/stream with the recorded log's chunks, +# in place of llm-deepseek. +- id: llm-replay + name: '@deepseek-ai/dsh-llm-replay' + +# Local bash executor (the agent's only tool, via agent-core's tool-bash schema). +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + timeoutMs: 60000 + +# The ACP server app — identical to cordis.yml's entry. +- id: acp-agent + name: '@deepseek-ai/dsh-acp-agent' + config: + model: deepseek-v4-flash + persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' + systemPrompt: | + You are a coding assistant driven over the Agent Client Protocol. + + Your only tools are bash (plus bash_output/bash_kill for background + tasks). Do ALL file operations through bash: read with cat/sed/head, + search with grep, write with heredocs (cat <<'EOF' > file), edit with + sed or a rewrite. Each bash call runs in a fresh shell — pass workdir + instead of cd. Check the [exit code: N] marker; verify your work. Keep + answers brief and factual. diff --git a/examples/acp-agent/cordis.yml b/examples/acp-agent/cordis.yml index b21b661851..e456e8ff05 100644 --- a/examples/acp-agent/cordis.yml +++ b/examples/acp-agent/cordis.yml @@ -1,43 +1,42 @@ -# The acp-agent plugin tree, loaded via @cordisjs/plugin-include. +# The acp-agent plugin tree: the ACP server. Also the snapshot RECORD config +# (the dsh-acp-agent bin selects it for DSH_SNAPSHOT=record): a real llm-deepseek +# run whose persisted log the snapshot harness harvests. Just the two swappable +# backends — the DeepSeek adapter and the local bash executor — plus the ACP +# server app (@deepseek-ai/dsh-acp-agent), which bundles the agent-core spine, +# JSONL persistence, and the ACP bridge. # -# CRITICAL: this example loads NO stdout logger (no @cordisjs/plugin-logger- -# console, no stdio-chat). stdout is reserved for the ACP JSON-RPC protocol — -# anything else written there corrupts the frames (see packages/acp, RFC 010 § -# Risks). Use a stderr exporter if you need logging. The timer plugin is loaded -# (no stdout writes); hmr is omitted (an editor manages the subprocess). +# CRITICAL: this tree loads NO stdout logger and NO hmr — stdout is reserved for +# the ACP JSON-RPC protocol (see packages/ui/acp). That guarantee is now a +# property of @deepseek-ai/dsh-acp-agent (it contains no logger entry), not a +# leaf convention: there is no logger here to get wrong. # -# Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) in the -# environment — start.ts loads the gitignored repo-root .env first. +# Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) — the +# dsh-acp-agent bin loads the gitignored repo-root .env first (on STDERR only). -- id: timer - name: '@cordisjs/plugin-timer' - -# Shared provider/tool core (llm, sessions, system-prompt, tools, agents, -# invariants, llm-deepseek, bash-local, tool-bash). Nested include resolved -# relative to THIS file's directory. -- id: base - name: '@cordisjs/plugin-include' +# The DeepSeek adapter. +- id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' config: - path: '../base.yml' + apiKey: !!js process.env.DEEPSEEK_API_KEY + baseURL: !!js process.env.DEEPSEEK_BASE_URL + models: + - deepseek-v4-flash + - deepseek-v4-pro -# agent-loop with NO pre-created agents: ACP `session/new` creates them on -# demand (unlike coding-agent, which pre-creates `main`). -- id: agent-loop - name: '@deepseek-ai/dsh-agent-loop' +# Local bash executor (the agent's only tool, via agent-core's tool-bash schema). +- id: bash + name: '@deepseek-ai/dsh-bash-local' config: - agents: [] + timeoutMs: 60000 -# Durable session persistence — required by the ACP bridge for `session/load`. -- id: session-persistence - name: '@deepseek-ai/dsh-session-persistence-jsonl' - config: - root: './.sessions' - -# The ACP bridge: wires AgentSideConnection to stdin/stdout. -- id: acp - name: '@deepseek-ai/dsh-acp' +# The ACP server app: the agent-core spine + JSONL persistence + the ACP bridge. +# Persistence root: $DSH_SNAPSHOT_SESSIONS_ROOT when the snapshot harness sets it +# (so it can harvest / isolate the log), else ./.sessions for the demo. +- id: acp-agent + name: '@deepseek-ai/dsh-acp-agent' config: model: deepseek-v4-flash + persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' systemPrompt: | You are a coding assistant driven over the Agent Client Protocol. diff --git a/examples/acp-agent/start.ts b/examples/acp-agent/start.ts deleted file mode 100644 index 61a7241c80..0000000000 --- a/examples/acp-agent/start.ts +++ /dev/null @@ -1,34 +0,0 @@ -import { fileURLToPath, pathToFileURL } from 'node:url' -import { Context } from 'cordis' -import Loader from '@cordisjs/plugin-loader' - -// Load DEEPSEEK_API_KEY / DEEPSEEK_BASE_URL from a gitignored repo-root .env -// (Node native). Absent file is fine — the environment may already carry them. -// -// IMPORTANT: this server speaks ACP JSON-RPC on stdout. Do NOT add any -// stdout logging here or in cordis.yml — it would corrupt the protocol frames. -// A present-but-unreadable/malformed .env is a real misconfiguration: surface -// it on STDERR (never stdout) rather than silently running with the wrong env. -try { - process.loadEnvFile(new URL('../../.env', import.meta.url).pathname) -} catch (error) { - if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { - process.stderr.write(`acp-agent: failed to load .env: ${String(error)}\n`) - } - // ENOENT (no .env) is fine — rely on the ambient environment. -} - -// Resolve relative cordis.yml paths from the repo root no matter where the -// editor launches this demo command. -process.chdir(fileURLToPath(new URL('../..', import.meta.url))) - -const ctx = new Context() -ctx.baseUrl = pathToFileURL(import.meta.dirname).href + '/' - -await ctx.plugin(Loader) -await ctx.loader.create({ - name: '@cordisjs/plugin-include', - config: { - path: './cordis.yml', - }, -}) diff --git a/examples/acp-agent/tests/acp.e2e.ts b/examples/acp-agent/tests/acp.e2e.ts index 8dd8af6d01..ee60a7d131 100644 --- a/examples/acp-agent/tests/acp.e2e.ts +++ b/examples/acp-agent/tests/acp.e2e.ts @@ -26,7 +26,11 @@ import { * WITHOUT a key, since it only needs the server to boot and answer initialize. */ -const startScript = fileURLToPath(new URL('../start.ts', import.meta.url)) +// The dsh-acp-agent bin (the demo:acp entry) and this example's cordis.yml. The +// bin resolves its config-path arg from CWD; the subprocess runs from a temp +// workdir, so pass the example config's ABSOLUTE path. +const binScript = fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)) +const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) // Resolve tsx's loader to an ABSOLUTE path: the subprocess runs with cwd set to // a temp workdir (this test launches there and uses it as the session cwd; the // bridge no longer requires cwd === the launch dir, but a temp dir keeps the @@ -55,7 +59,7 @@ interface Spawned { function spawnAcpAgent(cwd: string, env: NodeJS.ProcessEnv = process.env): Spawned { const child = spawn( process.execPath, - ['--import', tsxLoader, startScript], + ['--import', tsxLoader, binScript, configPath], { cwd, env: { ...env, TSX_TSCONFIG_PATH: repoTsconfig }, stdio: ['pipe', 'pipe', 'pipe'] }, ) const stderr: string[] = [] @@ -101,7 +105,7 @@ describe('acp-agent over real stdio (no key required)', () => { // A dummy key lets the deepseek adapter APPLY (it only checks the key is // present at boot, not valid — the key is used only on a real model call, // which this purity test never triggers). So this runs WITHOUT real creds. - const child = spawn(process.execPath, ['--import', tsxLoader, startScript], { + const child = spawn(process.execPath, ['--import', tsxLoader, binScript, configPath], { cwd: workdir, env: { ...process.env, DEEPSEEK_API_KEY: process.env.DEEPSEEK_API_KEY ?? 'sk-dummy-for-boot', TSX_TSCONFIG_PATH: repoTsconfig }, stdio: ['pipe', 'pipe', 'pipe'], diff --git a/examples/acp-agent/tests/acp.snapshot.ts b/examples/acp-agent/tests/acp.snapshot.ts new file mode 100644 index 0000000000..be6aabada0 --- /dev/null +++ b/examples/acp-agent/tests/acp.snapshot.ts @@ -0,0 +1,157 @@ +import { readFile, readdir, writeFile } from 'node:fs/promises' +import { existsSync } from 'node:fs' +import { fileURLToPath } from 'node:url' +import { dirname, join } from 'node:path' +import { describe, expect, it } from 'vitest' +import { type InputScript, runScenario } from './snapshot-harness.ts' +import { type NormalizeContext, normalizeSessionLog, normalizeStdout } from './snapshot-normalize.ts' + +/** + * ACP snapshot tests (REPLAY by default, keyless). Each scenario under + * `snapshots//` ships an `input.json` (the client stdin script) and a + * `session.jsonl` fixture; replay boots the real acp-agent subprocess, drives + * it, and diffs the normalized stdout transcript against the committed + * `stdout.golden.jsonl`. For model scenarios it ALSO checks the re-persisted + * session log — against the `session.jsonl` fixture itself, not a separate + * golden: the fixture doubles as the replay source (recorded scenarios) and the + * expected produced log (both sides normalized before comparing). + * + * `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the + * `session.jsonl` fixtures against the real API and refreshes the stdout golden + * in one pass. + */ + +const SNAPSHOTS_DIR = join(dirname(fileURLToPath(import.meta.url)), 'snapshots') +const RECORDING = process.env.DSH_SNAPSHOT === 'record' + +/** A snapshot scenario and how its fixtures are produced. */ +interface Scenario { + name: string + /** Whether the scenario drives at least one model turn (so a JSONL golden applies). */ + hasModelTurn: boolean + /** + * Whether `test:snapshot:record` regenerates this scenario's `session.jsonl` + * from the LIVE API. `recorded` scenarios are model-driven and reproducible; + * `authored` scenarios (a hand-written `replay.override.json` sidecar drives + * replay — e.g. a provider error or a cancel, which the live API can't be + * coaxed into deterministically) are NEVER re-recorded. + */ + recorded: boolean +} + +const SCENARIOS: Scenario[] = [ + { name: 'handshake', hasModelTurn: false, recorded: false }, + { name: 'reject-extra-dirs', hasModelTurn: false, recorded: false }, + { name: 'text-turn', hasModelTurn: true, recorded: true }, + { name: 'tool-call-turn', hasModelTurn: true, recorded: true }, + { name: 'workspace-edit', hasModelTurn: true, recorded: true }, + { name: 'multi-turn', hasModelTurn: true, recorded: true }, + { name: 'error-finish', hasModelTurn: true, recorded: false }, + { name: 'cancel', hasModelTurn: true, recorded: false }, +] + +/** + * Derive the {@link NormalizeContext} for a `session.jsonl` fixture from its own + * header line (`{ type: 'session', id, cwd }`). A committed fixture carries the + * session id and cwd of the run that harvested it — different from the live + * replay run — so normalizing it against the live run's ctx would leave those + * recorded values unscrubbed. Reading them from the header scrubs the fixture's + * own id/cwd to the same `{{sessionId}}`/`{{cwd}}` tokens the replay output gets. + * An authored fixture whose header is already normalized (`id:'{{sessionId}}'`, + * `cwd:'{{cwd}}'`) yields those tokens as the volatile values, so scrubbing them + * is an idempotent no-op. A header with no `cwd` falls back to a sentinel that + * cannot occur in a log (NOT `''`, which `String.split` would match on every + * character boundary and corrupt the output). + */ +function fixtureContext(fixture: string): NormalizeContext { + const firstLine = fixture.split('\n').find(line => line.trim().length > 0) ?? '{}' + const header = JSON.parse(firstLine) as { id?: unknown; cwd?: unknown } + return { + sessionIds: typeof header.id === 'string' ? [header.id] : [], + cwd: typeof header.cwd === 'string' ? header.cwd : '\0no-cwd\0', + } +} + +for (const scenario of SCENARIOS) { + describe(`snapshot: ${scenario.name}`, () => { + // In RECORD mode, only re-run the `recorded` (live-API) scenarios; the + // `authored` ones (sidecar-driven errors/cancel) are never re-recorded. + it.skipIf(RECORDING && !scenario.recorded)('matches the goldens', async () => { + const dir = join(SNAPSHOTS_DIR, scenario.name) + const input = JSON.parse(await readFile(join(dir, 'input.json'), 'utf8')) as InputScript + const overrideFile = join(dir, 'replay.override.json') + const workspaceDir = join(dir, 'workspace') + const result = await runScenario(input, { + mode: RECORDING ? 'record' : 'replay', + fixtureFile: join(dir, 'session.jsonl'), + ...existsSync(overrideFile) ? { overrideFile } : {}, + ...existsSync(workspaceDir) ? { workspaceDir } : {}, + }) + + const ctx: NormalizeContext = { + sessionIds: result.sessionId !== undefined ? [result.sessionId] : [], + cwd: result.cwd, + } + + // RECORD mode (recorded scenarios only): persist the freshly-harvested log + // back to the scenario's session.jsonl fixture. `--update` refreshes the + // Vitest goldens but NOT this fixture, so write it here. + if (RECORDING && scenario.recorded && scenario.hasModelTurn) { + expect(result.sessionLog, 'record produced no session log to harvest').toBeDefined() + await writeFile(join(dir, 'session.jsonl'), result.sessionLog as string) + } + + await expect(normalizeStdout(result.rawStdout, ctx)) + .toMatchFileSnapshot(join(dir, 'stdout.golden.jsonl')) + + if (scenario.hasModelTurn) { + expect(result.sessionLog, 'a model scenario must persist a session log').toBeDefined() + // Compare the replay run's persisted log against the `session.jsonl` + // fixture — there is no separate session golden. Both sides pass through + // normalizeSessionLog so the comparison is on normalized form: the + // fixture is raw-harvested (its own real session id / cwd / timestamps), + // the replay output has fresh ones, and each is scrubbed against ITS OWN + // volatile values. The fixture's are read from its header line (a + // committed file cannot share the live run's ctx), so the stale recorded + // cwd/id are scrubbed too, not left to leak past the run's `ctx`. + const fixture = await readFile(join(dir, 'session.jsonl'), 'utf8') + expect(normalizeSessionLog(result.sessionLog as string, ctx)) + .toEqual(normalizeSessionLog(fixture, fixtureContext(fixture))) + } + }) + }) +} + +describe('snapshot fixtures', () => { + it('every scenario directory is registered (no orphans)', async () => { + // toMatchFileSnapshot does not prune orphaned golden/fixture files, so a + // renamed/removed scenario could leave a stale dir that nothing exercises. + // Fail loud on any snapshots/ not present in SCENARIOS. + const entries = await readdir(SNAPSHOTS_DIR, { withFileTypes: true }) + const onDisk = entries.filter(e => e.isDirectory()).map(e => e.name).sort() + const registered = SCENARIOS.map(s => s.name).sort() + expect(onDisk).toEqual(registered) + }) + + it('every registered scenario has its required fixture files', async () => { + // Every scenario has an input script and an stdout golden. EVERY scenario + // also needs `session.jsonl`: the harness boots `llm-replay` with that path + // as the replay source for ALL scenarios (acp.snapshot.ts passes + // `fixtureFile: /session.jsonl` unconditionally), and `loadReplayScript` + // throws "fixture not found" when it is absent and no override replaces it. + // A no-model scenario ships a header-only `session.jsonl` (it derives to an + // empty script — no model call is made); a model scenario's fixture also + // doubles as the expected-log artifact the run is diffed against. An authored + // (non-`recorded`) model scenario additionally ships a `replay.override.json` + // sidecar for the throw/hang cases a derived script cannot express. + for (const { name, hasModelTurn, recorded } of SCENARIOS) { + const dir = join(SNAPSHOTS_DIR, name) + expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true) + expect(existsSync(join(dir, 'stdout.golden.jsonl')), `${name}/stdout.golden.jsonl`).toBe(true) + expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true) + if (hasModelTurn && !recorded) { + expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json`).toBe(true) + } + } + }) +}) diff --git a/examples/acp-agent/tests/snapshot-harness.ts b/examples/acp-agent/tests/snapshot-harness.ts new file mode 100644 index 0000000000..7545768b1d --- /dev/null +++ b/examples/acp-agent/tests/snapshot-harness.ts @@ -0,0 +1,323 @@ +/** + * Shared harness for the ACP snapshot tests. A plain module (NOT a *.spec.ts / + * *.snapshot.ts) so importing it never re-registers another file's tests. + * + * It boots the REAL examples/acp-agent subprocess via the cordis Loader (so the + * export-shape bug class stays guarded — see docs/postmortem/0001), drives it + * over real ACP JSON-RPC stdio with a deterministic input script, tees raw + * stdout (for the golden + a purity check) into an SDK `ClientSideConnection`, + * and — in record mode — harvests the persisted session JSONL after a graceful + * shutdown flush. Two pure normalizers turn the captured stdout frames and the + * session-log events into stable, snapshot-able text. + * + * See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md. + */ + +import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process' +import { cp, mkdtemp, readFile, readdir, rm } from 'node:fs/promises' +import { existsSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { Readable, Writable } from 'node:stream' +import { + ClientSideConnection, + ndJsonStream, + PROTOCOL_VERSION, + type Agent as AcpAgent, + type Client, + type RequestPermissionRequest, + type RequestPermissionResponse, + type SessionNotification, +} from '@agentclientprotocol/sdk' + +// The dsh-acp-agent bin (the demo:acp entry) and this example's cordis.yml. +// The bin resolves its config-path arg from CWD and, under DSH_SNAPSHOT=replay, +// swaps it for the sibling cordis.snapshot.yml. The child's cwd is a temp dir +// OUTSIDE the repo, so pass the example config's ABSOLUTE path. +const binScript = fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)) +const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) +const tsxLoader = fileURLToPath(import.meta.resolve('tsx')) +// The repo-root tsconfig: dev/test run UNBUILT and the `@deepseek-ai/dsh-*` +// imports resolve through its `paths` map. The child's cwd is a temp dir +// OUTSIDE the repo, so tsx's upward search would miss it — point tsx at the +// repo tsconfig explicitly (same fix the e2e harness uses). Repo root is four +// levels up from this file (examples/acp-agent/tests). +const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)) + +/** + * One step of a scenario's deterministic input script (`input.json`). The + * harness interprets these in order. `newSession` captures the server-issued + * (random) session id into a `{{sessionId}}` variable that later steps + * reference, since a committed file cannot know the id in advance. + * + * `promptAndCancel` sends a prompt WITHOUT awaiting its response, waits until + * the client observes the first streamed `agent_message_chunk` (so the emitted + * frames deterministically precede the cancellation), then cancels the turn — + * the only way to exercise a cancel deterministically (a plain `prompt` step + * awaits the response, which a cancel/hang scenario would block on forever). + */ +type InputStep = + | { op: 'initialize'; terminalOutput?: boolean } + | { op: 'newSession' } + | { op: 'newSessionExpectError'; additionalDirectories?: string[] } + | { op: 'prompt'; text: string } + | { op: 'promptExpectError'; text: string } + | { op: 'promptAndCancel'; text: string } + | { op: 'cancel' } + +/** A scenario's `input.json`: an ordered list of input steps. */ +export interface InputScript { + steps: InputStep[] +} + +/** The result of running a scenario: raw stdout + the harvested session log. */ +export interface RunResult { + /** Raw stdout bytes (decoded utf8), every newline-delimited JSON-RPC frame. */ + rawStdout: string + /** stderr (for diagnostics on failure). */ + stderr: string + /** The session id the server issued (undefined if no session was created). */ + sessionId?: string + /** The temp cwd the session ran in (the bash workspace). */ + cwd: string + /** The persisted session log's content, if one was produced. */ + sessionLog?: string +} + +interface RunOptions { + /** `replay` (default, keyless) or `record` (real API, harvests the log). */ + mode: 'replay' | 'record' + /** The recorded session JSONL fixture path (replay reads it; record writes near it). */ + fixtureFile: string + /** Optional sidecar override path (replay). */ + overrideFile?: string + /** + * Optional `/workspace/` directory whose contents are copied into + * the temp cwd BEFORE the run — the standard way to seed files the agent + * operates on (a file to read, edit, or grep). Absent for scenarios that + * start from an empty workspace. + */ + workspaceDir?: string +} + +/** + * Run a scenario end-to-end against a freshly-spawned subprocess. Owns the + * child and its temp dirs; always tears them down. Returns the captured stdout + * and (record mode) the harvested session-log path. + */ +export async function runScenario(input: InputScript, opts: RunOptions): Promise { + const cwd = await mkdtemp(join(tmpdir(), 'acp-snap-cwd-')) + const sessionsRoot = await mkdtemp(join(tmpdir(), 'acp-snap-sessions-')) + // Everything past the temp-dir creation runs under a try/finally that always + // removes both dirs — so a failure in workspace seeding, spawn, or any step + // never leaks them (the "e2e tests own their resources" rule). + let child: ChildProcessWithoutNullStreams | undefined + let sessionId: string | undefined + let sessionLog: string | undefined + const rawBuffers: Buffer[] = [] + const stderrChunks: string[] = [] + try { + // Seed the workspace if the scenario ships one (a file the agent reads/edits). + // Copied into the temp cwd so the agent's bash tools see it; the goldens + // normalize the cwd, so the seeded paths stay stable across runs. + if (opts.workspaceDir !== undefined && existsSync(opts.workspaceDir)) { + await cp(opts.workspaceDir, cwd, { recursive: true }) + } + const env: NodeJS.ProcessEnv = { + ...process.env, + TSX_TSCONFIG_PATH: repoTsconfig, + DSH_SNAPSHOT: opts.mode, + DSH_SNAPSHOT_FILE: opts.fixtureFile, + DSH_SNAPSHOT_SESSIONS_ROOT: sessionsRoot, + ...opts.overrideFile !== undefined ? { DSH_SNAPSHOT_OVERRIDE: opts.overrideFile } : {}, + } + + child = spawn( + process.execPath, + ['--import', tsxLoader, binScript, configPath], + { cwd, env, stdio: ['pipe', 'pipe', 'pipe'] }, + ) + + child.stderr.setEncoding('utf8') + child.stderr.on('data', (c: string) => stderrChunks.push(c)) + + // Tee raw stdout: accumulate the bytes for the golden + purity check, and ALSO + // feed the same bytes to the SDK client through a passthrough. Buffer the raw + // bytes (not per-chunk utf8 strings) and decode once at the end, so a + // multibyte sequence split across two 'data' events can't corrupt the golden. + const passthrough = new Readable({ read() {} }) + child.stdout.on('data', (buf: Buffer) => { + rawBuffers.push(buf) + passthrough.push(buf) + }) + child.stdout.on('end', () => passthrough.push(null)) + + const stream = ndJsonStream( + Writable.toWeb(child.stdin) as WritableStream, + Readable.toWeb(passthrough) as ReadableStream, + ) + // Watcher so a step can block until the client OBSERVES a particular + // session/update — used by promptAndCancel to pin frame order (send cancel + // only after the streamed agent_message_chunk has arrived, so those frames + // deterministically precede the cancelled prompt response). + const updateWaiters: { match: (u: SessionNotification['update']) => boolean; resolve: () => void }[] = [] + const waitForUpdate = (match: (u: SessionNotification['update']) => boolean): Promise => + new Promise(resolve => updateWaiters.push({ match, resolve })) + + const makeClient = (_agent: AcpAgent): Client => ({ + sessionUpdate(params: SessionNotification): Promise { + for (let i = updateWaiters.length - 1; i >= 0; i--) { + const waiter = updateWaiters[i] + if (waiter !== undefined && waiter.match(params.update)) { + updateWaiters.splice(i, 1) + waiter.resolve() + } + } + return Promise.resolve() + }, + requestPermission(_params: RequestPermissionRequest): Promise { + return Promise.resolve({ outcome: { outcome: 'cancelled' } }) + }, + }) + const client = new ClientSideConnection(makeClient, stream) + + for (const step of input.steps) { + await runStep(client, step, cwd, waitForUpdate, () => sessionId, (id) => { sessionId = id }) + } + // Done driving: close stdin so the server disposes gracefully (flushing + // persistence) and exits. Then await exit so the harvested log is complete. + child.stdin.end() + await waitForExit(child) + // Harvest the persisted log (if any) while the temp dirs still exist. + const sessionLogPath = await findSessionLog(sessionsRoot) + if (sessionLogPath !== undefined) sessionLog = await readFile(sessionLogPath, 'utf8') + } finally { + // Failure-safe teardown: kill a still-running child and drop the temp dirs + // even if seeding/spawn/a step/harvest threw, so a flaky run never leaks a + // process or dir. `child` is undefined only if spawn itself threw. + if (child !== undefined && child.exitCode === null && child.signalCode === null) { + child.kill('SIGKILL') + await waitForExit(child) + } + await rm(cwd, { recursive: true, force: true }) + await rm(sessionsRoot, { recursive: true, force: true }) + } + + return { + rawStdout: Buffer.concat(rawBuffers).toString('utf8'), + stderr: stderrChunks.join(''), + cwd, + ...sessionId !== undefined ? { sessionId } : {}, + ...sessionLog !== undefined ? { sessionLog } : {}, + } +} + +/** Drive one input step over the client connection. */ +async function runStep( + client: ClientSideConnection, + step: InputStep, + cwd: string, + waitForUpdate: (match: (u: SessionNotification['update']) => boolean) => Promise, + getSessionId: () => string | undefined, + setSessionId: (id: string) => void, +): Promise { + switch (step.op) { + case 'initialize': + await client.initialize({ + protocolVersion: PROTOCOL_VERSION, + clientCapabilities: step.terminalOutput === true ? { _meta: { terminal_output: true } } : {}, + }) + return + case 'newSession': { + const { sessionId } = await client.newSession({ cwd, mcpServers: [] }) + setSessionId(sessionId) + return + } + case 'newSessionExpectError': { + // The bridge rejects a session/new that widens the workspace scope + // (non-empty additionalDirectories / mcpServers — unimplemented). The SDK + // surfaces that as a rejected RPC; swallow it so the run completes and the + // error frame is captured in the transcript. + await client.newSession({ + cwd, + mcpServers: [], + ...step.additionalDirectories !== undefined ? { additionalDirectories: step.additionalDirectories } : {}, + }).then( + () => { throw new Error('snapshot-harness: expected session/new to be rejected but it succeeded') }, + () => { /* expected: the bridge rejected the unsupported workspace scope */ }, + ) + return + } + case 'prompt': { + const sessionId = getSessionId() + if (sessionId === undefined) throw new Error('snapshot-harness: prompt before newSession') + await client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) + return + } + case 'promptExpectError': { + const sessionId = getSessionId() + if (sessionId === undefined) throw new Error('snapshot-harness: promptExpectError before newSession') + // The model fails this turn (a recorded provider error), so the bridge + // answers the prompt with a JSON-RPC error and the SDK rejects. That + // rejection IS the expected editor experience — swallow it so the run + // completes and the stdout transcript (the error frame) is captured. + await client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) + .then(() => { throw new Error('snapshot-harness: expected the prompt to fail but it succeeded') }, + () => { /* expected: the turn failed and the bridge returned an error */ }) + return + } + case 'promptAndCancel': { + const sessionId = getSessionId() + if (sessionId === undefined) throw new Error('snapshot-harness: promptAndCancel before newSession') + // Dispatch the prompt WITHOUT awaiting (a hang fixture never resolves on + // its own). To pin frame order deterministically, wait until the client + // has OBSERVED the hang's streamed agent_message_chunk before cancelling — + // so those update frames always precede the cancelled prompt response in + // the transcript (without this, the late chunk and the response race; see + // the Codex review of commit 5). Then cancel and await the prompt, which + // the bridge settles as `cancelled` once the abort propagates. + const promptDone = client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) + await waitForUpdate(u => u.sessionUpdate === 'agent_message_chunk') + await client.cancel({ sessionId }) + await promptDone + return + } + case 'cancel': { + const sessionId = getSessionId() + if (sessionId === undefined) throw new Error('snapshot-harness: cancel before newSession') + await client.cancel({ sessionId }) + return + } + default: + throw new Error(`snapshot-harness: unknown input op ${JSON.stringify(step)}`) + } +} + +/** Resolve once the child process exits (any code/signal). */ +function waitForExit(child: ChildProcessWithoutNullStreams): Promise { + if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve() + return new Promise(resolve => child.once('exit', () => { resolve() })) +} + +/** Find the single produced `.jsonl` session log under a sessions root, if any. */ +async function findSessionLog(root: string): Promise { + let cwdDirs: string[] + try { + cwdDirs = await readdir(root) + } catch { + return undefined + } + for (const dir of cwdDirs) { + const sub = join(root, dir) + let files: string[] + try { + files = await readdir(sub) + } catch { + continue + } + const jsonl = files.find(f => f.endsWith('.jsonl')) + if (jsonl !== undefined) return join(sub, jsonl) + } + return undefined +} diff --git a/examples/acp-agent/tests/snapshot-normalize.spec.ts b/examples/acp-agent/tests/snapshot-normalize.spec.ts new file mode 100644 index 0000000000..b220344bb9 --- /dev/null +++ b/examples/acp-agent/tests/snapshot-normalize.spec.ts @@ -0,0 +1,93 @@ +import { describe, expect, it } from 'vitest' +import { type NormalizeContext, normalizeSessionLog, normalizeStdout } from '../tests/snapshot-normalize.ts' + +/** + * Unit tests for the pure snapshot normalizers. Live as a *.spec.ts (runs in + * the default unit gate) and import the harness-side normalizers directly. + */ + +const ctx: NormalizeContext = { + sessionIds: ['11111111-2222-3333-4444-555555555555'], + cwd: '/tmp/acp-snap-cwd-abc123', +} + +describe('normalizeStdout', () => { + it('rewrites JSON-RPC ids to a stable first-seen sequence', () => { + const raw = [ + JSON.stringify({ jsonrpc: '2.0', id: 42, method: 'initialize' }), + JSON.stringify({ jsonrpc: '2.0', id: 42, result: {} }), + JSON.stringify({ jsonrpc: '2.0', id: 99, method: 'session/new' }), + ].join('\n') + const out = normalizeStdout(raw, ctx) + expect(out).toContain('"id":1') + expect(out).toContain('"id":2') + expect(out).not.toContain('42') + expect(out).not.toContain('99') + }) + + it('scrubs the cwd and session id anywhere they appear', () => { + const raw = JSON.stringify({ + jsonrpc: '2.0', method: 'session/update', + params: { sessionId: ctx.sessionIds[0], cwd: ctx.cwd, note: `at ${ctx.cwd}/x` }, + }) + const out = normalizeStdout(raw, ctx) + expect(out).toContain('{{sessionId}}') + expect(out).toContain('{{cwd}}') + expect(out).not.toContain(ctx.cwd) + expect(out).not.toContain(ctx.sessionIds[0] as string) + }) + + it('scrubs a stray UUID not in the known list', () => { + const raw = JSON.stringify({ jsonrpc: '2.0', method: 'x', params: { id: 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee' } }) + expect(normalizeStdout(raw, ctx)).toContain('{{sessionId}}') + }) + + it('leaves notification frames without an id untouched in id-space', () => { + const raw = JSON.stringify({ jsonrpc: '2.0', method: 'session/update', params: {} }) + const out = normalizeStdout(raw, ctx) + expect(out).not.toContain('"id"') + }) + + it('throws on a non-JSON stdout line (the purity check)', () => { + const raw = `${JSON.stringify({ jsonrpc: '2.0', id: 1 })}\noops a log leaked\n` + expect(() => normalizeStdout(raw, ctx)).toThrow() + }) + + it('ignores blank lines', () => { + const raw = `\n${JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'm' })}\n\n` + expect(() => normalizeStdout(raw, ctx)).not.toThrow() + }) +}) + +describe('normalizeSessionLog', () => { + const header = (over: object) => JSON.stringify({ type: 'session', version: 0, id: 's', createdAt: 123, ...over }) + const event = (over: object) => JSON.stringify({ type: 'turn/start', seq: 1, time: 999, data: { turn: 1 }, ...over }) + + it('zeroes the header createdAt', () => { + const out = normalizeSessionLog(`${header({})}\n`, ctx) + expect(out).toContain('"createdAt":0') + expect(out).not.toContain('123') + }) + + it('zeroes each event time but keeps seq', () => { + const out = normalizeSessionLog(`${header({})}\n${event({ seq: 7, time: 999 })}\n`, ctx) + expect(out).toContain('"time":0') + expect(out).toContain('"seq":7') // seq is deterministic — NOT scrubbed + expect(out).not.toContain('999') + }) + + it('scrubs cwd and session id deep inside event data', () => { + const ev = JSON.stringify({ + type: 'tool/result', seq: 2, time: 5, + data: { content: [{ type: 'text', text: `wrote ${ctx.cwd}/proof.txt` }] }, + }) + const out = normalizeSessionLog(`${header({ cwd: ctx.cwd })}\n${ev}\n`, ctx) + expect(out).toContain('{{cwd}}') + expect(out).not.toContain(ctx.cwd) + }) + + it('scrubs the session id in the header', () => { + const out = normalizeSessionLog(`${header({ id: ctx.sessionIds[0] })}\n`, ctx) + expect(out).toContain('{{sessionId}}') + }) +}) diff --git a/examples/acp-agent/tests/snapshot-normalize.ts b/examples/acp-agent/tests/snapshot-normalize.ts new file mode 100644 index 0000000000..db0d493535 --- /dev/null +++ b/examples/acp-agent/tests/snapshot-normalize.ts @@ -0,0 +1,104 @@ +/** + * Pure normalizers for the ACP snapshot goldens. They replace the + * non-deterministic values in the two captured surfaces — the stdout JSON-RPC + * transcript and the persisted session JSONL — with stable tokens, so a golden + * compare reflects behavior, not run-to-run noise. Kept dependency-free and + * side-effect-free so they unit-test trivially. + * + * Scrubbed: `randomUUID()` session ids → `{{sessionId}}`; the temp `mkdtemp` + * cwd → `{{cwd}}` (it appears in terminal-card `_meta` and the log header); + * JSON-RPC request `id` → a stable per-transcript sequence; the log's per-event + * `time` (epoch ms) and header `createdAt` → 0. NOT scrubbed: the log's `seq` + * (deterministic — `seq = log.length`, part of the event-log contract). + * + * See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md. + */ + +const SESSION_ID = '{{sessionId}}' +const CWD = '{{cwd}}' + +/** A UUID v4 string, the shape `randomUUID()` produces for session ids. */ +const UUID_RE = /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi + +/** Inputs the normalizers need to recognize a run's volatile values. */ +export interface NormalizeContext { + /** The session id(s) the run issued — replaced with `{{sessionId}}`. */ + sessionIds: string[] + /** The temp cwd the run used — replaced with `{{cwd}}`. */ + cwd: string +} + +/** Replace cwd, session ids, and any stray UUID with stable tokens in a string. */ +function scrubString(value: string, ctx: NormalizeContext): string { + let out = value + // cwd first (longest, most specific), then explicit session ids, then any + // residual UUID (covers ids that appear in places we didn't enumerate). + out = out.split(ctx.cwd).join(CWD) + for (const id of ctx.sessionIds) out = out.split(id).join(SESSION_ID) + out = out.replace(UUID_RE, SESSION_ID) + return out +} + +/** Recursively scrub a parsed JSON value (strings replaced; structure kept). */ +function scrubValue(value: unknown, ctx: NormalizeContext): unknown { + if (typeof value === 'string') return scrubString(value, ctx) + if (Array.isArray(value)) return value.map(v => scrubValue(v, ctx)) + if (value !== null && typeof value === 'object') { + const out: Record = {} + for (const [k, v] of Object.entries(value)) out[k] = scrubValue(v, ctx) + return out + } + return value +} + +/** + * Normalize a raw stdout transcript (newline-delimited JSON-RPC frames) into a + * stable golden in the SAME shape as the wire: one compact JSON frame per line + * (NDJSON), with the JSON-RPC `id` rewritten to a per-transcript sequence + * (1, 2, 3, …) and all volatile strings scrubbed. Throws if any non-empty line + * is not valid JSON — that doubles as the stdout-purity check (no logger leaked + * onto the protocol). + */ +export function normalizeStdout(rawStdout: string, ctx: NormalizeContext): string { + const lines = rawStdout.split('\n').filter(line => line.trim().length > 0) + // Map each distinct JSON-RPC id (request/response correlate by id) to a stable + // sequence number, in first-seen order, so id churn doesn't perturb the golden. + const idSeq = new Map() + const stableId = (id: unknown): number => { + const key = JSON.stringify(id) + let n = idSeq.get(key) + if (n === undefined) { n = idSeq.size + 1; idSeq.set(key, n) } + return n + } + const frames = lines.map((line) => { + const frame = JSON.parse(line) as Record + if ('id' in frame && frame.id !== undefined && frame.id !== null) { + frame.id = stableId(frame.id) + } + return scrubValue(frame, ctx) as Record + }) + return frames.map(f => JSON.stringify(f)).join('\n') + '\n' +} + +/** + * Normalize a session JSONL log into a stable golden: the header line's + * volatile fields (`createdAt`, `id`, `cwd`) and every event's `time` are + * zeroed/scrubbed, all volatile strings scrubbed, and `seq` is LEFT INTACT + * (deterministic by contract). Output is JSONL in the same shape as the input — + * one compact record per line. + */ +export function normalizeSessionLog(rawLog: string, ctx: NormalizeContext): string { + const lines = rawLog.split('\n').filter(line => line.trim().length > 0) + const records = lines.map((line) => { + const record = JSON.parse(line) as Record + // Header line: { type: 'session', createdAt, id, cwd, … }. + if (record.type === 'session') { + if ('createdAt' in record) record.createdAt = 0 + } else if ('time' in record) { + // Event line: zero the epoch-ms timestamp; keep seq (deterministic). + record.time = 0 + } + return scrubValue(record, ctx) as Record + }) + return records.map(r => JSON.stringify(r)).join('\n') + '\n' +} diff --git a/examples/acp-agent/tests/snapshots/cancel/input.json b/examples/acp-agent/tests/snapshots/cancel/input.json new file mode 100644 index 0000000000..0bc989ed10 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "promptAndCancel", "text": "Start a long task; this turn will be cancelled mid-stream." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/cancel/replay.override.json b/examples/acp-agent/tests/snapshots/cancel/replay.override.json new file mode 100644 index 0000000000..8436b9ca06 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel/replay.override.json @@ -0,0 +1,3 @@ +[ + { "kind": "hang" } +] diff --git a/examples/acp-agent/tests/snapshots/cancel/session.jsonl b/examples/acp-agent/tests/snapshots/cancel/session.jsonl new file mode 100644 index 0000000000..fd2d0c3f23 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel/session.jsonl @@ -0,0 +1,8 @@ +{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"} +{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"Start a long task; this turn will be cancelled mid-stream."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":2,"time":0,"data":{"turn":1,"step":1}} +{"type":"assistant/chunk","seq":3,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","seq":4,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"partial"}}} +{"type":"step/end","seq":5,"time":0,"data":{"turn":1,"step":1}} +{"type":"turn/end","seq":6,"time":0,"data":{"turn":1,"reason":{"kind":"aborted","reason":"session/cancel"}}} diff --git a/examples/acp-agent/tests/snapshots/cancel/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/cancel/stdout.golden.jsonl new file mode 100644 index 0000000000..3400223f4d --- /dev/null +++ b/examples/acp-agent/tests/snapshots/cancel/stdout.golden.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"partial"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"cancelled"}} diff --git a/examples/acp-agent/tests/snapshots/error-finish/input.json b/examples/acp-agent/tests/snapshots/error-finish/input.json new file mode 100644 index 0000000000..29079a89a3 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/error-finish/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "promptExpectError", "text": "This prompt triggers a recorded provider error." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/error-finish/replay.override.json b/examples/acp-agent/tests/snapshots/error-finish/replay.override.json new file mode 100644 index 0000000000..eea32f25ca --- /dev/null +++ b/examples/acp-agent/tests/snapshots/error-finish/replay.override.json @@ -0,0 +1,3 @@ +[ + { "kind": "throw", "chunks": [], "message": "simulated provider error (HTTP 401)", "code": "AUTH", "status": 401 } +] diff --git a/examples/acp-agent/tests/snapshots/error-finish/session.jsonl b/examples/acp-agent/tests/snapshots/error-finish/session.jsonl new file mode 100644 index 0000000000..42dd973de2 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/error-finish/session.jsonl @@ -0,0 +1,6 @@ +{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"} +{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"This prompt triggers a recorded provider error."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":2,"time":0,"data":{"turn":1,"step":1}} +{"type":"step/end","seq":3,"time":0,"data":{"turn":1,"step":1}} +{"type":"turn/end","seq":4,"time":0,"data":{"turn":1,"reason":{"kind":"error","step":1,"message":"simulated provider error (HTTP 401)","code":"AUTH"}}} diff --git a/examples/acp-agent/tests/snapshots/error-finish/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/error-finish/stdout.golden.jsonl new file mode 100644 index 0000000000..8515dacf5a --- /dev/null +++ b/examples/acp-agent/tests/snapshots/error-finish/stdout.golden.jsonl @@ -0,0 +1,3 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","id":3,"error":{"code":-32603,"message":"Internal error: turn failed: simulated provider error (HTTP 401)"}} diff --git a/examples/acp-agent/tests/snapshots/handshake/input.json b/examples/acp-agent/tests/snapshots/handshake/input.json new file mode 100644 index 0000000000..e1e84be919 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/handshake/input.json @@ -0,0 +1,6 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" } + ] +} diff --git a/examples/acp-agent/tests/snapshots/handshake/session.jsonl b/examples/acp-agent/tests/snapshots/handshake/session.jsonl new file mode 100644 index 0000000000..a6f73319bc --- /dev/null +++ b/examples/acp-agent/tests/snapshots/handshake/session.jsonl @@ -0,0 +1 @@ +{"type":"session","version":0,"id":"00000000-0000-0000-0000-000000000000","createdAt":0} diff --git a/examples/acp-agent/tests/snapshots/handshake/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/handshake/stdout.golden.jsonl new file mode 100644 index 0000000000..ad3f94a046 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/handshake/stdout.golden.jsonl @@ -0,0 +1,2 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} diff --git a/examples/acp-agent/tests/snapshots/multi-turn/input.json b/examples/acp-agent/tests/snapshots/multi-turn/input.json new file mode 100644 index 0000000000..40cf159a46 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/multi-turn/input.json @@ -0,0 +1,8 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "prompt", "text": "Reply with exactly the word: ONE. No tools." }, + { "op": "prompt", "text": "Reply with exactly the word: TWO. No tools." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/multi-turn/session.jsonl b/examples/acp-agent/tests/snapshots/multi-turn/session.jsonl new file mode 100644 index 0000000000..914eecdcb6 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/multi-turn/session.jsonl @@ -0,0 +1,64 @@ +{"type":"session","version":0,"id":"803f0752-a3db-4394-9c93-3b6fcd410664","createdAt":1781834688308,"cwd":"/tmp/acp-snap-cwd-QVaaKH"} +{"type":"turn/start","seq":0,"time":1781834688311,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":1781834688312,"data":{"content":[{"type":"text","text":"Reply with exactly the word: ONE. No tools."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":2,"time":1781834688312,"data":{"turn":1,"step":1}} +{"type":"assistant/chunk","seq":3,"time":1781834688735,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":4,"time":1781834688735,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":5,"time":1781834688864,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" user"}}} +{"type":"assistant/chunk","seq":6,"time":1781834688889,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" wants"}}} +{"type":"assistant/chunk","seq":7,"time":1781834688890,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" me"}}} +{"type":"assistant/chunk","seq":8,"time":1781834688890,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":9,"time":1781834688890,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" reply"}}} +{"type":"assistant/chunk","seq":10,"time":1781834688890,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":11,"time":1781834688920,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" exactly"}}} +{"type":"assistant/chunk","seq":12,"time":1781834688920,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" the"}}} +{"type":"assistant/chunk","seq":13,"time":1781834688921,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" word"}}} +{"type":"assistant/chunk","seq":14,"time":1781834688921,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":15,"time":1781834688921,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"ONE"}}} +{"type":"assistant/chunk","seq":16,"time":1781834688921,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\""}}} +{"type":"assistant/chunk","seq":17,"time":1781834688948,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" and"}}} +{"type":"assistant/chunk","seq":18,"time":1781834688948,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" use"}}} +{"type":"assistant/chunk","seq":19,"time":1781834688979,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" no"}}} +{"type":"assistant/chunk","seq":20,"time":1781834688979,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" tools"}}} +{"type":"assistant/chunk","seq":21,"time":1781834688979,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":22,"time":1781834689007,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"text"}}} +{"type":"assistant/chunk","seq":23,"time":1781834689007,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":1,"text":"ONE"}}} +{"type":"assistant/chunk","seq":24,"time":1781834689008,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The user wants me to reply with exactly the word \"ONE\" and use no tools."}}}} +{"type":"assistant/chunk","seq":25,"time":1781834689008,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"text","text":"ONE"}}}} +{"type":"assistant/chunk","seq":26,"time":1781834689008,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":106,"outputTokens":20,"cacheReadTokens":768,"reasoningTokens":18}}}} +{"type":"assistant/chunk","seq":27,"time":1781834689008,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":28,"time":1781834689009,"data":{"turn":1,"step":1,"content":[{"type":"reasoning","text":"The user wants me to reply with exactly the word \"ONE\" and use no tools."},{"type":"text","text":"ONE"}],"usage":{"inputTokens":106,"outputTokens":20,"cacheReadTokens":768,"reasoningTokens":18}}} +{"type":"step/end","seq":29,"time":1781834689010,"data":{"turn":1,"step":1}} +{"type":"turn/end","seq":30,"time":1781834689010,"data":{"turn":1,"reason":{"kind":"completed"}}} +{"type":"turn/start","seq":31,"time":1781834689017,"data":{"turn":2,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":32,"time":1781834689017,"data":{"content":[{"type":"text","text":"Reply with exactly the word: TWO. No tools."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":33,"time":1781834689017,"data":{"turn":2,"step":1}} +{"type":"assistant/chunk","seq":34,"time":1781834689551,"data":{"turn":2,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":35,"time":1781834689551,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":36,"time":1781834689643,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" user"}}} +{"type":"assistant/chunk","seq":37,"time":1781834689676,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" wants"}}} +{"type":"assistant/chunk","seq":38,"time":1781834689676,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" me"}}} +{"type":"assistant/chunk","seq":39,"time":1781834689676,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":40,"time":1781834689676,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" reply"}}} +{"type":"assistant/chunk","seq":41,"time":1781834689676,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":42,"time":1781834689702,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" exactly"}}} +{"type":"assistant/chunk","seq":43,"time":1781834689702,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" the"}}} +{"type":"assistant/chunk","seq":44,"time":1781834689702,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" word"}}} +{"type":"assistant/chunk","seq":45,"time":1781834689702,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":46,"time":1781834689702,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"T"}}} +{"type":"assistant/chunk","seq":47,"time":1781834689703,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"WO"}}} +{"type":"assistant/chunk","seq":48,"time":1781834689731,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\""}}} +{"type":"assistant/chunk","seq":49,"time":1781834689731,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" and"}}} +{"type":"assistant/chunk","seq":50,"time":1781834689732,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" no"}}} +{"type":"assistant/chunk","seq":51,"time":1781834689759,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" tools"}}} +{"type":"assistant/chunk","seq":52,"time":1781834689759,"data":{"turn":2,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":53,"time":1781834689759,"data":{"turn":2,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"text"}}} +{"type":"assistant/chunk","seq":54,"time":1781834689759,"data":{"turn":2,"step":1,"chunk":{"type":"text-delta","index":1,"text":"T"}}} +{"type":"assistant/chunk","seq":55,"time":1781834689760,"data":{"turn":2,"step":1,"chunk":{"type":"text-delta","index":1,"text":"WO"}}} +{"type":"assistant/chunk","seq":56,"time":1781834689788,"data":{"turn":2,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The user wants me to reply with exactly the word \"TWO\" and no tools."}}}} +{"type":"assistant/chunk","seq":57,"time":1781834689788,"data":{"turn":2,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"text","text":"TWO"}}}} +{"type":"assistant/chunk","seq":58,"time":1781834689788,"data":{"turn":2,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":122,"outputTokens":21,"cacheReadTokens":768,"reasoningTokens":18}}}} +{"type":"assistant/chunk","seq":59,"time":1781834689788,"data":{"turn":2,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":60,"time":1781834689789,"data":{"turn":2,"step":1,"content":[{"type":"reasoning","text":"The user wants me to reply with exactly the word \"TWO\" and no tools."},{"type":"text","text":"TWO"}],"usage":{"inputTokens":122,"outputTokens":21,"cacheReadTokens":768,"reasoningTokens":18}}} +{"type":"step/end","seq":61,"time":1781834689789,"data":{"turn":2,"step":1}} +{"type":"turn/end","seq":62,"time":1781834689789,"data":{"turn":2,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/multi-turn/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/multi-turn/stdout.golden.jsonl new file mode 100644 index 0000000000..22bb6757fd --- /dev/null +++ b/examples/acp-agent/tests/snapshots/multi-turn/stdout.golden.jsonl @@ -0,0 +1,43 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" user"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" wants"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" me"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" exactly"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" the"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" word"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" and"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" use"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" no"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" tools"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" user"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" wants"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" me"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" exactly"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" the"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" word"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"T"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"WO"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" and"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" no"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" tools"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"T"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"WO"}}}} +{"jsonrpc":"2.0","id":4,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/reject-extra-dirs/input.json b/examples/acp-agent/tests/snapshots/reject-extra-dirs/input.json new file mode 100644 index 0000000000..6a10f9a40c --- /dev/null +++ b/examples/acp-agent/tests/snapshots/reject-extra-dirs/input.json @@ -0,0 +1,6 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSessionExpectError", "additionalDirectories": ["/extra-dir"] } + ] +} diff --git a/examples/acp-agent/tests/snapshots/reject-extra-dirs/session.jsonl b/examples/acp-agent/tests/snapshots/reject-extra-dirs/session.jsonl new file mode 100644 index 0000000000..a6f73319bc --- /dev/null +++ b/examples/acp-agent/tests/snapshots/reject-extra-dirs/session.jsonl @@ -0,0 +1 @@ +{"type":"session","version":0,"id":"00000000-0000-0000-0000-000000000000","createdAt":0} diff --git a/examples/acp-agent/tests/snapshots/reject-extra-dirs/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/reject-extra-dirs/stdout.golden.jsonl new file mode 100644 index 0000000000..4b864fe7f3 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/reject-extra-dirs/stdout.golden.jsonl @@ -0,0 +1,2 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"error":{"code":-32602,"message":"Invalid params: additionalDirectories is not supported in this MVP"}} diff --git a/examples/acp-agent/tests/snapshots/text-turn/input.json b/examples/acp-agent/tests/snapshots/text-turn/input.json new file mode 100644 index 0000000000..5fe0259a4e --- /dev/null +++ b/examples/acp-agent/tests/snapshots/text-turn/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "prompt", "text": "Reply with exactly the word: PONG. Do not use any tools." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/text-turn/session.jsonl b/examples/acp-agent/tests/snapshots/text-turn/session.jsonl new file mode 100644 index 0000000000..7509bf8ef9 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/text-turn/session.jsonl @@ -0,0 +1,34 @@ +{"type":"session","version":0,"id":"b8c052fd-b33f-475a-8a0c-bc3b75527602","createdAt":1781834679270,"cwd":"/tmp/acp-snap-cwd-TJst85"} +{"type":"turn/start","seq":0,"time":1781834679273,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":1781834679273,"data":{"content":[{"type":"text","text":"Reply with exactly the word: PONG. Do not use any tools."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":2,"time":1781834679273,"data":{"turn":1,"step":1}} +{"type":"assistant/chunk","seq":3,"time":1781834680004,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":4,"time":1781834680004,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":5,"time":1781834680079,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" user"}}} +{"type":"assistant/chunk","seq":6,"time":1781834680113,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" wants"}}} +{"type":"assistant/chunk","seq":7,"time":1781834680114,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" me"}}} +{"type":"assistant/chunk","seq":8,"time":1781834680114,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":9,"time":1781834680114,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" reply"}}} +{"type":"assistant/chunk","seq":10,"time":1781834680114,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":11,"time":1781834680137,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" exactly"}}} +{"type":"assistant/chunk","seq":12,"time":1781834680138,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" the"}}} +{"type":"assistant/chunk","seq":13,"time":1781834680138,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" word"}}} +{"type":"assistant/chunk","seq":14,"time":1781834680138,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":15,"time":1781834680138,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"P"}}} +{"type":"assistant/chunk","seq":16,"time":1781834680138,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"ONG"}}} +{"type":"assistant/chunk","seq":17,"time":1781834680167,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\""}}} +{"type":"assistant/chunk","seq":18,"time":1781834680167,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" without"}}} +{"type":"assistant/chunk","seq":19,"time":1781834680167,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" using"}}} +{"type":"assistant/chunk","seq":20,"time":1781834680196,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" any"}}} +{"type":"assistant/chunk","seq":21,"time":1781834680196,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" tools"}}} +{"type":"assistant/chunk","seq":22,"time":1781834680196,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":23,"time":1781834680196,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"text"}}} +{"type":"assistant/chunk","seq":24,"time":1781834680196,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":1,"text":"P"}}} +{"type":"assistant/chunk","seq":25,"time":1781834680225,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":1,"text":"ONG"}}} +{"type":"assistant/chunk","seq":26,"time":1781834680226,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The user wants me to reply with exactly the word \"PONG\" without using any tools."}}}} +{"type":"assistant/chunk","seq":27,"time":1781834680226,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"text","text":"PONG"}}}} +{"type":"assistant/chunk","seq":28,"time":1781834680226,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":878,"outputTokens":22,"cacheReadTokens":0,"reasoningTokens":19}}}} +{"type":"assistant/chunk","seq":29,"time":1781834680226,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":30,"time":1781834680227,"data":{"turn":1,"step":1,"content":[{"type":"reasoning","text":"The user wants me to reply with exactly the word \"PONG\" without using any tools."},{"type":"text","text":"PONG"}],"usage":{"inputTokens":878,"outputTokens":22,"cacheReadTokens":0,"reasoningTokens":19}}} +{"type":"step/end","seq":31,"time":1781834680228,"data":{"turn":1,"step":1}} +{"type":"turn/end","seq":32,"time":1781834680228,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/text-turn/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/text-turn/stdout.golden.jsonl new file mode 100644 index 0000000000..e35715ccad --- /dev/null +++ b/examples/acp-agent/tests/snapshots/text-turn/stdout.golden.jsonl @@ -0,0 +1,24 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" user"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" wants"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" me"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" exactly"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" the"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" word"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"P"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ONG"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" without"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" using"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" any"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" tools"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"P"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"ONG"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/tool-call-turn/input.json b/examples/acp-agent/tests/snapshots/tool-call-turn/input.json new file mode 100644 index 0000000000..92da4668af --- /dev/null +++ b/examples/acp-agent/tests/snapshots/tool-call-turn/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "prompt", "text": "Use the bash tool to run exactly: echo SNAPSHOT_OK. Then reply with the single word DONE and stop." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/tool-call-turn/session.jsonl b/examples/acp-agent/tests/snapshots/tool-call-turn/session.jsonl new file mode 100644 index 0000000000..3aae0d7ccb --- /dev/null +++ b/examples/acp-agent/tests/snapshots/tool-call-turn/session.jsonl @@ -0,0 +1,105 @@ +{"type":"session","version":0,"id":"0e5b2fc1-d220-4a81-b48b-939c14dea057","createdAt":1781834681068,"cwd":"/tmp/acp-snap-cwd-5F2H38"} +{"type":"turn/start","seq":0,"time":1781834681072,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":1781834681073,"data":{"content":[{"type":"text","text":"Use the bash tool to run exactly: echo SNAPSHOT_OK. Then reply with the single word DONE and stop."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":2,"time":1781834681073,"data":{"turn":1,"step":1}} +{"type":"assistant/chunk","seq":3,"time":1781834681472,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":4,"time":1781834681472,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":5,"time":1781834681566,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" user"}}} +{"type":"assistant/chunk","seq":6,"time":1781834681597,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" wants"}}} +{"type":"assistant/chunk","seq":7,"time":1781834681598,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" me"}}} +{"type":"assistant/chunk","seq":8,"time":1781834681598,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":9,"time":1781834681598,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" run"}}} +{"type":"assistant/chunk","seq":10,"time":1781834681598,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" `"}}} +{"type":"assistant/chunk","seq":11,"time":1781834681599,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"echo"}}} +{"type":"assistant/chunk","seq":12,"time":1781834681629,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" S"}}} +{"type":"assistant/chunk","seq":13,"time":1781834681630,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"NA"}}} +{"type":"assistant/chunk","seq":14,"time":1781834681630,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"PS"}}} +{"type":"assistant/chunk","seq":15,"time":1781834681630,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"H"}}} +{"type":"assistant/chunk","seq":16,"time":1781834681630,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"OT"}}} +{"type":"assistant/chunk","seq":17,"time":1781834681630,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"_OK"}}} +{"type":"assistant/chunk","seq":18,"time":1781834681661,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"`"}}} +{"type":"assistant/chunk","seq":19,"time":1781834681661,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" and"}}} +{"type":"assistant/chunk","seq":20,"time":1781834681662,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" then"}}} +{"type":"assistant/chunk","seq":21,"time":1781834681662,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" reply"}}} +{"type":"assistant/chunk","seq":22,"time":1781834681695,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":23,"time":1781834681696,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":24,"time":1781834681696,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"D"}}} +{"type":"assistant/chunk","seq":25,"time":1781834681696,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"ONE"}}} +{"type":"assistant/chunk","seq":26,"time":1781834681696,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\"."}}} +{"type":"assistant/chunk","seq":27,"time":1781834681790,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":28,"time":1781834681790,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":""}}} +{"type":"assistant/chunk","seq":29,"time":1781834681823,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"{"}}} +{"type":"assistant/chunk","seq":30,"time":1781834681824,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":31,"time":1781834681824,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"command"}}} +{"type":"assistant/chunk","seq":32,"time":1781834681824,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":33,"time":1781834681824,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":": "}}} +{"type":"assistant/chunk","seq":34,"time":1781834681856,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":35,"time":1781834681856,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"echo"}}} +{"type":"assistant/chunk","seq":36,"time":1781834681856,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":" S"}}} +{"type":"assistant/chunk","seq":37,"time":1781834681856,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"NA"}}} +{"type":"assistant/chunk","seq":38,"time":1781834681856,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"PS"}}} +{"type":"assistant/chunk","seq":39,"time":1781834681894,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"H"}}} +{"type":"assistant/chunk","seq":40,"time":1781834681895,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"OT"}}} +{"type":"assistant/chunk","seq":41,"time":1781834681895,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"_OK"}}} +{"type":"assistant/chunk","seq":42,"time":1781834681895,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":43,"time":1781834681923,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":", "}}} +{"type":"assistant/chunk","seq":44,"time":1781834681923,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":45,"time":1781834681955,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"description"}}} +{"type":"assistant/chunk","seq":46,"time":1781834681956,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":47,"time":1781834681956,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":": "}}} +{"type":"assistant/chunk","seq":48,"time":1781834681956,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":49,"time":1781834681989,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"Run"}}} +{"type":"assistant/chunk","seq":50,"time":1781834681989,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":" echo"}}} +{"type":"assistant/chunk","seq":51,"time":1781834681989,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":" S"}}} +{"type":"assistant/chunk","seq":52,"time":1781834682020,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"NA"}}} +{"type":"assistant/chunk","seq":53,"time":1781834682020,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"PS"}}} +{"type":"assistant/chunk","seq":54,"time":1781834682020,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"H"}}} +{"type":"assistant/chunk","seq":55,"time":1781834682020,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"OT"}}} +{"type":"assistant/chunk","seq":56,"time":1781834682020,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"_OK"}}} +{"type":"assistant/chunk","seq":57,"time":1781834682052,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":58,"time":1781834682053,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","argumentsDelta":"}"}}} +{"type":"assistant/chunk","seq":59,"time":1781834682119,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The user wants me to run `echo SNAPSHOT_OK` and then reply with \"DONE\"."}}}} +{"type":"assistant/chunk","seq":60,"time":1781834682119,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","arguments":"{\"command\": \"echo SNAPSHOT_OK\", \"description\": \"Run echo SNAPSHOT_OK\"}"}}}} +{"type":"assistant/chunk","seq":61,"time":1781834682119,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":122,"outputTokens":95,"cacheReadTokens":768,"reasoningTokens":23}}}} +{"type":"assistant/chunk","seq":62,"time":1781834682119,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":63,"time":1781834682121,"data":{"turn":1,"step":1,"content":[{"type":"reasoning","text":"The user wants me to run `echo SNAPSHOT_OK` and then reply with \"DONE\"."},{"type":"tool-call","id":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","arguments":"{\"command\": \"echo SNAPSHOT_OK\", \"description\": \"Run echo SNAPSHOT_OK\"}"}],"usage":{"inputTokens":122,"outputTokens":95,"cacheReadTokens":768,"reasoningTokens":23}}} +{"type":"tool/call","seq":64,"time":1781834682121,"data":{"turn":1,"step":1,"callId":"call_00_7bmU1TAadx8ADiZJ3BqC9330","name":"bash","arguments":"{\"command\": \"echo SNAPSHOT_OK\", \"description\": \"Run echo SNAPSHOT_OK\"}"}} +{"type":"tool/result","seq":65,"time":1781834682136,"data":{"turn":1,"step":1,"callId":"call_00_7bmU1TAadx8ADiZJ3BqC9330","content":[{"type":"text","text":"SNAPSHOT_OK\n"}],"isError":false}} +{"type":"step/end","seq":66,"time":1781834682137,"data":{"turn":1,"step":1}} +{"type":"step/start","seq":67,"time":1781834682137,"data":{"turn":1,"step":2}} +{"type":"assistant/chunk","seq":68,"time":1781834682760,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":69,"time":1781834682761,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":70,"time":1781834682826,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" command"}}} +{"type":"assistant/chunk","seq":71,"time":1781834682855,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" ran"}}} +{"type":"assistant/chunk","seq":72,"time":1781834682885,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" successfully"}}} +{"type":"assistant/chunk","seq":73,"time":1781834682886,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" and"}}} +{"type":"assistant/chunk","seq":74,"time":1781834682886,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" output"}}} +{"type":"assistant/chunk","seq":75,"time":1781834682915,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":76,"time":1781834682915,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"S"}}} +{"type":"assistant/chunk","seq":77,"time":1781834682915,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"NA"}}} +{"type":"assistant/chunk","seq":78,"time":1781834682915,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"PS"}}} +{"type":"assistant/chunk","seq":79,"time":1781834682915,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"H"}}} +{"type":"assistant/chunk","seq":80,"time":1781834682916,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"OT"}}} +{"type":"assistant/chunk","seq":81,"time":1781834682946,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"_OK"}}} +{"type":"assistant/chunk","seq":82,"time":1781834682947,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"\"."}}} +{"type":"assistant/chunk","seq":83,"time":1781834682947,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" Now"}}} +{"type":"assistant/chunk","seq":84,"time":1781834682947,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" I"}}} +{"type":"assistant/chunk","seq":85,"time":1781834682947,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" need"}}} +{"type":"assistant/chunk","seq":86,"time":1781834682947,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":87,"time":1781834682976,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" reply"}}} +{"type":"assistant/chunk","seq":88,"time":1781834682977,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":89,"time":1781834682977,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" just"}}} +{"type":"assistant/chunk","seq":90,"time":1781834682977,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":91,"time":1781834682977,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"D"}}} +{"type":"assistant/chunk","seq":92,"time":1781834683007,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"ONE"}}} +{"type":"assistant/chunk","seq":93,"time":1781834683007,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"\"."}}} +{"type":"assistant/chunk","seq":94,"time":1781834683007,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":1,"blockType":"text"}}} +{"type":"assistant/chunk","seq":95,"time":1781834683007,"data":{"turn":1,"step":2,"chunk":{"type":"text-delta","index":1,"text":"D"}}} +{"type":"assistant/chunk","seq":96,"time":1781834683007,"data":{"turn":1,"step":2,"chunk":{"type":"text-delta","index":1,"text":"ONE"}}} +{"type":"assistant/chunk","seq":97,"time":1781834683008,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The command ran successfully and output \"SNAPSHOT_OK\". Now I need to reply with just \"DONE\"."}}}} +{"type":"assistant/chunk","seq":98,"time":1781834683008,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":1,"block":{"type":"text","text":"DONE"}}}} +{"type":"assistant/chunk","seq":99,"time":1781834683008,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":235,"outputTokens":28,"cacheReadTokens":768,"reasoningTokens":25}}}} +{"type":"assistant/chunk","seq":100,"time":1781834683008,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":101,"time":1781834683008,"data":{"turn":1,"step":2,"content":[{"type":"reasoning","text":"The command ran successfully and output \"SNAPSHOT_OK\". Now I need to reply with just \"DONE\"."},{"type":"text","text":"DONE"}],"usage":{"inputTokens":235,"outputTokens":28,"cacheReadTokens":768,"reasoningTokens":25}}} +{"type":"step/end","seq":102,"time":1781834683008,"data":{"turn":1,"step":2}} +{"type":"turn/end","seq":103,"time":1781834683008,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/tool-call-turn/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/tool-call-turn/stdout.golden.jsonl new file mode 100644 index 0000000000..dff67d8e3a --- /dev/null +++ b/examples/acp-agent/tests/snapshots/tool-call-turn/stdout.golden.jsonl @@ -0,0 +1,55 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" user"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" wants"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" me"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" run"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" `"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"echo"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" S"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"NA"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"PS"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"H"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"OT"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"_OK"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"`"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" and"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" then"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"D"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_7bmU1TAadx8ADiZJ3BqC9330","title":"echo SNAPSHOT_OK","kind":"execute","status":"in_progress","rawInput":"echo SNAPSHOT_OK","content":[{"type":"content","content":{"type":"text","text":"Run echo SNAPSHOT_OK"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_7bmU1TAadx8ADiZJ3BqC9330","status":"completed","content":[{"type":"content","content":{"type":"text","text":"```console\nSNAPSHOT_OK\n```"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" command"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" ran"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" successfully"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" and"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" output"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"S"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"NA"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"PS"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"H"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"OT"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"_OK"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" Now"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" I"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" need"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" just"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"D"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"D"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/workspace-edit/input.json b/examples/acp-agent/tests/snapshots/workspace-edit/input.json new file mode 100644 index 0000000000..30d61b908a --- /dev/null +++ b/examples/acp-agent/tests/snapshots/workspace-edit/input.json @@ -0,0 +1,7 @@ +{ + "steps": [ + { "op": "initialize" }, + { "op": "newSession" }, + { "op": "prompt", "text": "A file named greeting.txt in the current directory contains one word. Use the bash tool to append a second line containing the word WORLD to it (so it has two lines), then read the file back with `cat greeting.txt` to confirm, and reply with the single word DONE. Use a single bash call per action." } + ] +} diff --git a/examples/acp-agent/tests/snapshots/workspace-edit/session.jsonl b/examples/acp-agent/tests/snapshots/workspace-edit/session.jsonl new file mode 100644 index 0000000000..235267ad36 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/workspace-edit/session.jsonl @@ -0,0 +1,189 @@ +{"type":"session","version":0,"id":"06bd4899-ec95-43e3-82ca-42c719d8b19b","createdAt":1781834683850,"cwd":"/tmp/acp-snap-cwd-Jsq2M2"} +{"type":"turn/start","seq":0,"time":1781834683853,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}} +{"type":"user/message","seq":1,"time":1781834683854,"data":{"content":[{"type":"text","text":"A file named greeting.txt in the current directory contains one word. Use the bash tool to append a second line containing the word WORLD to it (so it has two lines), then read the file back with `cat greeting.txt` to confirm, and reply with the single word DONE. Use a single bash call per action."}],"source":{"kind":"user"}}} +{"type":"step/start","seq":2,"time":1781834683854,"data":{"turn":1,"step":1}} +{"type":"assistant/chunk","seq":3,"time":1781834684260,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":4,"time":1781834684260,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":5,"time":1781834684370,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" user"}}} +{"type":"assistant/chunk","seq":6,"time":1781834684399,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" wants"}}} +{"type":"assistant/chunk","seq":7,"time":1781834684399,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" me"}}} +{"type":"assistant/chunk","seq":8,"time":1781834684400,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":9,"time":1781834684400,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":":\n"}}} +{"type":"assistant/chunk","seq":10,"time":1781834684400,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"1"}}} +{"type":"assistant/chunk","seq":11,"time":1781834684400,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":12,"time":1781834684428,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" Append"}}} +{"type":"assistant/chunk","seq":13,"time":1781834684429,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" a"}}} +{"type":"assistant/chunk","seq":14,"time":1781834684430,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" second"}}} +{"type":"assistant/chunk","seq":15,"time":1781834684430,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" line"}}} +{"type":"assistant/chunk","seq":16,"time":1781834684430,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" containing"}}} +{"type":"assistant/chunk","seq":17,"time":1781834684458,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":18,"time":1781834684458,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"WOR"}}} +{"type":"assistant/chunk","seq":19,"time":1781834684458,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"LD"}}} +{"type":"assistant/chunk","seq":20,"time":1781834684459,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\""}}} +{"type":"assistant/chunk","seq":21,"time":1781834684459,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":22,"time":1781834684459,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" greeting"}}} +{"type":"assistant/chunk","seq":23,"time":1781834684486,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":".txt"}}} +{"type":"assistant/chunk","seq":24,"time":1781834684487,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\n"}}} +{"type":"assistant/chunk","seq":25,"time":1781834684487,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"2"}}} +{"type":"assistant/chunk","seq":26,"time":1781834684487,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":27,"time":1781834684487,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" Read"}}} +{"type":"assistant/chunk","seq":28,"time":1781834684516,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" the"}}} +{"type":"assistant/chunk","seq":29,"time":1781834684516,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" file"}}} +{"type":"assistant/chunk","seq":30,"time":1781834684516,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" back"}}} +{"type":"assistant/chunk","seq":31,"time":1781834684545,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":32,"time":1781834684545,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" `"}}} +{"type":"assistant/chunk","seq":33,"time":1781834684545,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"cat"}}} +{"type":"assistant/chunk","seq":34,"time":1781834684545,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" greeting"}}} +{"type":"assistant/chunk","seq":35,"time":1781834684574,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":".txt"}}} +{"type":"assistant/chunk","seq":36,"time":1781834684574,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"`"}}} +{"type":"assistant/chunk","seq":37,"time":1781834684574,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" to"}}} +{"type":"assistant/chunk","seq":38,"time":1781834684574,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" confirm"}}} +{"type":"assistant/chunk","seq":39,"time":1781834684603,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\n"}}} +{"type":"assistant/chunk","seq":40,"time":1781834684604,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"3"}}} +{"type":"assistant/chunk","seq":41,"time":1781834684604,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":42,"time":1781834684604,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" Reply"}}} +{"type":"assistant/chunk","seq":43,"time":1781834684604,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":44,"time":1781834684632,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":45,"time":1781834684633,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"D"}}} +{"type":"assistant/chunk","seq":46,"time":1781834684633,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"ONE"}}} +{"type":"assistant/chunk","seq":47,"time":1781834684662,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\"\n\n"}}} +{"type":"assistant/chunk","seq":48,"time":1781834684663,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"But"}}} +{"type":"assistant/chunk","seq":49,"time":1781834684663,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" they"}}} +{"type":"assistant/chunk","seq":50,"time":1781834684663,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" want"}}} +{"type":"assistant/chunk","seq":51,"time":1781834684695,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" \""}}} +{"type":"assistant/chunk","seq":52,"time":1781834684695,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"a"}}} +{"type":"assistant/chunk","seq":53,"time":1781834684724,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" single"}}} +{"type":"assistant/chunk","seq":54,"time":1781834684724,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" bash"}}} +{"type":"assistant/chunk","seq":55,"time":1781834684724,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" call"}}} +{"type":"assistant/chunk","seq":56,"time":1781834684724,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" per"}}} +{"type":"assistant/chunk","seq":57,"time":1781834684725,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" action"}}} +{"type":"assistant/chunk","seq":58,"time":1781834684725,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"\""}}} +{"type":"assistant/chunk","seq":59,"time":1781834684753,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" -"}}} +{"type":"assistant/chunk","seq":60,"time":1781834684782,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" so"}}} +{"type":"assistant/chunk","seq":61,"time":1781834684783,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" I"}}} +{"type":"assistant/chunk","seq":62,"time":1781834684783,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"'ll"}}} +{"type":"assistant/chunk","seq":63,"time":1781834684811,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" do"}}} +{"type":"assistant/chunk","seq":64,"time":1781834684840,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" two"}}} +{"type":"assistant/chunk","seq":65,"time":1781834684873,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" separate"}}} +{"type":"assistant/chunk","seq":66,"time":1781834684873,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" bash"}}} +{"type":"assistant/chunk","seq":67,"time":1781834684873,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" calls"}}} +{"type":"assistant/chunk","seq":68,"time":1781834684902,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":":"}}} +{"type":"assistant/chunk","seq":69,"time":1781834684903,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" one"}}} +{"type":"assistant/chunk","seq":70,"time":1781834684934,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" for"}}} +{"type":"assistant/chunk","seq":71,"time":1781834684935,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" app"}}} +{"type":"assistant/chunk","seq":72,"time":1781834684966,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"ending"}}} +{"type":"assistant/chunk","seq":73,"time":1781834684966,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":","}}} +{"type":"assistant/chunk","seq":74,"time":1781834684967,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" one"}}} +{"type":"assistant/chunk","seq":75,"time":1781834684967,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" for"}}} +{"type":"assistant/chunk","seq":76,"time":1781834684967,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":" reading"}}} +{"type":"assistant/chunk","seq":77,"time":1781834684967,"data":{"turn":1,"step":1,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":78,"time":1781834685092,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":1,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":79,"time":1781834685092,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":""}}} +{"type":"assistant/chunk","seq":80,"time":1781834685120,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"{"}}} +{"type":"assistant/chunk","seq":81,"time":1781834685120,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":82,"time":1781834685120,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"command"}}} +{"type":"assistant/chunk","seq":83,"time":1781834685120,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":84,"time":1781834685120,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":": "}}} +{"type":"assistant/chunk","seq":85,"time":1781834685149,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":86,"time":1781834685149,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"echo"}}} +{"type":"assistant/chunk","seq":87,"time":1781834685149,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" '"}}} +{"type":"assistant/chunk","seq":88,"time":1781834685149,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"WOR"}}} +{"type":"assistant/chunk","seq":89,"time":1781834685149,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"LD"}}} +{"type":"assistant/chunk","seq":90,"time":1781834685149,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"'"}}} +{"type":"assistant/chunk","seq":91,"time":1781834685178,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" >>"}}} +{"type":"assistant/chunk","seq":92,"time":1781834685179,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" greeting"}}} +{"type":"assistant/chunk","seq":93,"time":1781834685179,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":".txt"}}} +{"type":"assistant/chunk","seq":94,"time":1781834685179,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":95,"time":1781834685207,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":", "}}} +{"type":"assistant/chunk","seq":96,"time":1781834685207,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":97,"time":1781834685236,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"description"}}} +{"type":"assistant/chunk","seq":98,"time":1781834685237,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":99,"time":1781834685237,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":": "}}} +{"type":"assistant/chunk","seq":100,"time":1781834685237,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":101,"time":1781834685265,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"App"}}} +{"type":"assistant/chunk","seq":102,"time":1781834685266,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"end"}}} +{"type":"assistant/chunk","seq":103,"time":1781834685266,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" WORLD"}}} +{"type":"assistant/chunk","seq":104,"time":1781834685266,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" line"}}} +{"type":"assistant/chunk","seq":105,"time":1781834685309,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" to"}}} +{"type":"assistant/chunk","seq":106,"time":1781834685310,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":" greeting"}}} +{"type":"assistant/chunk","seq":107,"time":1781834685310,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":".txt"}}} +{"type":"assistant/chunk","seq":108,"time":1781834685310,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":109,"time":1781834685326,"data":{"turn":1,"step":1,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","argumentsDelta":"}"}}} +{"type":"assistant/chunk","seq":110,"time":1781834685385,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The user wants me to:\n1. Append a second line containing \"WORLD\" to greeting.txt\n2. Read the file back with `cat greeting.txt` to confirm\n3. Reply with \"DONE\"\n\nBut they want \"a single bash call per action\" - so I'll do two separate bash calls: one for appending, one for reading."}}}} +{"type":"assistant/chunk","seq":111,"time":1781834685385,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","arguments":"{\"command\": \"echo 'WORLD' >> greeting.txt\", \"description\": \"Append WORLD line to greeting.txt\"}"}}}} +{"type":"assistant/chunk","seq":112,"time":1781834685386,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":161,"outputTokens":146,"cacheReadTokens":768,"reasoningTokens":74}}}} +{"type":"assistant/chunk","seq":113,"time":1781834685386,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":114,"time":1781834685387,"data":{"turn":1,"step":1,"content":[{"type":"reasoning","text":"The user wants me to:\n1. Append a second line containing \"WORLD\" to greeting.txt\n2. Read the file back with `cat greeting.txt` to confirm\n3. Reply with \"DONE\"\n\nBut they want \"a single bash call per action\" - so I'll do two separate bash calls: one for appending, one for reading."},{"type":"tool-call","id":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","arguments":"{\"command\": \"echo 'WORLD' >> greeting.txt\", \"description\": \"Append WORLD line to greeting.txt\"}"}],"usage":{"inputTokens":161,"outputTokens":146,"cacheReadTokens":768,"reasoningTokens":74}}} +{"type":"tool/call","seq":115,"time":1781834685387,"data":{"turn":1,"step":1,"callId":"call_00_r3tvHl3fD0tmV0GKQt032338","name":"bash","arguments":"{\"command\": \"echo 'WORLD' >> greeting.txt\", \"description\": \"Append WORLD line to greeting.txt\"}"}} +{"type":"tool/result","seq":116,"time":1781834685400,"data":{"turn":1,"step":1,"callId":"call_00_r3tvHl3fD0tmV0GKQt032338","content":[{"type":"text","text":"(no output)"}],"isError":false}} +{"type":"step/end","seq":117,"time":1781834685400,"data":{"turn":1,"step":1}} +{"type":"step/start","seq":118,"time":1781834685400,"data":{"turn":1,"step":2}} +{"type":"assistant/chunk","seq":119,"time":1781834686163,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":120,"time":1781834686163,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"App"}}} +{"type":"assistant/chunk","seq":121,"time":1781834686261,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"ended"}}} +{"type":"assistant/chunk","seq":122,"time":1781834686290,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" successfully"}}} +{"type":"assistant/chunk","seq":123,"time":1781834686318,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":124,"time":1781834686319,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" Now"}}} +{"type":"assistant/chunk","seq":125,"time":1781834686319,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" read"}}} +{"type":"assistant/chunk","seq":126,"time":1781834686352,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" the"}}} +{"type":"assistant/chunk","seq":127,"time":1781834686352,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":" file"}}} +{"type":"assistant/chunk","seq":128,"time":1781834686381,"data":{"turn":1,"step":2,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":129,"time":1781834686469,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":1,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":130,"time":1781834686469,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":""}}} +{"type":"assistant/chunk","seq":131,"time":1781834686497,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"{"}}} +{"type":"assistant/chunk","seq":132,"time":1781834686498,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":133,"time":1781834686498,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"command"}}} +{"type":"assistant/chunk","seq":134,"time":1781834686498,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":135,"time":1781834686498,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":": "}}} +{"type":"assistant/chunk","seq":136,"time":1781834686530,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":137,"time":1781834686530,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"cat"}}} +{"type":"assistant/chunk","seq":138,"time":1781834686530,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":" greeting"}}} +{"type":"assistant/chunk","seq":139,"time":1781834686530,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":".txt"}}} +{"type":"assistant/chunk","seq":140,"time":1781834686559,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":141,"time":1781834686591,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":", "}}} +{"type":"assistant/chunk","seq":142,"time":1781834686591,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":143,"time":1781834686591,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"description"}}} +{"type":"assistant/chunk","seq":144,"time":1781834686591,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":145,"time":1781834686591,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":": "}}} +{"type":"assistant/chunk","seq":146,"time":1781834686623,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":147,"time":1781834686623,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"Read"}}} +{"type":"assistant/chunk","seq":148,"time":1781834686623,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":" greeting"}}} +{"type":"assistant/chunk","seq":149,"time":1781834686655,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":".txt"}}} +{"type":"assistant/chunk","seq":150,"time":1781834686655,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":" to"}}} +{"type":"assistant/chunk","seq":151,"time":1781834686684,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":" confirm"}}} +{"type":"assistant/chunk","seq":152,"time":1781834686684,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"\""}}} +{"type":"assistant/chunk","seq":153,"time":1781834686713,"data":{"turn":1,"step":2,"chunk":{"type":"tool-call-delta","index":1,"id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","argumentsDelta":"}"}}} +{"type":"assistant/chunk","seq":154,"time":1781834686745,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"Appended successfully. Now read the file."}}}} +{"type":"assistant/chunk","seq":155,"time":1781834686745,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":1,"block":{"type":"tool-call","id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","arguments":"{\"command\": \"cat greeting.txt\", \"description\": \"Read greeting.txt to confirm\"}"}}}} +{"type":"assistant/chunk","seq":156,"time":1781834686745,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":193,"outputTokens":74,"cacheReadTokens":896,"reasoningTokens":9}}}} +{"type":"assistant/chunk","seq":157,"time":1781834686745,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":158,"time":1781834686745,"data":{"turn":1,"step":2,"content":[{"type":"reasoning","text":"Appended successfully. Now read the file."},{"type":"tool-call","id":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","arguments":"{\"command\": \"cat greeting.txt\", \"description\": \"Read greeting.txt to confirm\"}"}],"usage":{"inputTokens":193,"outputTokens":74,"cacheReadTokens":896,"reasoningTokens":9}}} +{"type":"tool/call","seq":159,"time":1781834686745,"data":{"turn":1,"step":2,"callId":"call_00_SkCP8dgN8aCbLiZDcYa68316","name":"bash","arguments":"{\"command\": \"cat greeting.txt\", \"description\": \"Read greeting.txt to confirm\"}"}} +{"type":"tool/result","seq":160,"time":1781834686758,"data":{"turn":1,"step":2,"callId":"call_00_SkCP8dgN8aCbLiZDcYa68316","content":[{"type":"text","text":"hello\nWORLD\n"}],"isError":false}} +{"type":"step/end","seq":161,"time":1781834686758,"data":{"turn":1,"step":2}} +{"type":"step/start","seq":162,"time":1781834686758,"data":{"turn":1,"step":3}} +{"type":"assistant/chunk","seq":163,"time":1781834687255,"data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":0,"blockType":"reasoning"}}} +{"type":"assistant/chunk","seq":164,"time":1781834687255,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":"The"}}} +{"type":"assistant/chunk","seq":165,"time":1781834687336,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" file"}}} +{"type":"assistant/chunk","seq":166,"time":1781834687365,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" now"}}} +{"type":"assistant/chunk","seq":167,"time":1781834687365,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" has"}}} +{"type":"assistant/chunk","seq":168,"time":1781834687365,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" two"}}} +{"type":"assistant/chunk","seq":169,"time":1781834687366,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" lines"}}} +{"type":"assistant/chunk","seq":170,"time":1781834687396,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":171,"time":1781834687425,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" I"}}} +{"type":"assistant/chunk","seq":172,"time":1781834687426,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":"'ll"}}} +{"type":"assistant/chunk","seq":173,"time":1781834687455,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" reply"}}} +{"type":"assistant/chunk","seq":174,"time":1781834687455,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" with"}}} +{"type":"assistant/chunk","seq":175,"time":1781834687455,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":" D"}}} +{"type":"assistant/chunk","seq":176,"time":1781834687455,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":"ONE"}}} +{"type":"assistant/chunk","seq":177,"time":1781834687455,"data":{"turn":1,"step":3,"chunk":{"type":"reasoning-delta","index":0,"text":"."}}} +{"type":"assistant/chunk","seq":178,"time":1781834687484,"data":{"turn":1,"step":3,"chunk":{"type":"block-start","index":1,"blockType":"text"}}} +{"type":"assistant/chunk","seq":179,"time":1781834687484,"data":{"turn":1,"step":3,"chunk":{"type":"text-delta","index":1,"text":"D"}}} +{"type":"assistant/chunk","seq":180,"time":1781834687484,"data":{"turn":1,"step":3,"chunk":{"type":"text-delta","index":1,"text":"ONE"}}} +{"type":"assistant/chunk","seq":181,"time":1781834687488,"data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":0,"block":{"type":"reasoning","text":"The file now has two lines. I'll reply with DONE."}}}} +{"type":"assistant/chunk","seq":182,"time":1781834687488,"data":{"turn":1,"step":3,"chunk":{"type":"block-end","index":1,"block":{"type":"text","text":"DONE"}}}} +{"type":"assistant/chunk","seq":183,"time":1781834687488,"data":{"turn":1,"step":3,"chunk":{"type":"usage","usage":{"inputTokens":156,"outputTokens":17,"cacheReadTokens":1024,"reasoningTokens":14}}}} +{"type":"assistant/chunk","seq":184,"time":1781834687488,"data":{"turn":1,"step":3,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":185,"time":1781834687489,"data":{"turn":1,"step":3,"content":[{"type":"reasoning","text":"The file now has two lines. I'll reply with DONE."},{"type":"text","text":"DONE"}],"usage":{"inputTokens":156,"outputTokens":17,"cacheReadTokens":1024,"reasoningTokens":14}}} +{"type":"step/end","seq":186,"time":1781834687489,"data":{"turn":1,"step":3}} +{"type":"turn/end","seq":187,"time":1781834687489,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/workspace-edit/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/workspace-edit/stdout.golden.jsonl new file mode 100644 index 0000000000..fa0d19623e --- /dev/null +++ b/examples/acp-agent/tests/snapshots/workspace-edit/stdout.golden.jsonl @@ -0,0 +1,106 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" user"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" wants"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" me"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":":\n"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"1"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" Append"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" a"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" second"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" line"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" containing"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"WOR"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"LD"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" greeting"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":".txt"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\n"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"2"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" Read"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" the"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" file"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" back"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" `"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"cat"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" greeting"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":".txt"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"`"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" to"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" confirm"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\n"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"3"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" Reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"D"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\"\n\n"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"But"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" they"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" want"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" \""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"a"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" single"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" bash"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" call"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" per"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" action"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"\""}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" -"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" so"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" I"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"'ll"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" do"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" two"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" separate"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" bash"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" calls"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":":"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" one"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" for"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" app"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ending"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":","}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" one"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" for"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reading"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_r3tvHl3fD0tmV0GKQt032338","title":"echo 'WORLD' >> greeting.txt","kind":"execute","status":"in_progress","rawInput":"echo 'WORLD' >> greeting.txt","content":[{"type":"content","content":{"type":"text","text":"Append WORLD line to greeting.txt"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_r3tvHl3fD0tmV0GKQt032338","status":"completed","content":[{"type":"content","content":{"type":"text","text":"```console\n(no output)\n```"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"App"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ended"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" successfully"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" Now"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" read"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" the"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" file"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_SkCP8dgN8aCbLiZDcYa68316","title":"cat greeting.txt","kind":"execute","status":"in_progress","rawInput":"cat greeting.txt","content":[{"type":"content","content":{"type":"text","text":"Read greeting.txt to confirm"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_SkCP8dgN8aCbLiZDcYa68316","status":"completed","content":[{"type":"content","content":{"type":"text","text":"```console\nhello\nWORLD\n```"}}]}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" file"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" now"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" has"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" two"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" lines"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" I"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"'ll"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" reply"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" with"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" D"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"D"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"ONE"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/workspace-edit/workspace/greeting.txt b/examples/acp-agent/tests/snapshots/workspace-edit/workspace/greeting.txt new file mode 100644 index 0000000000..ce01362503 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/workspace-edit/workspace/greeting.txt @@ -0,0 +1 @@ +hello diff --git a/examples/base.yml b/examples/base.yml deleted file mode 100644 index 992f6a2c8b..0000000000 --- a/examples/base.yml +++ /dev/null @@ -1,56 +0,0 @@ -# Shared provider/tool core for the example agents, loaded via a nested -# @cordisjs/plugin-include from each example's cordis.yml. Contains everything -# the model and tools need; each example adds its own infra (logger/timer/hmr), -# its agent-loop config (the examples disagree — see below), and its UI plugin. -# -# Deliberately EXCLUDES: -# - the console logger: it writes to stdout, which the acp-agent reserves for -# the JSON-RPC protocol (see packages/acp). Each example loads logging itself. -# - agent-loop: AgentLoop pre-creates its configured `agents` in its -# constructor, and the examples disagree — coding-agent needs a pre-created -# `main` (its stdio-chat calls ctx.agents.get('main')), while acp-agent must -# pre-create NONE (ACP session/new creates agents on demand). So each example -# declares agent-loop with its own `agents` list. -# -# Plugin entries here use package names (resolved from node_modules), so they -# are insensitive to the baseUrl reset that plugin-include performs per file. -# Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) in the env. - -- id: llm - name: '@deepseek-ai/dsh-llm' - -- id: sessions - name: '@deepseek-ai/dsh-session' - -- id: system-prompt - name: '@deepseek-ai/dsh-system-prompt' - -- id: tools - name: '@deepseek-ai/dsh-tools' - -- id: agents - name: '@deepseek-ai/dsh-agent' - -# Dev-mode event-contract assertions + session-log freeze (off in prod). -- id: invariants - name: '@deepseek-ai/dsh-invariants' - -# The DeepSeek adapter. Swap to '@deepseek-ai/dsh-llm-pi-ai' for the pi-ai-backed -# twin (same config shape; `reasoning: high` replaces thinking/reasoningEffort). -- id: llm-deepseek - name: '@deepseek-ai/dsh-llm-deepseek' - config: - apiKey: !!js process.env.DEEPSEEK_API_KEY - baseURL: !!js process.env.DEEPSEEK_BASE_URL - models: - - deepseek-v4-flash - - deepseek-v4-pro - -# Bash execution: the local executor implementation + the tool schemas. -- id: bash - name: '@deepseek-ai/dsh-bash-local' - config: - timeoutMs: 60000 - -- id: tool-bash - name: '@deepseek-ai/dsh-tool-bash' diff --git a/examples/coding-agent/README.md b/examples/coding-agent/README.md index 44445adc5d..7585129382 100644 --- a/examples/coding-agent/README.md +++ b/examples/coding-agent/README.md @@ -32,15 +32,16 @@ RESUME_SESSION_ID= pnpm run demo:coding The id is wired through `cordis.yml` (`resumeSessionId: !!js process.env.RESUME_SESSION_ID`); unset, the agent starts a new session. A missing/unreadable id is non-fatal — it logs a warning and starts no `main` agent. -## What each plugin demonstrates +## What each leaf entry demonstrates + +This example is a thin leaf `cordis.yml`: it picks the swappable backends and loads one app package. The spine (sessions, system-prompt, tools, agents, invariants, `agent-loop`) and the front-door cluster (console logger, JSONL persistence, readline UI, the pre-created `main` agent) all live inside the [`@deepseek-ai/dsh-stdio-agent`](../../packages/ui/stdio-agent) app and the [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) bundle it loads — so the leaf has only four entries: | Entry | Demonstrates | |---|---| +| `hmr` (`@cordisjs/plugin-hmr`) | the dev/demo edit-reload loop — a **leaf** entry (not baked into the app) because it is Loader-only and needs `node --expose-internals`, which `demo:coding` passes | | `llm-deepseek` | real `LlmAdapter` via config (`!!js process.env.…` secrets); swap one line to `@deepseek-ai/dsh-llm-pi-ai` for the library-backed twin | -| `bash` (`dsh-bash-local`) + `tool-bash` | the executor seam + tool schemas as separate plugins | -| `agent-loop` | agent created from config with a coding system prompt | -| `session-persistence` (`dsh-session-persistence-jsonl`) | durable JSONL persistence (`root: ./.sessions`): append-only event log per session, crash-safe atomic writes — the shared backend, no per-example file | -| `src/stdio-chat.ts` | UI as a plugin; copied from echo-agent with reasoning-dimming and an exit-on-idle close handler for piped stdin. Example-local on purpose — extract a shared UI package when a third example needs it | +| `bash` (`dsh-bash-local`) | the executor implementation — the swappable half of the bash seam. The model-facing `bash`/`bash_output`/`bash_kill` tool schemas (`tool-bash`) come from `agent-core`, so only the executor is a leaf choice | +| `stdio-agent` (`@deepseek-ai/dsh-stdio-agent`) | the app bundle: the agent-core spine + console logger + JSONL persistence + readline UI + a pre-created `main` agent. Its config carries the model, system prompt, `persistenceRoot` (`./.sessions`), and `resumeSessionId` — so persistence and the agent are configured here, not wired as separate leaf plugins | ## End-to-end tests (`pnpm run test:e2e`, key-gated) diff --git a/examples/coding-agent/cordis.yml b/examples/coding-agent/cordis.yml index f901a0d68b..497aa8896a 100644 --- a/examples/coding-agent/cordis.yml +++ b/examples/coding-agent/cordis.yml @@ -1,57 +1,59 @@ -# The coding-agent plugin tree, loaded via @cordisjs/plugin-include. -# Infra (logger/timer/hmr) first, then the shared provider/tool core (nested -# include of ../base.yml), then this example's agent-loop config + UI. +# The coding-agent plugin tree: the real coding agent. The two swappable +# backends — the DeepSeek adapter and the local bash executor — plus `hmr` for +# the dev/demo reload loop, then the stdio chat app (@deepseek-ai/dsh-stdio- +# agent), which bundles the whole agent-core spine (timer, llm, sessions, +# system-prompt, tools, agents, invariants, tool-bash, agent-loop), the console +# logger, JSONL persistence, the readline UI, and a pre-created `main` agent. # -# Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) in the -# environment — start.ts loads the gitignored repo-root .env first. - -- id: logger - name: '@cordisjs/plugin-logger-console' - -- id: timer - name: '@cordisjs/plugin-timer' +# `hmr` is a leaf entry (not baked into dsh-stdio-agent): it is a Loader-only +# dev plugin that needs `--expose-internals` — the `demo:coding` script passes +# it. Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) in the +# environment — the dsh-stdio-agent bin loads the gitignored repo-root .env +# first. cordis.yml reads them via the `!!js` tag. +# Hot-module reload for the dev/demo loop (needs `node --expose-internals`). - id: hmr name: '@cordisjs/plugin-hmr' config: root: ['.'] -# Shared provider/tool core (llm, sessions, system-prompt, tools, agents, -# invariants, llm-deepseek, bash-local, tool-bash). Nested include: the path is -# resolved relative to THIS file's directory. -- id: base - name: '@cordisjs/plugin-include' +# The DeepSeek adapter. Swap to '@deepseek-ai/dsh-llm-pi-ai' for the pi-ai-backed +# twin (same config shape; `reasoning: high` replaces thinking/reasoningEffort). +- id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' config: - path: '../base.yml' + apiKey: !!js process.env.DEEPSEEK_API_KEY + baseURL: !!js process.env.DEEPSEEK_BASE_URL + models: + - deepseek-v4-flash + - deepseek-v4-pro -# agent-loop is per-example (NOT in base.yml): coding-agent pre-creates a `main` -# agent its stdio-chat drives via ctx.agents.get('main'). -- id: agent-loop - name: '@deepseek-ai/dsh-agent-loop' +# Local bash executor (the model's only tool, via agent-core's tool-bash schema). +- id: bash + name: '@deepseek-ai/dsh-bash-local' config: - agents: - - id: main - model: deepseek-v4-flash - # Set RESUME_SESSION_ID to continue a prior persisted session (the ids - # live under ./.sessions); unset starts a fresh session each run. - resumeSessionId: !!js process.env.RESUME_SESSION_ID - systemPrompt: | - You are coding-agent, a CLI coding assistant. + timeoutMs: 60000 - Your only tools are bash (plus bash_output/bash_kill for background - tasks). Do ALL file operations through bash: read with cat/sed/head, - search with grep, write with heredocs (cat <<'EOF' > file), edit - with sed or a rewrite. Each bash call runs in a fresh shell — pass - workdir instead of cd, and never rely on shell state between calls. - - Check the [exit code: N] marker on every command; investigate - failures before moving on. Verify your work by running the code or - tests. Keep answers brief and factual. - -- id: session-persistence - name: '@deepseek-ai/dsh-session-persistence-jsonl' +# The stdio chat app: the whole spine + front-door cluster, configured for a +# real coding agent driving a pre-created `main` agent. +- id: stdio-agent + name: '@deepseek-ai/dsh-stdio-agent' config: - root: './.sessions' + model: deepseek-v4-flash + # Set RESUME_SESSION_ID to continue a prior persisted session (the ids live + # under ./.sessions); unset starts a fresh session each run. + resumeSessionId: !!js process.env.RESUME_SESSION_ID + persistenceRoot: './.sessions' + welcome: 'coding-agent ready. Give it a coding task (bash is its only tool).' + systemPrompt: | + You are coding-agent, a CLI coding assistant. -- id: stdio-chat - name: './src/stdio-chat.ts' + Your only tools are bash (plus bash_output/bash_kill for background + tasks). Do ALL file operations through bash: read with cat/sed/head, + search with grep, write with heredocs (cat <<'EOF' > file), edit + with sed or a rewrite. Each bash call runs in a fresh shell — pass + workdir instead of cd, and never rely on shell state between calls. + + Check the [exit code: N] marker on every command; investigate + failures before moving on. Verify your work by running the code or + tests. Keep answers brief and factual. diff --git a/examples/coding-agent/src/stdio-chat.ts b/examples/coding-agent/src/stdio-chat.ts deleted file mode 100644 index 237e976878..0000000000 --- a/examples/coding-agent/src/stdio-chat.ts +++ /dev/null @@ -1,123 +0,0 @@ -import { createInterface } from 'node:readline' -import type { Context } from 'cordis' -import type {} from '@deepseek-ai/dsh-agent' - -export const name = 'stdio-chat' -export const inject = ['agents'] - -// Copied from examples/echo-agent (welcome text + reasoning rendering -// adjusted). Deliberately example-local rather than a shared package — two -// examples don't justify the abstraction yet; revisit at the third. - -/** - * Minimal UI plugin: reads lines from stdin → agent.send(); renders the - * agent's stream chunks and tool activity to stdout. Demonstrates that a UI - * is "just a plugin" — it only consumes the agent/* event taxonomy. - */ -export function apply(ctx: Context) { - let inReasoning = false - ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => { - if (chunk.type === 'reasoning-delta') { - // Dim the chain-of-thought so the answer stands out. - if (!inReasoning) process.stdout.write('\x1B[2m') - inReasoning = true - process.stdout.write(chunk.text) - } else if (chunk.type === 'text-delta') { - if (inReasoning) process.stdout.write('\x1B[0m\n') - inReasoning = false - process.stdout.write(chunk.text) - } - }) - - ctx.on('agent/turn-start', (agent, turn) => { - process.stdout.write(`\n[${agent.id} turn ${turn}] `) - }) - - ctx.on('agent/turn-end', () => { - if (inReasoning) process.stdout.write('\x1B[0m') - inReasoning = false - process.stdout.write('\n> ') - }) - - ctx.on('session/event', (_session, event) => { - if (event.type === 'tool/call') { - const { name: toolName, arguments: args } = event.data - if (inReasoning) process.stdout.write('\x1B[0m') - inReasoning = false - process.stdout.write(`\n [tool call] ${toolName}(${args})`) - } else if (event.type === 'tool/result') { - const { content } = event.data - const text = content.filter(b => b.type === 'text').map(b => b.text).join('') - process.stdout.write(`\n [tool result] ${text}\n `) - } - }) - - ctx.effect(() => { - const reader = createInterface({ - input: process.stdin, - output: process.stdout, - terminal: process.stdin.isTTY && process.stdout.isTTY, - }) - // Piped-input exit, once stdin reaches EOF: - // - If no line ever submitted work (empty stdin, blank-only lines), exit - // immediately — no turn will ever start, so there is nothing to wait - // for. (Gating on an observed 'running' here would hang forever.) - // - If work WAS submitted, exit the next time the agent settles to idle - // AFTER having run. Two subtleties this handles: the loop batches - // several queued messages into ONE turn (one idle), so we don't count - // sends; and agent.send() does NOT synchronously flip status to - // 'running', so requiring an observed 'running' first (`sawRunning`) - // avoids exiting in the gap before the turn starts and dropping work. - let stdinClosed = false - let disposed = false - let submittedWork = false - let sawRunning = false - - const maybeExit = (): void => { - if (disposed || !stdinClosed) return - // No work submitted: nothing will ever run, exit straight away. - // Work submitted: wait until a turn has run and the agent is idle. - if (submittedWork) { - if (!sawRunning) return - const agent = ctx.agents.get('main') - if (agent && agent.status !== 'idle') return // a turn is still running - } - // Let any final output flush, then exit. - setTimeout(() => process.exit(0), 200) - } - - const disposeStatusListener = ctx.on('agent/status', (subject, status) => { - if (subject.id !== 'main') return - if (status === 'running') sawRunning = true - if (status === 'idle') maybeExit() - }) - - reader.on('line', (line) => { - const text = line.trim() - if (!text) return - const agent = ctx.agents.get('main') - if (!agent) { - console.error('agent "main" is not running') - return - } - submittedWork = true - if (agent.status === 'running') { - agent.steer([{ type: 'text', text }]) - } else { - agent.send([{ type: 'text', text }]) - } - }) - reader.on('close', () => { - // Fires for BOTH stdin EOF and plugin disposal (reader.close() below); - // `disposed` guards teardown so HMR/dispose never exits the process. - stdinClosed = true - maybeExit() - }) - process.stdout.write('coding-agent ready. Give it a coding task (bash is its only tool).\n> ') - return () => { - disposed = true - disposeStatusListener() - reader.close() - } - }, 'stdio-chat') -} diff --git a/examples/coding-agent/start.ts b/examples/coding-agent/start.ts deleted file mode 100644 index 1794b6b510..0000000000 --- a/examples/coding-agent/start.ts +++ /dev/null @@ -1,30 +0,0 @@ -import { pathToFileURL } from 'node:url' -import { Context } from 'cordis' -import Loader from '@cordisjs/plugin-loader' - -// Load DEEPSEEK_API_KEY / DEEPSEEK_BASE_URL from a gitignored repo-root .env -// (Node >= 21.7 native). Absent file is fine — the environment may already -// carry the variables; cordis.yml reads them via the `!!js` tag. A -// present-but-unreadable/malformed .env is a real misconfiguration: surface it -// rather than silently running with the wrong environment. -try { - process.loadEnvFile(new URL('../../.env', import.meta.url).pathname) -} catch (error) { - if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { - process.stderr.write(`coding-agent: failed to load .env: ${String(error)}\n`) - } - // ENOENT (no .env) is fine — rely on the ambient environment. -} - -// Boot a Cordis app from this example's cordis.yml — the same shape as the -// upstream `cordis` bin, pinned to this directory. -const ctx = new Context() -ctx.baseUrl = pathToFileURL(import.meta.dirname).href + '/' - -await ctx.plugin(Loader) -await ctx.loader.create({ - name: '@cordisjs/plugin-include', - config: { - path: './cordis.yml', - }, -}) diff --git a/examples/coding-agent/tests/coding-task.e2e.ts b/examples/coding-agent/tests/coding-task.e2e.ts index 4684301725..68bca5cdfa 100644 --- a/examples/coding-agent/tests/coding-task.e2e.ts +++ b/examples/coding-agent/tests/coding-task.e2e.ts @@ -4,6 +4,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' import type { Context } from 'cordis' +import { AgentId } from '@deepseek-ai/dsh-agent' import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts' /** @@ -53,7 +54,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('coding task: fix a failing test expect(before.status).not.toBe(0) ctx = await codingHarness(workdir) - const agent = ctx.agentLoop.create('e2e-task', { + const agent = ctx.agentLoop.create(AgentId('e2e-task'), { model: 'deepseek-v4-flash', systemPrompt: SYSTEM_PROMPT, }) diff --git a/examples/coding-agent/tests/full-loop.e2e.ts b/examples/coding-agent/tests/full-loop.e2e.ts index 93bc0b1fac..2b70d6f339 100644 --- a/examples/coding-agent/tests/full-loop.e2e.ts +++ b/examples/coding-agent/tests/full-loop.e2e.ts @@ -1,5 +1,6 @@ import { afterEach, describe, expect, it } from 'vitest' import type { Context } from 'cordis' +import { AgentId } from '@deepseek-ai/dsh-agent' import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts' /** @@ -20,7 +21,7 @@ afterEach(async () => { describe.skipIf(!process.env.DEEPSEEK_API_KEY)('full loop: real model + real bash tool', () => { it('runs a bash command on request and reports its output', async () => { ctx = await codingHarness(process.cwd()) - const agent = ctx.agentLoop.create('e2e-loop', { + const agent = ctx.agentLoop.create(AgentId('e2e-loop'), { model: 'deepseek-v4-flash', systemPrompt: SYSTEM_PROMPT, }) diff --git a/examples/coding-agent/tests/harness.ts b/examples/coding-agent/tests/harness.ts index ca303691e9..fbe9b10db5 100644 --- a/examples/coding-agent/tests/harness.ts +++ b/examples/coding-agent/tests/harness.ts @@ -5,7 +5,7 @@ import type { SessionEvent } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' import AgentRegistry from '@deepseek-ai/dsh-agent' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' import * as ToolBash from '@deepseek-ai/dsh-tool-bash' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' @@ -39,7 +39,7 @@ export async function codingHarness(workdir: string, persistenceRoot?: string): return ctx } -export function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +export function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { diff --git a/examples/coding-agent/tests/keyless-smoke.e2e.ts b/examples/coding-agent/tests/keyless-smoke.e2e.ts new file mode 100644 index 0000000000..4e5f3e78dc --- /dev/null +++ b/examples/coding-agent/tests/keyless-smoke.e2e.ts @@ -0,0 +1,99 @@ +import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' + +/** + * Keyless Loader-path smoke for examples/coding-agent: boot the REAL example + * through the `@deepseek-ai/dsh-stdio-agent` bin against its `cordis.yml` (the + * cordis Loader, `unwrapExports`, the full plugin tree incl. the + * `@deepseek-ai/dsh-agent-core` bundle and the extracted + * `@deepseek-ai/dsh-ui-stdio`), then close stdin with no prompt and assert the + * ready banner + a clean exit. + * + * No prompt is ever sent, so the model is NEVER called — this is why it runs + * without a real key. coding-agent's `cordis.yml` loads `llm-deepseek`, whose + * `apply()` only requires a key to be PRESENT (it does not validate it and only + * uses it when a stream actually starts), so a dummy key lets the tree boot + * while the absence of any prompt guarantees no network call. The value is the + * real-Loader-path guard for the app + bundle + UI plugin export shapes (a broken + * `export default` that drops `inject`/`Config` would crash here — see postmortem + * 0001), complementing coding-agent's with-key e2e suites which prove the real + * product. + */ + +// The dsh-stdio-agent bin (the demo:coding entry) and this example's cordis.yml. +// The bin resolves its config-path arg from CWD; the test spawns from a temp +// cwd, so we pass the example config's ABSOLUTE path. +const binScript = fileURLToPath(new URL('../../../packages/ui/stdio-agent/src/bin.ts', import.meta.url)) +const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) +const tsxLoader = fileURLToPath(import.meta.resolve('tsx')) +// Dev/test run UNBUILT: resolve `@deepseek-ai/dsh-*` through the root tsconfig +// `paths` map; tsx searches UP from cwd, and we spawn from a temp dir outside +// the repo, so point it at the repo tsconfig (root is four levels up). +const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)) + +let child: ChildProcessWithoutNullStreams | undefined +let workdir: string | undefined + +afterEach(async () => { + if (child !== undefined && child.exitCode === null) child.kill('SIGKILL') + child = undefined + if (workdir !== undefined) await rm(workdir, { recursive: true, force: true }) + workdir = undefined +}) + +async function bootAndEof(): Promise<{ stdout: string; code: number }> { + workdir = await mkdtemp(join(tmpdir(), 'coding-smoke-')) + const cwd = workdir + return new Promise((resolve, reject) => { + const proc = spawn( + process.execPath, + // --expose-internals: cordis.yml loads the HMR plugin (mirrors demo:coding). + ['--expose-internals', '--import', tsxLoader, binScript, configPath], + { + cwd, + env: { + ...process.env, + TSX_TSCONFIG_PATH: repoTsconfig, + // A dummy key so llm-deepseek's apply() (key-PRESENT check only) boots. + // No prompt is sent, so the adapter never streams — no network call. + DEEPSEEK_API_KEY: 'keyless-smoke-no-call', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }, + ) + child = proc + let stdout = '' + let stderr = '' + proc.stdout.setEncoding('utf8') + proc.stdout.on('data', (chunk: string) => { stdout += chunk }) + proc.stderr.setEncoding('utf8') + proc.stderr.on('data', (chunk: string) => { stderr += chunk }) + + const timer = setTimeout(() => { + proc.kill('SIGKILL') + reject(new Error(`coding-agent did not exit within 10s. stdout:\n${stdout}\nstderr:\n${stderr}`)) + }, 10_000) + + proc.on('exit', (code) => { + clearTimeout(timer) + if (code === 0) resolve({ stdout, code }) + else reject(new Error(`coding-agent exited ${code}. stderr:\n${stderr}`)) + }) + proc.on('error', (err) => { clearTimeout(timer); reject(err) }) + + // No prompt — just EOF, so the stdio UI exits without ever running a turn. + proc.stdin.end() + }) +} + +describe('coding-agent keyless smoke (real cordis.yml via the Loader)', () => { + it('boots the full plugin tree, prints its banner, and exits cleanly on EOF', async () => { + const { stdout, code } = await bootAndEof() + expect(code).toBe(0) + expect(stdout).toContain('coding-agent ready.') + }, 15_000) +}) diff --git a/examples/coding-agent/tests/resume.e2e.ts b/examples/coding-agent/tests/resume.e2e.ts index 6fd2dcb567..450938fc6d 100644 --- a/examples/coding-agent/tests/resume.e2e.ts +++ b/examples/coding-agent/tests/resume.e2e.ts @@ -3,7 +3,9 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' import type { Context } from 'cordis' -import type { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import type { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' +import { AgentId } from '@deepseek-ai/dsh-agent' +import { SessionId } from '@deepseek-ai/dsh-session' import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness.ts' /** @@ -15,7 +17,7 @@ import { codingHarness, finalText, SYSTEM_PROMPT, waitForIdle } from './harness. */ const SECRET = 'plum-galaxy-1791' -const SESSION_ID = 'resume-e2e-session' +const SESSION_ID = SessionId('resume-e2e-session') let ctx: Context | undefined let root: string | undefined @@ -38,10 +40,10 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('resume: continue a persisted ses // log on disk survives. ctx = await codingHarness(process.cwd(), root) const first = ctx.agents.create({ - agentId: 'resume-1', + agentId: AgentId('resume-1'), sessionId: SESSION_ID, agentOptions: { model: 'deepseek-v4-flash', systemPrompt: SYSTEM_PROMPT }, - }) as LoopAgent + }).agent as ReactLoopAgent first.send([{ type: 'text', text: `Remember this code for later: ${SECRET}. Just acknowledge it.` }]) await waitForIdle(ctx, first) await ctx.fiber.dispose() @@ -51,11 +53,11 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('resume: continue a persisted ses // session. The loaded event log seeds the live session, so the model sees // run 1's exchange as conversation history. ctx = await codingHarness(process.cwd(), root) - const resumed = await ctx.agents.resume({ - agentId: 'resume-2', + const resumed = (await ctx.agents.resume({ + agentId: AgentId('resume-2'), resumeSessionId: SESSION_ID, agentOptions: { model: 'deepseek-v4-flash', systemPrompt: SYSTEM_PROMPT }, - }) as LoopAgent + })).agent as ReactLoopAgent expect(resumed.session.id).toBe(SESSION_ID) // The prior user turn is in the rehydrated log before the model is asked. expect(JSON.stringify(resumed.session.deriveMessages())).toContain(SECRET) diff --git a/examples/coding-agent/tests/stdio-chat.spec.ts b/examples/coding-agent/tests/stdio-chat.spec.ts deleted file mode 100644 index 3cb128701f..0000000000 --- a/examples/coding-agent/tests/stdio-chat.spec.ts +++ /dev/null @@ -1,33 +0,0 @@ -import { EventEmitter } from 'node:events' -import type { Context } from 'cordis' -import { describe, expect, test, vi } from 'vitest' - -const createInterface = vi.hoisted(() => vi.fn(() => { - const reader = new EventEmitter() as EventEmitter & { close(): void } - reader.close = vi.fn() - return reader -})) - -vi.mock('node:readline', () => ({ createInterface })) - -function fakeContext(): Context { - return { - agents: { get: vi.fn() }, - on: vi.fn(() => vi.fn()), - effect: vi.fn((callback: () => () => void) => callback()), - } as unknown as Context -} - -describe('coding-agent stdio chat', () => { - test('creates a terminal readline interface so TTY editing keys work', async () => { - const { apply } = await import('../src/stdio-chat.ts') - - apply(fakeContext()) - - expect(createInterface).toHaveBeenCalledWith({ - input: process.stdin, - output: process.stdout, - terminal: process.stdin.isTTY && process.stdout.isTTY, - }) - }) -}) diff --git a/examples/echo-agent/README.md b/examples/echo-agent/README.md index a5311e1fd5..de42234ef1 100644 --- a/examples/echo-agent/README.md +++ b/examples/echo-agent/README.md @@ -1,32 +1,32 @@ # echo-agent -Runnable demo: stdin chat with a scripted mock model and an echo tool. +Runnable demo: stdin chat with a scripted mock model and an echo tool. The all-mock skeleton — "swap the backend, keep the app". ## What it shows -- A complete Cordis app loaded from `cordis.yml` — the standard "stack of plugins" pattern -- `mock-llm.ts` — a mock `LlmAdapter` that streams scripted responses and calls the `echo` tool when the user types "echo " -- `echo-tool.ts` — a tool registered via `ctx.tools.register()` that echoes text back uppercased -- `@deepseek-ai/dsh-session-persistence-jsonl` — the durable JSONL persistence backend (loaded from `cordis.yml`, `root: ./.sessions`): append-only event log per session with crash-safe atomic writes, replacing the old write-only example plugin -- `stdio-chat.ts` — a minimal UI plugin: reads stdin lines and `send`/`steer`s the agent, renders stream deltas, tool calls, and tool results +This example is just a leaf `cordis.yml`: it loads the [`@deepseek-ai/dsh-stdio-agent`](../../packages/ui/stdio-agent) app (which bundles the whole [`@deepseek-ai/dsh-agent-core`](../../packages/core/agent-core) spine, the console logger, JSONL persistence, the readline UI, and a pre-created `main` agent), and swaps in two example-local backends plus `hmr`: + +- `mock-llm.ts` — a mock `LlmAdapter` that streams scripted responses and calls the `echo` tool when the user types "echo ". Registered with `ctx.llm.registerAdapter(['mock-echo'], …)`. +- `echo-tool.ts` — a tool registered via `ctx.tools.register(defineTool(…))` with typed `execute` args; echoes text back uppercased. + +Swapping `mock-llm` for the real `llm-deepseek` adapter is all that separates this from `coding-agent` — the same app, a different backend. ## Plugin files | File | Role | Key patterns demonstrated | |---|---|---| -| `mock-llm.ts` | `LlmAdapter` registration | `ctx.llm.registerAdapter(['mock-echo'], …)`, streaming chunks with proper `block-start`/`block-end` protocol | -| `echo-tool.ts` | Tool registration | `ctx.tools.register(defineTool(…))` with typed `execute` args, tool execution returning `ContentBlock[]` | -| `stdio-chat.ts` | UI | `agent/stream-chunk`, `session/event` (tool/*), stdin→send/steer | -| `start.ts` | Bootstrap | `Context` + `Loader` + `plugin-include` wired to `cordis.yml` | +| `src/mock-llm.ts` | `LlmAdapter` registration | `ctx.llm.registerAdapter(['mock-echo'], …)`, streaming chunks with the proper `block-start`/`block-end` protocol | +| `src/echo-tool.ts` | Tool registration | `ctx.tools.register(defineTool(…))` with typed `execute` args, returning `ContentBlock[]` | +| `cordis.yml` | Leaf wiring | the two backends + `hmr` + one `@deepseek-ai/dsh-stdio-agent` entry carrying the app config | -Persistence is the shared `@deepseek-ai/dsh-session-persistence-jsonl` plugin (not a per-example file). +The spine, UI, persistence, and boot glue all live in `@deepseek-ai/dsh-stdio-agent` and the bundle it loads — this folder holds only the demo-specific mocks and the leaf wiring. ## Run ```sh pnpm run demo:echo # or: -node --expose-internals --import tsx examples/echo-agent/start.ts +node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/echo-agent/cordis.yml ``` Type a message and press Enter. "echo " triggers a tool call round-trip (the mock model requests the `echo` tool, which echoes the text uppercased, and the next model step acknowledges it). diff --git a/examples/echo-agent/cordis.yml b/examples/echo-agent/cordis.yml index a226ea3ce5..9eef3d1a1b 100644 --- a/examples/echo-agent/cordis.yml +++ b/examples/echo-agent/cordis.yml @@ -1,55 +1,38 @@ -# The echo-agent plugin tree, loaded via @cordisjs/plugin-include. -# Core services first, then the demo plugins, then the agent itself. - -- id: logger - name: '@cordisjs/plugin-logger-console' - -- id: timer - name: '@cordisjs/plugin-timer' +# The echo-agent plugin tree: the stdio chat app with its LLM backend swapped to +# the local `mock-echo` mock and the local `echo` tool added. The clean +# demonstration of "swap the backend, keep the app" — every service the agent +# needs lives in @deepseek-ai/dsh-stdio-agent (which bundles @deepseek-ai/dsh- +# agent-core); this leaf only picks the backends, `hmr`, and the app config. +# +# No API key: the `mock-echo` adapter never touches the network. +# Hot-module reload for the dev/demo loop (a leaf entry, not baked into +# dsh-stdio-agent — it needs `node --expose-internals`, which `demo:echo` passes). - id: hmr name: '@cordisjs/plugin-hmr' config: root: ['.'] -- id: llm - name: '@deepseek-ai/dsh-llm' - -- id: sessions - name: '@deepseek-ai/dsh-session' - -- id: system-prompt - name: '@deepseek-ai/dsh-system-prompt' - -- id: tools - name: '@deepseek-ai/dsh-tools' - -- id: agents - name: '@deepseek-ai/dsh-agent' - -# Dev-mode event-contract assertions + session-log freeze (off in prod; -# on here so the demo smoke test exercises the contract). -- id: invariants - name: '@deepseek-ai/dsh-invariants' - -- id: agent-loop - name: '@deepseek-ai/dsh-agent-loop' - config: - agents: - - id: main - model: mock-echo - systemPrompt: 'You are echo-agent, a demo agent.' - +# The mock model (registers the `mock-echo` adapter) and the demo `echo` tool — +# example-local teaching plugins, resolved relative to THIS file's directory. - id: mock-llm name: './src/mock-llm.ts' - id: echo-tool name: './src/echo-tool.ts' -- id: session-persistence - name: '@deepseek-ai/dsh-session-persistence-jsonl' - config: - root: './.sessions' +# Local bash executor: agent-core ships the `tool-bash` consumer schema, so the +# leaf provides the executor it runs on (the echo demo doesn't drive bash, but +# the tool is part of the shared spine). +- id: bash + name: '@deepseek-ai/dsh-bash-local' -- id: stdio-chat - name: './src/stdio-chat.ts' +# The stdio chat app: console logger + the agent-core spine (pre-creating the +# `main` agent on the mock model) + JSONL persistence + the readline UI. +- id: stdio-agent + name: '@deepseek-ai/dsh-stdio-agent' + config: + model: mock-echo + systemPrompt: 'You are echo-agent, a demo agent.' + welcome: 'echo-agent ready. Type a message ("echo " triggers the tool).' + persistenceRoot: './.sessions' diff --git a/examples/echo-agent/src/stdio-chat.ts b/examples/echo-agent/src/stdio-chat.ts deleted file mode 100644 index e2242441d4..0000000000 --- a/examples/echo-agent/src/stdio-chat.ts +++ /dev/null @@ -1,64 +0,0 @@ -import { createInterface } from 'node:readline' -import type { Context } from 'cordis' -import type {} from '@deepseek-ai/dsh-agent' - -export const name = 'stdio-chat' -export const inject = ['agents'] - -/** - * Minimal UI plugin: reads lines from stdin → agent.send(); renders the - * agent's stream chunks and tool activity to stdout. Demonstrates that a UI - * is "just a plugin" — it only consumes the agent/* event taxonomy. - */ -export function apply(ctx: Context) { - ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => { - if (chunk.type === 'text-delta') process.stdout.write(chunk.text) - }) - - ctx.on('agent/turn-start', (agent, turn) => { - process.stdout.write(`\n[${agent.id} turn ${turn}] `) - }) - - ctx.on('agent/turn-end', () => { - process.stdout.write('\n> ') - }) - - ctx.on('session/event', (_session, event) => { - if (event.type === 'tool/call') { - const { name: toolName, arguments: args } = event.data - process.stdout.write(`\n [tool call] ${toolName}(${args})`) - } else if (event.type === 'tool/result') { - const { content } = event.data - const text = content.filter(b => b.type === 'text').map(b => b.text).join('') - process.stdout.write(`\n [tool result] ${text}\n `) - } - }) - - ctx.effect(() => { - const reader = createInterface({ - input: process.stdin, - output: process.stdout, - terminal: process.stdin.isTTY && process.stdout.isTTY, - }) - reader.on('line', (line) => { - const text = line.trim() - if (!text) return - const agent = ctx.agents.get('main') - if (!agent) { - console.error('agent "main" is not running') - return - } - if (agent.status === 'running') { - agent.steer([{ type: 'text', text }]) - } else { - agent.send([{ type: 'text', text }]) - } - }) - reader.on('close', () => { - // allow the process to exit when stdin ends (piped input) - setTimeout(() => process.exit(0), 200) - }) - process.stdout.write('echo-agent ready. Type a message ("echo " triggers the tool).\n> ') - return () => { reader.close() } - }, 'stdio-chat') -} diff --git a/examples/echo-agent/start.ts b/examples/echo-agent/start.ts deleted file mode 100644 index 90dba7b5b0..0000000000 --- a/examples/echo-agent/start.ts +++ /dev/null @@ -1,16 +0,0 @@ -import { pathToFileURL } from 'node:url' -import { Context } from 'cordis' -import Loader from '@cordisjs/plugin-loader' - -// Boot a Cordis app from this example's cordis.yml — the same shape as the -// upstream `cordis` bin, pinned to this directory. -const ctx = new Context() -ctx.baseUrl = pathToFileURL(import.meta.dirname).href + '/' - -await ctx.plugin(Loader) -await ctx.loader.create({ - name: '@cordisjs/plugin-include', - config: { - path: './cordis.yml', - }, -}) diff --git a/examples/echo-agent/tests/echo.e2e.ts b/examples/echo-agent/tests/echo.e2e.ts new file mode 100644 index 0000000000..944a36e429 --- /dev/null +++ b/examples/echo-agent/tests/echo.e2e.ts @@ -0,0 +1,113 @@ +import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' + +/** + * Keyless Loader-path smoke for examples/echo-agent: boot the REAL example + * through the `@deepseek-ai/dsh-stdio-agent` bin against this example's + * `cordis.yml` (the cordis Loader, `unwrapExports`, the whole plugin tree), + * pipe a script of stdin lines, and assert the rendered stdout. + * + * This is the guard the per-file unit suite structurally cannot be: it drives + * the `@deepseek-ai/dsh-stdio-agent` app plugin, the `@deepseek-ai/dsh-agent-core` + * bundle it loads, the extracted `@deepseek-ai/dsh-ui-stdio` plugin, AND the + * example-local `mock-llm.ts` / `echo-tool.ts` through their REAL load path, so + * a broken plugin export shape (a stray `export default` that `unwrapExports` + * would collapse, dropping `inject`/`Config`) fails here even though hand-mounted + * unit tests stay green (see docs/postmortem/0001). It needs no API key — the + * `mock-echo` adapter never touches the network — so it runs in the default e2e + * gate. + * + * Both branches of mock-llm.ts are exercised: an `echo …` line (the tool + * round-trip → `ECHO: …`) and a plain line (the direct canned reply). + */ + +// The dsh-stdio-agent bin (the demo:echo entry) and this example's cordis.yml. +// The bin resolves its config-path arg from CWD; the test spawns from a temp +// cwd, so we pass the example config's ABSOLUTE path. +const binScript = fileURLToPath(new URL('../../../packages/ui/stdio-agent/src/bin.ts', import.meta.url)) +const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) +const tsxLoader = fileURLToPath(import.meta.resolve('tsx')) +// Dev/test run UNBUILT: `@deepseek-ai/dsh-*` imports resolve through the root +// tsconfig `paths` map, which tsx finds by searching UP from cwd. We spawn from +// a temp cwd OUTSIDE the repo, so point tsx at the repo tsconfig explicitly +// (repo root is four levels up from examples/echo-agent/tests). +const repoTsconfig = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)) + +let child: ChildProcessWithoutNullStreams | undefined +let workdir: string | undefined + +afterEach(async () => { + if (child !== undefined && child.exitCode === null) child.kill('SIGKILL') + child = undefined + if (workdir !== undefined) await rm(workdir, { recursive: true, force: true }) + workdir = undefined +}) + +/** + * Boot echo-agent, write `lines` to its stdin, close stdin, and resolve with + * the full stdout once the process exits (the stdio UI exits on EOF after the + * agent settles). Rejects on a non-zero exit or a 10s timeout. + */ +async function runEcho(lines: string[]): Promise<{ stdout: string; code: number }> { + workdir = await mkdtemp(join(tmpdir(), 'echo-smoke-')) + const cwd = workdir + return new Promise((resolve, reject) => { + const proc = spawn( + process.execPath, + // --expose-internals: the example's cordis.yml loads the HMR plugin, which + // requires it (mirrors the `demo:echo` script). The whole point is to boot + // the example EXACTLY as it really runs, through the bin + Loader. + ['--expose-internals', '--import', tsxLoader, binScript, configPath], + { cwd, env: { ...process.env, TSX_TSCONFIG_PATH: repoTsconfig }, stdio: ['pipe', 'pipe', 'pipe'] }, + ) + child = proc + let stdout = '' + let stderr = '' + proc.stdout.setEncoding('utf8') + proc.stdout.on('data', (chunk: string) => { stdout += chunk }) + proc.stderr.setEncoding('utf8') + proc.stderr.on('data', (chunk: string) => { stderr += chunk }) + + const timer = setTimeout(() => { + proc.kill('SIGKILL') + reject(new Error(`echo-agent did not exit within 10s. stdout:\n${stdout}\nstderr:\n${stderr}`)) + }, 10_000) + + proc.on('exit', (code) => { + clearTimeout(timer) + if (code === 0) resolve({ stdout, code }) + else reject(new Error(`echo-agent exited ${code}. stderr:\n${stderr}`)) + }) + proc.on('error', (err) => { clearTimeout(timer); reject(err) }) + + // Feed the script, then EOF so the stdio UI exits after the agent settles. + for (const line of lines) proc.stdin.write(`${line}\n`) + proc.stdin.end() + }) +} + +describe('echo-agent keyless smoke (real cordis.yml via the Loader)', () => { + it('boots, prints its welcome banner, and exits cleanly on stdin EOF', async () => { + const { stdout, code } = await runEcho([]) + expect(code).toBe(0) + expect(stdout).toContain('echo-agent ready.') + }, 15_000) + + it('runs the echo tool round-trip for an "echo …" line', async () => { + const { stdout } = await runEcho(['echo hello world']) + // mock-llm.ts emits a tool-call for the echo tool; echo-tool.ts uppercases. + expect(stdout).toContain('[tool call] echo') + expect(stdout).toContain('[tool result] ECHO: HELLO WORLD') + }, 15_000) + + it('streams a direct canned reply for a non-echo line', async () => { + const { stdout } = await runEcho(['just chatting']) + // The direct-response branch of mock-llm.ts quotes the input back. + expect(stdout).toContain('just chatting') + expect(stdout).not.toContain('[tool call]') + }, 15_000) +}) diff --git a/examples/echo-agent/tests/stdio-chat.spec.ts b/examples/echo-agent/tests/stdio-chat.spec.ts deleted file mode 100644 index 4437e55c53..0000000000 --- a/examples/echo-agent/tests/stdio-chat.spec.ts +++ /dev/null @@ -1,33 +0,0 @@ -import { EventEmitter } from 'node:events' -import type { Context } from 'cordis' -import { describe, expect, test, vi } from 'vitest' - -const createInterface = vi.hoisted(() => vi.fn(() => { - const reader = new EventEmitter() as EventEmitter & { close(): void } - reader.close = vi.fn() - return reader -})) - -vi.mock('node:readline', () => ({ createInterface })) - -function fakeContext(): Context { - return { - agents: { get: vi.fn() }, - on: vi.fn(() => vi.fn()), - effect: vi.fn((callback: () => () => void) => callback()), - } as unknown as Context -} - -describe('echo-agent stdio chat', () => { - test('creates a terminal readline interface so TTY editing keys work', async () => { - const { apply } = await import('../src/stdio-chat.ts') - - apply(fakeContext()) - - expect(createInterface).toHaveBeenCalledWith({ - input: process.stdin, - output: process.stdout, - terminal: process.stdin.isTTY && process.stdout.isTTY, - }) - }) -}) diff --git a/knip.json b/knip.json index 189da8d695..b3ce7c1d4b 100644 --- a/knip.json +++ b/knip.json @@ -6,20 +6,34 @@ ".": { "entry": [ "examples/echo-agent/src/*.ts", - "examples/coding-agent/src/*.ts", - "examples/acp-agent/tests/**/*.e2e.ts" + "examples/echo-agent/tests/**/*.e2e.ts", + "examples/coding-agent/tests/**/*.e2e.ts", + "examples/acp-agent/tests/**/*.e2e.ts", + "examples/acp-agent/tests/**/*.snapshot.ts" ], "project": ["scripts/**/*.ts", "examples/**/*.ts"] }, - "packages/*": { + "packages/*/*": { "entry": ["tests/**/*.spec.ts"], "project": ["src/**/*.ts", "tests/**/*.ts"] }, - "packages/llm-deepseek": { + "packages/util/brand": { + "project": ["src/**/*.ts"], + "ignoreDependencies": ["cordis"] + }, + "packages/llm/llm-deepseek": { "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], "project": ["src/**/*.ts", "tests/**/*.ts"] }, - "packages/llm-pi-ai": { + "packages/llm/llm-pi-ai": { + "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], + "project": ["src/**/*.ts", "tests/**/*.ts"] + }, + "packages/ui/acp-agent": { + "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], + "project": ["src/**/*.ts", "tests/**/*.ts"] + }, + "packages/ui/stdio-agent": { "entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"], "project": ["src/**/*.ts", "tests/**/*.ts"] } diff --git a/lefthook.yml b/lefthook.yml index 24600e4985..2a255424fa 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -25,6 +25,9 @@ pre-push: - name: test run: pnpm run test + - name: snapshot + run: pnpm run test:snapshot + - name: hygiene run: pnpm run hygiene diff --git a/package.json b/package.json index 222bf3fb70..2e83cdca61 100644 --- a/package.json +++ b/package.json @@ -9,30 +9,39 @@ }, "workspaces": [ "vendor/*", - "packages/*" + "packages/*/*" ], "scripts": { "build": "tsc -b tsconfig.build.json && tsdown", - "typecheck": "tsc -b tsconfig.build.json && tsc -p tsconfig.typecheck.json", + "clean:build": "rm -rf .typecheck packages/*/*/lib vendor/*/lib *.tsbuildinfo", + "typecheck": "tsc -b tsconfig.json", "lint": "eslint .", "lint:fix": "eslint . --fix", "test": "vitest run", "test:coverage": "vitest run --coverage", "test:e2e": "vitest run --config vitest.e2e.config.ts", - "knip": "knip", + "test:snapshot": "vitest run --config vitest.snapshot.config.ts", + "test:snapshot:record": "DSH_SNAPSHOT=record vitest run --config vitest.snapshot.config.ts --update", + "knip": "knip --treat-config-hints-as-errors", "publint": "tsx scripts/publint-all.ts", "doc-typecheck": "tsx scripts/doc-typecheck.ts", - "verify-event-taxonomy": "tsx scripts/verify-event-taxonomy.ts", "verify-md-wrap": "tsx scripts/verify-md-wrap.ts", "verify-md-links": "tsx scripts/verify-md-links.ts", + "verify-doc-refs": "tsx scripts/verify-doc-refs.ts", + "verify-package-paths": "tsx scripts/verify-package-paths.ts", + "verify-rfc-classification": "tsx scripts/verify-rfc-classification.ts", + "verify-type-equiv": "tsx scripts/verify-type-equiv.ts", + "verify-node-next-types": "tsx scripts/verify-node-next-types.ts", + "gen-cordis-catalog": "tsx scripts/gen-cordis-catalog.ts", + "verify-cordis-catalog": "tsx scripts/gen-cordis-catalog.ts --check", "gen-module-graph": "tsx scripts/gen-module-graph.ts", "verify-module-graph": "tsx scripts/gen-module-graph.ts --check", "constraints": "tsx scripts/check-workspace-constraints.ts", - "doc-sync": "pnpm run doc-typecheck && pnpm run verify-event-taxonomy && pnpm run verify-md-wrap && pnpm run verify-md-links", - "hygiene": "pnpm run knip && pnpm run publint && pnpm run constraints", - "demo:echo": "node --expose-internals --import tsx examples/echo-agent/start.ts", - "demo:coding": "node --expose-internals --import tsx examples/coding-agent/start.ts", - "demo:acp": "node --expose-internals --import tsx examples/acp-agent/start.ts", + "doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-rfc-classification && pnpm run verify-type-equiv", + "hygiene": "pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types", + "demo:echo": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/echo-agent/cordis.yml", + "demo:coding": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/coding-agent/cordis.yml", + "demo:acp": "node --import tsx packages/ui/acp-agent/src/bin.ts examples/acp-agent/cordis.yml", "postinstall": "node scripts/install-lefthook.mjs" }, "devDependencies": { diff --git a/packages/AGENTS.md b/packages/AGENTS.md index 31f42c26c9..62d37a3354 100644 --- a/packages/AGENTS.md +++ b/packages/AGENTS.md @@ -7,12 +7,12 @@ This directory contains all `@deepseek-ai/dsh-*` harness packages. When editing - **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)`; call `next()` to delegate, or return without it to short-circuit (veto). Never call `next()` after returning. - **Plugin export shape — namespace OR default, never both.** A *service* package exports the service class as `export default` (the Loader instantiates it). A *function/namespace* plugin exports `name` / `inject` / `Config` / `apply` as separate named exports and **must NOT add `export default`** — the cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray default export collapses the module to the bare `apply` function and silently discards the `inject`/`name`/`Config` namespace, leaving the plugin with no injected services (it then throws `cannot get property … without inject` at load). See [docs/postmortem/0001](../docs/postmortem/0001-acp-default-export-drops-inject.md). - **Read an optional (non-injected) service via `ctx.get(name)`, not `ctx.`.** For a service a plugin reads opportunistically but deliberately leaves out of `static inject` (e.g. `AgentLoop` reading `sessionPersistence`), the `ctx.` property proxy resolves by an ancestor-only fiber walk that throws when the call arrives through a foreign traceable shadow (the service lives on a sibling fiber). `ctx.get(name)` is the topology-independent global-store lookup, strict by default (an inactive/absent backend reads as `undefined` — prefer it over the `ctx.get(name, false)` overload, which also skips the active-state check). Services that ARE in `static inject` resolve fine via `ctx.`. See [docs/postmortem/0001](../docs/postmortem/0001-acp-default-export-drops-inject.md). -- **Tests**: vitest in `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test (register a plugin, dispose its fiber, assert cleanup). Err on the side of more tests — edge cases, error paths, event ordering, races. A plugin shipped via `cordis.yml` also needs at least one test that drives it through the REAL Loader/export path (hand-built `ctx.plugin({...})` mounts bypass `unwrapExports` and cannot catch a broken export shape) — see AGENTS.md § Defensive patterns "Line coverage is not behavior coverage". Real-API (with-key) e2e tests are cheap here (we are DeepSeek) and welcome — write many, especially smoke tests; see AGENTS.md § Secrets / .env. +- **Tests**: vitest in `packages///tests/*.spec.ts`. Every registry needs an HMR-safety test (register a plugin, dispose its fiber, assert cleanup). Err on the side of more tests — edge cases, error paths, event ordering, races. A plugin shipped via `cordis.yml` also needs at least one test that drives it through the REAL Loader/export path (hand-built `ctx.plugin({...})` mounts bypass `unwrapExports` and cannot catch a broken export shape) — see AGENTS.md § Defensive patterns "Line coverage is not behavior coverage". Real-API (with-key) e2e tests are cheap here (we are DeepSeek) and welcome — write many, especially smoke tests; see AGENTS.md § Secrets / .env. Naming notes: - A *service* `src/index.ts` exports the service class as `export default` + all public types; a *function/namespace plugin* `src/index.ts` exports `name`/`inject`/`Config`/`apply` as named exports and NO default (see the plugin-export-shape rule above) - `src/types.ts` contain only types — no runtime code - Tests live at package level under `tests/`, not `src/__tests__/` -- A package's README and module/JSDoc comments are part of the change: when you alter behavior (config keys, defaults, error codes, wire fields), update them in the same commit. CI runs `pnpm run doc-sync`, which typechecks fenced `ts` blocks in `packages/*/README.md`, verifies the event-taxonomy table, and checks markdown wrapping across this file too — but it does NOT catch prose drift (config keys, defaults, error codes), so those stay on the author. +- A package's README and module/JSDoc comments are part of the change: when you alter behavior (config keys, defaults, error codes, wire fields), update them in the same commit. CI runs `pnpm run doc-sync`, which typechecks fenced `ts` blocks in `packages/*/*.md` and `packages/*/*/*.md`, regenerates the cordis events/services catalog from the `interface Events` / `interface Context` declarations (failing if the committed copy is stale), and checks markdown wrapping across this file too — but it does NOT catch prose drift (config keys, defaults, error codes), so those stay on the author. A new event needs an `@mode` tag on its JSDoc (the catalog generator hard-errors without it — see the root AGENTS.md). Read the per-package README.md for package-specific details: service API, events, extension points, TODOs. diff --git a/packages/README.md b/packages/README.md index 8d4c81286c..1fcc0c44d0 100644 --- a/packages/README.md +++ b/packages/README.md @@ -2,14 +2,31 @@ Harness packages, all under the `@deepseek-ai/dsh-*` scope. Each package is a Cordis plugin (microkernel-style): it exports either a default `Service` subclass or a functional plugin that gets registered via `ctx.plugin()`, declares its ctx key/events where applicable through declaration merging, and exposes extension points through `ctx.effect()`, `ctx.on()`, and `ctx.waterfall()`. +## Hierarchy + +Packages are grouped by modular role at `packages///`. The group directory is a pure container (no `package.json` of its own); the package name stays `@deepseek-ai/dsh-` regardless of group. Each group has a `README.md` describing its role and whether it is product or support infrastructure. + +| Group | Role | Release expectation | +|---|---|---| +| [`core/`](core/README.md) | Product API spine: session, system-prompt, tools, agent, and the concrete loop | Product — stable surface | +| [`llm/`](llm/README.md) | LLM capability family: the abstract service + provider adapters | Product — stable surface | +| [`bash/`](bash/README.md) | Bash capability family: the executor seam, a local impl, and the model-facing tool | Product — stable surface | +| [`session-persistence/`](session-persistence/README.md) | Persistence capability family: the seam + JSONL/SQLite backends | Product — stable surface | +| [`ui/`](ui/README.md) | Editor/client integration surfaces (the ACP bridge) | Product — stable surface | +| [`support/`](support/README.md) | Dev/test/example infrastructure (invariants, stdio UI, replay adapter) | Support — lower compatibility expectations | +| [`util/`](util/README.md) | Low-level zero-dependency utilities shared across groups (the `Branded` primitive) | Support — small, stable, harness-dep-free | + +The split is the point: a package's group says whether it is part of the product API or support/test/example infrastructure, so release and removal decisions do not have to treat every package as an equal public contract. New packages join an existing group; adding a new top-level group is a deliberate act (extend the group READMEs and the hierarchy docs). + ## Dependency graph ``` -dsh-llm (no harness deps — pure vocabulary) -dsh-bash (no harness deps — abstract executor seam) -dsh-session ← dsh-llm +dsh-brand (no harness deps — type-only Branded primitive) +dsh-llm ← dsh-brand (vocabulary; brands CallId) +dsh-bash ← dsh-brand (abstract executor seam; brands BashTaskId/OwnerToken) +dsh-session ← dsh-llm, dsh-brand dsh-system-prompt ← dsh-llm -dsh-agent ← dsh-llm, dsh-session +dsh-agent ← dsh-llm, dsh-session, dsh-brand dsh-tools ← dsh-llm, dsh-system-prompt, dsh-agent dsh-bash-local ← dsh-bash (BashExecutor impl) dsh-tool-bash ← dsh-bash, dsh-tools (bash tool schemas) @@ -18,27 +35,41 @@ dsh-llm-pi-ai ← dsh-llm (pi-ai-backed adapter) dsh-agent-loop ← dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent dsh-invariants ← dsh-llm, dsh-session, dsh-agent (dev-mode contract checks) dsh-acp ← dsh-agent, dsh-llm, dsh-session, dsh-session-persistence (ACP JSON-RPC bridge) +dsh-ui-stdio ← dsh-agent, dsh-llm, dsh-session (stdio readline UI plugin) +dsh-llm-replay ← dsh-llm, dsh-session (record/replay adapter for keyless snapshot tests) +dsh-agent-core ← timer, dsh-llm, dsh-session, dsh-system-prompt, dsh-tools, dsh-agent, dsh-invariants, dsh-tool-bash, dsh-agent-loop (the providerless spine, as one bundle plugin) +dsh-stdio-agent ← dsh-agent-core, dsh-ui-stdio, dsh-session-persistence-jsonl, dsh-agent, dsh-session (stdio chat APP + bin) +dsh-acp-agent ← dsh-agent-core, dsh-acp, dsh-session-persistence-jsonl (ACP server APP + bin) ``` -The rule: plugins depend on interfaces, never on the concrete loop. `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/2026-06-13-capability-seams.md)). +The rule: **extension** plugins depend on interfaces, never on the concrete loop. `dsh-agent-loop` is swappable — UI/hook/tool plugins keep working against the `dsh-agent` vocabulary if the loop is replaced. The sanctioned exception is a **composition/bundle** package like `dsh-agent-core`, whose whole job is to assemble the concrete spine: it depends on `dsh-agent-loop` (and the other concrete spine plugins) on purpose. The rule constrains plugins that EXTEND the system, not the bundle that COMPOSES it — swapping the loop means shipping a different bundle, not rewiring every extension. A swappable capability splits into interface / implementation / consumer packages (the bash trio is the template — see [capability seams](../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)). ## What goes where -| Package | Role | ctx key | -|---|---|---| -| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` | -| `session/` | Event-sourced session log + in-memory store | `ctx.sessions` | -| `system-prompt/` | Prompt-section + tool-schema assembly registry | `ctx.systemPrompt` | -| `tools/` | Tool registry + `tools/execute` waterfall | `ctx.tools` | -| `agent/` | Agent interface, registry, `agent/*` event vocabulary | `ctx.agents` | -| `agent-loop/` | THE concrete loop plugin: `LoopAgent` + the loop driver | `ctx.agentLoop` | -| `bash/` | Abstract bash executor seam (interface + vocabulary) | `ctx.bash` | -| `bash-local/` | Local-subprocess `BashExecutor` implementation | (registers `ctx.bash`) | -| `tool-bash/` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) | -| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | -| `llm-pi-ai/` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) | -| `invariants/` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) | -| `acp/` | Agent Client Protocol bridge: serves the agent to an ACP editor over JSON-RPC stdio | (drives `ctx.agents`/`ctx.sessions`) | +| Package | Group | Role | ctx key | +|---|---|---|---| +| `llm/` | `llm` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` | +| `session/` | `core` | Event-sourced session log + in-memory store | `ctx.sessions` | +| `system-prompt/` | `core` | Prompt-section + tool-schema assembly registry | `ctx.systemPrompt` | +| `tools/` | `core` | Tool registry + `tools/execute` waterfall | `ctx.tools` | +| `agent/` | `core` | Agent interface, registry, `agent/*` event vocabulary | `ctx.agents` | +| `agent-loop/` | `core` | THE concrete loop plugin: `ReactLoopAgent` + the loop driver | `ctx.agentLoop` | +| `agent-core/` | `core` | Bundle plugin: the providerless/executor-less/UI-less spine as code (forwards `agent-loop`'s `agents`) | (loads the spine) | +| `bash/` | `bash` | Abstract bash executor seam (interface + vocabulary) | `ctx.bash` | +| `bash-local/` | `bash` | Local-subprocess `BashExecutor` implementation | (registers `ctx.bash`) | +| `tool-bash/` | `bash` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) | +| `llm-deepseek/` | `llm` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | +| `llm-pi-ai/` | `llm` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) | +| `session-persistence/` | `session-persistence` | Persistence seam + write coordinator | `ctx.sessionPersistence` | +| `session-persistence-jsonl/` | `session-persistence` | JSONL-sidecar persistence backend | (registers `ctx.sessionPersistence`) | +| `session-persistence-sqlite/` | `session-persistence` | SQLite persistence backend | (registers `ctx.sessionPersistence`) | +| `invariants/` | `support` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) | +| `acp/` | `ui` | Agent Client Protocol bridge: serves the agent to an ACP editor over JSON-RPC stdio | (drives `ctx.agents`/`ctx.sessions`) | +| `stdio-agent/` | `ui` | Terminal stdio chat APP: agent-core spine + console logger + readline UI + a pre-created `main` agent, with a `bin` | (composition + `bin`) | +| `acp-agent/` | `ui` | ACP server APP: agent-core spine + JSONL persistence + the `acp` bridge (no stdout logger), with a `bin` | (composition + `bin`) | +| `ui-stdio/` | `support` | Minimal stdio (readline) UI plugin: renders `agent/*` events, feeds stdin lines to the agent | (drives `ctx.agents`) | +| `llm-replay/` | `support` | Record/replay adapter: short-circuits `llm/stream` with chunks from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) | +| `brand/` | `util` | Type-only `Branded` nominal-typing primitive (no runtime code, no harness deps) | (none — type-only) | Each package has its own `README.md` with purpose, service API, events, extension points, and deliberate non-goals (TODOs). @@ -48,5 +79,5 @@ Each package has its own `README.md` with purpose, service API, events, extensio - **Declaration merging for events and ctx**: services declare their events in `declare module 'cordis' { interface Events { ... } }` and their ctx key in `interface Context`. - **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)` and MUST call `next()` to delegate; returning without it short-circuits (the veto mechanism). - **Extensible unions**: `ContentBlockMap`, `MessageSourceMap`, `FinishReasonMap`, `TurnTriggerMap`, `TurnEndReasonMap`, and `SessionEventMap` use the merge-extensible-map pattern so plugins can add variants via declaration merging. -- **ESM everywhere**; imports use package names across package boundaries, `.ts` extensions within a package. -- **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test. Err on the side of more tests. +- **ESM everywhere**; imports use package names across package boundaries and explicit `.ts` relative specifiers within a package. +- **Tests**: vitest, colocated under `packages///tests/*.spec.ts`. Every registry needs an HMR-safety test. Err on the side of more tests. diff --git a/packages/acp/tests/dispose.spec.ts b/packages/acp/tests/dispose.spec.ts deleted file mode 100644 index 45e351ced5..0000000000 --- a/packages/acp/tests/dispose.spec.ts +++ /dev/null @@ -1,143 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { mkdtemp, rm } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' -import { makeBridgeHarness } from './harness.ts' - -describe('acp bridge — disposal & HMR safety', () => { - let storageDir: string - - beforeEach(async () => { storageDir = await mkdtemp(join(tmpdir(), 'acp-dispose-')) }) - afterEach(async () => { await rm(storageDir, { recursive: true, force: true }) }) - - it('disposal reaches quiescence: a running turn is aborted and awaited before dispose returns', async () => { - const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) - const agent = harness.ctx.agents.get(sessionId)! - - // Start a prompt that hangs in the model stream. - const promptDone = harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }) - await new Promise(r => setTimeout(r, 30)) - expect(agent.status).toBe('running') - - // Dispose the whole context. The bridge's teardown must abort the agent and - // AWAIT whenIdle() — so right after dispose resolves, the agent is settled - // (not still running). Proves disposal waited, not just requested. - await harness.ctx.fiber.dispose() - expect(agent.status).not.toBe('running') - - // The in-flight prompt settled (cancelled) rather than hanging forever. - const res = await promptDone - expect(res.stopReason).toBe('cancelled') - }) - - it('after an ACP-only HMR dispose, a late session/new creates no orphan agent (closed guard)', async () => { - // Dispose JUST the bridge's fiber (an HMR reload) while agents/agent-loop - // stay up and the transport is still live. A late session/new must hit the - // `closed` guard and reject — NOT create an agent the disposed bridge can no - // longer stream or settle. Verify the world: no agent appeared. - const harness = await makeBridgeHarness({ storageDir, script: [] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const before = harness.ctx.agents.list().length - await harness.acpFiber.dispose() // tear down ONLY the bridge - await expect(harness.client.newSession({ cwd: process.cwd(), mcpServers: [] })) - .rejects.toThrow(/disposed/) - expect(harness.ctx.agents.list().length).toBe(before) - await harness.dispose() - }) - - it('an agent created through the bridge is unregistered when ONLY the bridge fiber is disposed', async () => { - // The factory (`ctx.agents.create`) is reached through the bridge's - // traceable service proxy, so `AgentLoop.start`'s `this.ctx.effect(...)` - // registration binds to the CALLER context — the bridge fiber — not the - // AgentLoop fiber. Disposing JUST the bridge fiber (an ACP-only HMR reload) - // must therefore reclaim the agent's registry entry, even though agents/ - // agent-loop stay up. This pins the fiber-ownership the bridge's teardown - // doc comment relies on; if a refactor rebinds the registration to the - // AgentLoop fiber, the agent would survive bridge dispose and this fails. - const harness = await makeBridgeHarness({ storageDir, script: [] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) - expect(harness.ctx.agents.get(sessionId)).toBeDefined() - - await harness.acpFiber.dispose() // tear down ONLY the bridge - expect(harness.ctx.agents.get(sessionId)).toBeUndefined() - await harness.dispose() - }) - - it('no agent is created by a session/new after the bridge has closed (closed guard)', async () => { - // After teardown (here a client disconnect sets `closed`), a late - // `session/new` must NOT create an orphan agent the bridge can no longer - // drive/settle. The transport is gone so the RPC rejects; assert the world: - // no new agent appeared in the registry. - const harness = await makeBridgeHarness({ storageDir, script: [] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const before = harness.ctx.agents.list().length - await harness.closeClientTransport() // teardown → closed = true - await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }).catch(() => {}) - await new Promise(r => setTimeout(r, 10)) - expect(harness.ctx.agents.list().length).toBe(before) - await harness.dispose() - }) - - it('a client disconnect mid-prompt tears the session down to quiescence', async () => { - // The ACP transport closes (editor quits) while a turn runs. The bridge must - // settle the in-flight prompt cancelled and abort+drain the agent rather - // than leaving an orphaned running agent whose updates are swallowed. - const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) - const agent = harness.ctx.agents.get(sessionId)! - // Start a prompt that hangs in the model stream. The prompt RPC will never - // return (its transport is severed), so do not await it. - void harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }).catch(() => {}) - await new Promise(r => setTimeout(r, 30)) - expect(agent.status).toBe('running') - - // Sever the transport — the bridge's conn.closed teardown runs and drives - // the agent to quiescence on its OWN (assert before any dispose() runs). - await harness.closeClientTransport() - await agent.whenIdle() - expect(agent.status).toBe('idle') - - await harness.dispose() // idempotent with the close teardown - }) - - it('a client disconnect racing fiber dispose both reach quiescence (shared teardown)', async () => { - // conn.closed teardown and ctx.fiber.dispose() can fire near-simultaneously. - // They must share one teardown promise: dispose() must NOT return before the - // disconnect teardown's whenIdle() has settled (a `record === undefined`-only - // guard would let the second caller return early mid-teardown). - const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) - const agent = harness.ctx.agents.get(sessionId)! - void harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }).catch(() => {}) - await new Promise(r => setTimeout(r, 30)) - expect(agent.status).toBe('running') - - // Fire both teardown paths without awaiting the first, then await both. - const close = harness.closeClientTransport() - const dispose = harness.ctx.fiber.dispose() - await Promise.all([close, dispose]) - // After BOTH settle, the agent has fully drained (not still running). - expect(agent.status).not.toBe('running') - }) - - it('after dispose, session/update listeners are gone (no further updates emitted)', async () => { - const harness = await makeBridgeHarness({ storageDir, script: [] }) - await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) - const session = harness.ctx.agents.get(sessionId)!.session - - await harness.ctx.fiber.dispose() - const before = harness.updates.length - // Append an event directly to the (now-detached) session: the bridge's - // session/event listener should have been disposed, so no update fires. - session.append('turn/start', { turn: 99, trigger: { kind: 'message', source: { kind: 'user' } } }) - await new Promise(r => setTimeout(r, 10)) - expect(harness.updates.length).toBe(before) - }) -}) diff --git a/packages/acp/tsconfig.json b/packages/acp/tsconfig.json deleted file mode 100644 index 83330256e3..0000000000 --- a/packages/acp/tsconfig.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../llm" }, - { "path": "../session" }, - { "path": "../agent" }, - { "path": "../tools" }, - { "path": "../session-persistence" } - ] -} diff --git a/packages/agent-loop/tsconfig.json b/packages/agent-loop/tsconfig.json deleted file mode 100644 index 6751664d5c..0000000000 --- a/packages/agent-loop/tsconfig.json +++ /dev/null @@ -1,19 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../llm" }, - { "path": "../session" }, - { "path": "../session-persistence" }, - { "path": "../system-prompt" }, - { "path": "../tools" }, - { "path": "../agent" } - ] -} diff --git a/packages/agent/tsconfig.json b/packages/agent/tsconfig.json deleted file mode 100644 index 0806132292..0000000000 --- a/packages/agent/tsconfig.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../llm" }, - { "path": "../session" } - ] -} diff --git a/packages/bash-local/tsconfig.json b/packages/bash-local/tsconfig.json deleted file mode 100644 index a657d8bf8e..0000000000 --- a/packages/bash-local/tsconfig.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../bash" } - ] -} diff --git a/packages/bash/README.md b/packages/bash/README.md index 8de529fee1..9a9dba88d5 100644 --- a/packages/bash/README.md +++ b/packages/bash/README.md @@ -1,30 +1,11 @@ -# @deepseek-ai/dsh-bash +# bash/ — bash capability family -The **bash executor seam**: an abstract `BashExecutor` service (`ctx.bash`) defining WHAT a bash backend does — run commands, manage background tasks — without saying HOW. +The canonical three-package capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract executor interface, a concrete local implementation, and the model-facing tool that consumes it. All **product** packages. -This package is one third of the bash capability, split so each concern can evolve (and be swapped) independently: +| Package | Role | ctx key | +|---|---|---| +| `bash/` | Abstract bash executor seam (interface + vocabulary) | `ctx.bash` | +| `bash-local/` | Local-subprocess `BashExecutor` implementation | (registers `ctx.bash`) | +| `tool-bash/` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) | -| Package | Role | -|---|---| -| `@deepseek-ai/dsh-bash` (this) | the interface: abstract service + vocabulary types | -| `@deepseek-ai/dsh-bash-local` | an implementation: local subprocesses | -| `@deepseek-ai/dsh-tool-bash` | the model-facing tool schemas over `ctx.bash` | - -The split mirrors the LLM seam (`LlmService`/`LlmAdapter`) and the agent-tool survey: pi hides execution behind a `BashOperations` interface (local shell / SSH / VM backends), Codex behind an exec-server protocol. A future sandboxed, containerized, or remote executor implements this interface and the tool schemas don't change. - -## Service API (`ctx.bash`) - -| Member | Semantics | -|---|---| -| `run(spec)` | Foreground execution. Resolves when the command finishes. **Rejects only for infrastructure failures** (unusable workdir, missing shell, pre-aborted signal); nonzero exits, timeout kills, and abort kills resolve with a descriptive `BashRunResult`. | -| `start(spec)` | Background execution. Returns a `BashTask` handle immediately; **no timeout applies** (stop tasks via `kill`). | -| `get(id)` / `list()` | Task lookup. | -| `readOutput(id)` | **Incremental** output read — consecutive reads never re-deliver. Reads that lost data to buffer bounds flag `lossy` and point at full-stream spill files. Throws for unknown ids. | -| `kill(id)` | Kill a running task. Returns `false` when it already finished; throws for unknown ids. | -| `onTaskDone(listener)` | Completion listener (effect-based, disposer returned). Fires exactly once per task; never after the service is disposed. | - -Implementations subclass `BashExecutor`, implement the abstract methods, and call `notifyTaskDone(task)` on background completion. Disposal must kill every running task (no orphan processes) — see the HMR-safety tests. - -## Vocabulary - -`BashExecRequest` (command, workdir?, timeoutMs?, signal?) resolves to `BashExecSpec` (command, workdir, timeoutMs, signal?) before execution; `run()` returns `BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr as `CollectedOutput`) and `start()`/`readOutput()` use `BashTask`/`BashTaskRead` for the background side. See `src/types.ts` for the full contracts. +The interface lives at `bash/bash/`. A sandboxed executor would replace `bash-local` without touching the interface or the tool — the split is what makes that possible. diff --git a/packages/bash-local/README.md b/packages/bash/bash-local/README.md similarity index 90% rename from packages/bash-local/README.md rename to packages/bash/bash-local/README.md index d8062bc6ff..016f57d2a9 100644 --- a/packages/bash-local/README.md +++ b/packages/bash/bash-local/README.md @@ -22,7 +22,7 @@ Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi; - **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after a 3s grace (OpenCode's escalation; pipelines and subshells die with the parent). ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools. - **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. If the final spill close reports a delayed writeback failure, the executor still returns the tail but withholds the path rather than advertising a possibly incomplete file. - **Model-friendly env** — `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` (Codex's hardcoded set) so pagers and ANSI color don't garble results. -- **Background tasks** — `start()` returns immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), `readOutput()` is incremental with whole-stream byte offsets, and disposal kills everything. +- **Background tasks** — `start()` returns immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), `readOutput()` is incremental with whole-stream byte offsets, and disposal kills everything. The spec's opaque `owner` token is stored on the tracked task and returned by `ownerOf(id)` — the executor never interprets it (the consumer's access policy does), and because it lives with the task here it survives a `tool-bash` HMR reload. ## Sandboxing diff --git a/packages/bash-local/package.json b/packages/bash/bash-local/package.json similarity index 81% rename from packages/bash-local/package.json rename to packages/bash/bash-local/package.json index b786c1bc37..bc1dc7eb40 100644 --- a/packages/bash-local/package.json +++ b/packages/bash/bash-local/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/bash-local/src/index.ts b/packages/bash/bash-local/src/index.ts similarity index 89% rename from packages/bash-local/src/index.ts rename to packages/bash/bash-local/src/index.ts index 7320276a1a..05f1ed75dd 100644 --- a/packages/bash-local/src/index.ts +++ b/packages/bash/bash-local/src/index.ts @@ -15,8 +15,8 @@ import { Context } from 'cordis' import z from 'schemastery' -import { BashExecutor } from '@deepseek-ai/dsh-bash' -import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead } from '@deepseek-ai/dsh-bash' +import { BashExecutor, BashTaskId } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash' import { runBash } from './run.ts' import type { RunInternals, RunningBash } from './run.ts' @@ -49,6 +49,8 @@ interface TrackedTask extends BashTask { /** Whole-stream byte offsets already delivered via {@link LocalBashExecutor.readOutput}. */ stdoutOffset: number stderrOffset: number + /** Opaque owner token from the {@link BashExecSpec} (the consumer's isolation key). */ + owner: OwnerToken | undefined } /** @@ -65,7 +67,7 @@ export class LocalBashExecutor extends BashExecutor { maxOutputBytes: z.number().default(64_000), }) - private tasks = new Map() + private tasks = new Map() private nextTaskId = 1 /** Test seam: timer/spill knobs forwarded to runBash. */ internals: RunInternals = {} @@ -114,6 +116,9 @@ export class LocalBashExecutor extends BashExecutor { workdir: request.workdir ?? this.config.cwd ?? process.cwd(), timeoutMs, ...request.signal ? { signal: request.signal } : {}, + // Carry the owner through verbatim (required-but-nullable on the spec): + // the executor never interprets it — the consumer's access policy does. + owner: request.owner, } } @@ -142,13 +147,14 @@ export class LocalBashExecutor extends BashExecutor { signal: spec.signal, }, this.internals) - const id = `bash-${this.nextTaskId++}` + const id = BashTaskId(`bash-${this.nextTaskId++}`) const task: TrackedTask = { id, command: spec.command, status: 'running', exitCode: null, signal: null, + owner: spec.owner, running, stdoutOffset: 0, stderrOffset: 0, @@ -170,15 +176,21 @@ export class LocalBashExecutor extends BashExecutor { return task } - get(id: string): BashTask | undefined { + get(id: BashTaskId): BashTask | undefined { return this.tasks.get(id) } + ownerOf(id: BashTaskId): OwnerToken | undefined { + // Unknown id and known-but-ownerless both read as undefined — the consumer + // treats undefined as "open" and a truly unknown id fails at readOutput/kill. + return this.tasks.get(id)?.owner + } + list(): BashTask[] { return [...this.tasks.values()] } - readOutput(id: string): BashTaskRead { + readOutput(id: BashTaskId): BashTaskRead { const task = this.tasks.get(id) if (!task) throw new Error(`unknown bash task "${id}"`) @@ -201,7 +213,7 @@ export class LocalBashExecutor extends BashExecutor { } } - kill(id: string): boolean { + kill(id: BashTaskId): boolean { const task = this.tasks.get(id) if (!task) throw new Error(`unknown bash task "${id}"`) if (task.status !== 'running') return false diff --git a/packages/bash-local/src/run.ts b/packages/bash/bash-local/src/run.ts similarity index 96% rename from packages/bash-local/src/run.ts rename to packages/bash/bash-local/src/run.ts index 46d48cf5fd..8a8d2065d2 100644 --- a/packages/bash-local/src/run.ts +++ b/packages/bash/bash-local/src/run.ts @@ -159,7 +159,11 @@ export class OutputCollector { writeSync(this.spillFd, chunk) } - /** Read the collected tail without finalizing (used by background polling). */ + // TODO(snapshot-scope): `snapshot()` has one internal caller (`finalize()` at + // the bottom of this file) and `totalBytes` is read only by a test. The live + // background-poll path goes through `readFrom()`, so inline snapshot() into + // finalize() and drop or privatize the totalBytes getter. + /** Read the collected tail without finalizing (the final-result snapshot). */ snapshot(): CollectedOutput { return { text: Buffer.concat(this.chunks).toString('utf8'), diff --git a/packages/bash-local/tests/executor.spec.ts b/packages/bash/bash-local/tests/executor.spec.ts similarity index 92% rename from packages/bash-local/tests/executor.spec.ts rename to packages/bash/bash-local/tests/executor.spec.ts index 3dd7f7983a..2a27f29731 100644 --- a/packages/bash-local/tests/executor.spec.ts +++ b/packages/bash/bash-local/tests/executor.spec.ts @@ -4,7 +4,8 @@ import { join } from 'node:path' import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' -import type {} from '@deepseek-ai/dsh-bash' +import { BashTaskId } from '@deepseek-ai/dsh-bash' +import type { BashTaskRead } from '@deepseek-ai/dsh-bash' const spillDir = mkdtempSync(join(tmpdir(), 'dsh-bash-exec-spec-')) @@ -30,6 +31,22 @@ async function waitGone(pid: number, timeoutMs = 5_000): Promise { throw new Error(`pid ${pid} still alive after ${timeoutMs}ms`) } +async function readUntil( + bash: LocalBashExecutor, + id: BashTaskId, + expected: string, + timeoutMs = 5_000, +): Promise { + const deadline = Date.now() + timeoutMs + let last: BashTaskRead | undefined + while (Date.now() < deadline) { + last = bash.readOutput(id) + if (last.delta.includes(expected)) return last + await new Promise(resolve => setTimeout(resolve, 20)) + } + throw new Error(`task ${id} output did not include ${JSON.stringify(expected)}; last delta was ${JSON.stringify(last?.delta ?? '')}`) +} + describe('LocalBashExecutor.run', () => { it('resolves with output and the effective timeout', async () => { const { bash } = await setup({ timeoutMs: 5_000 }) @@ -116,9 +133,8 @@ describe('LocalBashExecutor background tasks', () => { it('readOutput returns increments without re-delivery', async () => { const { bash } = await setup() - const task = bash.start(bash.resolve({ command: 'echo first; sleep 0.3; echo second' })) - await new Promise(resolve => setTimeout(resolve, 150)) - const first = bash.readOutput(task.id) + const task = bash.start(bash.resolve({ command: 'echo first; sleep 1; echo second' })) + const first = await readUntil(bash, task.id, 'first\n') expect(first.delta).toBe('first\n') expect(first.lossy).toBe(false) await task.done @@ -155,7 +171,7 @@ describe('LocalBashExecutor background tasks', () => { it('readOutput throws for unknown ids', async () => { const { bash } = await setup() - expect(() => bash.readOutput('nope')).toThrow(/unknown bash task "nope"/) + expect(() => bash.readOutput(BashTaskId('nope'))).toThrow(/unknown bash task "nope"/) }) it('kill terminates the process group and reports status killed', async () => { @@ -172,7 +188,7 @@ describe('LocalBashExecutor background tasks', () => { const task = bash.start(bash.resolve({ command: 'true' })) await task.done expect(bash.kill(task.id)).toBe(false) - expect(() => bash.kill('nope')).toThrow(/unknown bash task "nope"/) + expect(() => bash.kill(BashTaskId('nope'))).toThrow(/unknown bash task "nope"/) }) it('notifies onTaskDone listeners on completion', async () => { diff --git a/packages/bash-local/tests/run.spec.ts b/packages/bash/bash-local/tests/run.spec.ts similarity index 94% rename from packages/bash-local/tests/run.spec.ts rename to packages/bash/bash-local/tests/run.spec.ts index ff1d92b519..b770c4a6c1 100644 --- a/packages/bash-local/tests/run.spec.ts +++ b/packages/bash/bash-local/tests/run.spec.ts @@ -3,6 +3,7 @@ import { tmpdir } from 'node:os' import { dirname, join } from 'node:path' import { describe, expect, it, vi } from 'vitest' import { killGroup, OutputCollector, runBash } from '@deepseek-ai/dsh-bash-local' +import type { RunningBash } from '@deepseek-ai/dsh-bash-local' const { failNextClose } = vi.hoisted(() => ({ failNextClose: { value: false } })) vi.mock('node:fs', async (importOriginal) => { @@ -45,6 +46,15 @@ async function waitGone(pid: number, timeoutMs = 5_000): Promise { throw new Error(`pid ${pid} still alive after ${timeoutMs}ms`) } +async function waitForStdout(running: RunningBash, expected: string, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs + while (Date.now() < deadline) { + if (running.stdout.snapshot().text.includes(expected)) return + await new Promise(resolve => setTimeout(resolve, 20)) + } + throw new Error(`stdout did not include ${JSON.stringify(expected)} after ${timeoutMs}ms`) +} + describe('runBash', () => { it('captures stdout on success', async () => { const result = await runBash(spec('echo hello')).done @@ -96,11 +106,10 @@ describe('runBash', () => { }) it('escalates to SIGKILL when SIGTERM is trapped', async () => { - const result = await runBash( - spec('trap \'\' TERM; sleep 60', { timeoutMs: 100 }), - { graceMs: 200 }, - ).done - expect(result.timedOut).toBe(true) + const running = runBash(spec('trap \'\' TERM; echo ready; sleep 60'), { graceMs: 200 }) + await waitForStdout(running, 'ready\n') + running.kill() + const result = await running.done expect(result.signal).toBe('SIGKILL') }) diff --git a/packages/bash/bash-local/tsconfig.json b/packages/bash/bash-local/tsconfig.json new file mode 100644 index 0000000000..ae31546543 --- /dev/null +++ b/packages/bash/bash-local/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../bash/bash" + } + ] +} diff --git a/packages/bash/bash/README.md b/packages/bash/bash/README.md new file mode 100644 index 0000000000..6123565e7a --- /dev/null +++ b/packages/bash/bash/README.md @@ -0,0 +1,31 @@ +# @deepseek-ai/dsh-bash + +The **bash executor seam**: an abstract `BashExecutor` service (`ctx.bash`) defining WHAT a bash backend does — run commands, manage background tasks — without saying HOW. + +This package is one third of the bash capability, split so each concern can evolve (and be swapped) independently: + +| Package | Role | +|---|---| +| `@deepseek-ai/dsh-bash` (this) | the interface: abstract service + vocabulary types | +| `@deepseek-ai/dsh-bash-local` | an implementation: local subprocesses | +| `@deepseek-ai/dsh-tool-bash` | the model-facing tool schemas over `ctx.bash` | + +The split mirrors the LLM seam (`LlmService`/`LlmAdapter`) and the agent-tool survey: pi hides execution behind a `BashOperations` interface (local shell / SSH / VM backends), Codex behind an exec-server protocol. A future sandboxed, containerized, or remote executor implements this interface and the tool schemas don't change. + +## Service API (`ctx.bash`) + +| Member | Semantics | +|---|---| +| `run(spec)` | Foreground execution. Resolves when the command finishes. **Rejects only for infrastructure failures** (unusable workdir, missing shell, pre-aborted signal); nonzero exits, timeout kills, and abort kills resolve with a descriptive `BashRunResult`. | +| `start(spec)` | Background execution. Returns a `BashTask` handle immediately; **no timeout applies** (stop tasks via `kill`). | +| `get(id)` / `list()` | Task lookup. | +| `ownerOf(id)` | The opaque OWNER token recorded for a background task at `start` (from the spec's `owner`), or `undefined` for an unknown id OR a known-but-ownerless task. The executor stores/returns it verbatim and NEVER interprets it — the access POLICY lives in the consumer (`dsh-tool-bash`), which compares `ownerOf(id)` to the caller's token. Storing ownership here (disposed with the executor's fiber) is what makes it survive a consumer HMR reload. | +| `readOutput(id)` | **Incremental** output read — consecutive reads never re-deliver. Reads that lost data to buffer bounds flag `lossy` and point at full-stream spill files. Throws for unknown ids. | +| `kill(id)` | Kill a running task. Returns `false` when it already finished; throws for unknown ids. | +| `onTaskDone(listener)` | Completion listener (effect-based, disposer returned). Fires exactly once per task; never after the service is disposed. | + +Implementations subclass `BashExecutor`, implement the abstract methods, and call `notifyTaskDone(task)` on background completion. Disposal must kill every running task (no orphan processes) — see the HMR-safety tests. + +## Vocabulary + +`BashExecRequest` (command, workdir?, timeoutMs?, signal?, owner?) resolves to `BashExecSpec` (command, workdir, timeoutMs, signal?, owner) before execution; `owner` is optional on the request and **required-but-nullable** (`OwnerToken | undefined`) on the resolved spec, so a forgotten owner is a visible `undefined` rather than a silently-absent property. The task id (`BashTaskId`) and the `owner` token (`OwnerToken`) are [branded](../../util/brand) — `OwnerToken` is a DISTINCT brand from `SessionId` (the seam never imports `dsh-session`; the `dsh-tool-bash` consumer is the single boundary that casts its `SessionId` into one). `run()` returns `BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr as `CollectedOutput`) and `start()`/`readOutput()` use `BashTask`/`BashTaskRead` for the background side. See `src/types.ts` for the full contracts. diff --git a/packages/bash/package.json b/packages/bash/bash/package.json similarity index 68% rename from packages/bash/package.json rename to packages/bash/bash/package.json index 52bf80282f..f1bc43c2b4 100644 --- a/packages/bash/package.json +++ b/packages/bash/bash/package.json @@ -5,24 +5,28 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", "cordis": "^4.0.0-rc.6" } } diff --git a/packages/bash/src/index.ts b/packages/bash/bash/src/index.ts similarity index 80% rename from packages/bash/src/index.ts rename to packages/bash/bash/src/index.ts index e22aad5ff3..01c5c081c3 100644 --- a/packages/bash/src/index.ts +++ b/packages/bash/bash/src/index.ts @@ -15,8 +15,9 @@ */ import { Context, Service } from 'cordis' -import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskListener, BashTaskRead } from './types.ts' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskId, BashTaskListener, BashTaskRead, OwnerToken } from './types.ts' +export { BashTaskId, OwnerToken } from './types.ts' export type { BashExecRequest, BashExecSpec, @@ -86,19 +87,34 @@ export abstract class BashExecutor extends Service { abstract start(spec: BashExecSpec): BashTask /** Look up a background task by id. */ - abstract get(id: string): BashTask | undefined + abstract get(id: BashTaskId): BashTask | undefined + + /** + * The opaque OWNER token recorded for a background task at {@link start} + * (from the {@link BashExecSpec}'s `owner`), or `undefined` for an unknown id + * OR a known-but-ownerless task. The executor stores and returns the token + * verbatim — it never interprets it; the access POLICY (who may read/kill a + * task) lives in the consumer (`@deepseek-ai/dsh-tool-bash`), which compares + * `ownerOf(id)` to the caller's token. Collapsing unknown-id and + * known-but-unowned into the same `undefined` is fine: the consumer's access + * gate treats `undefined` as "open", and a genuinely unknown id then fails + * loudly at the subsequent {@link readOutput}/{@link kill} ("unknown task"). + * Storing ownership in the executor (disposed with ITS fiber) — not in the + * tool plugin — is what makes ownership survive a `tool-bash` HMR reload. + */ + abstract ownerOf(id: BashTaskId): OwnerToken | undefined /** All tracked background tasks (insertion order). */ abstract list(): BashTask[] /** Read output produced since the previous read. Throws for unknown ids. */ - abstract readOutput(id: string): BashTaskRead + abstract readOutput(id: BashTaskId): BashTaskRead /** * Kill a running background task. Returns false when it had already * finished (no-op). Throws for unknown ids. */ - abstract kill(id: string): boolean + abstract kill(id: BashTaskId): boolean /** * Register a background-task completion listener (disposed with the diff --git a/packages/bash/src/types.ts b/packages/bash/bash/src/types.ts similarity index 65% rename from packages/bash/src/types.ts rename to packages/bash/bash/src/types.ts index cfa60ab331..d9ab9f9b4d 100644 --- a/packages/bash/src/types.ts +++ b/packages/bash/bash/src/types.ts @@ -6,6 +6,31 @@ * @module dsh-bash/types */ +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** Identifies one background task within an executor (generated `bash-N`). */ +export type BashTaskId = Branded<'BashTaskId'> + +/** Brand a string as a {@link BashTaskId}. */ +export function BashTaskId(id: string): BashTaskId { + return id as BashTaskId +} + +/** + * A background task's opaque isolation key — the CONSUMER's owner identity, not + * the bash seam's. The executor stores and returns it verbatim and never + * interprets it; the access policy lives in the consumer (`dsh-tool-bash`), + * which is the single boundary that casts its own id vocabulary into one. A + * DISTINCT brand (not a `SessionId` alias) keeps the seam decoupled — a + * sandboxed/remote executor inherits no session dependency. + */ +export type OwnerToken = Branded<'OwnerToken'> + +/** Brand a string as an {@link OwnerToken}. */ +export function OwnerToken(id: string): OwnerToken { + return id as OwnerToken +} + /** * A caller's execution REQUEST: `workdir` and `timeoutMs` are optional and * filled by {@link BashExecutor.resolve} from the implementation's config. @@ -20,6 +45,15 @@ export interface BashExecRequest { timeoutMs?: number | undefined /** Abort signal — implementations kill the command when it fires. */ signal?: AbortSignal | undefined + /** + * Opaque OWNER token for a background task — the consumer's isolation key + * (the tool layer passes the owning agent's `session.header.id`). The + * executor stores it on the task and exposes it via {@link BashExecutor.ownerOf}; + * the executor itself NEVER interprets it (no access policy lives in the + * seam — that is the consumer's job). Absent for foreground runs and for an + * ownerless background start (a non-agent caller). + */ + owner?: OwnerToken | undefined } /** @@ -36,6 +70,15 @@ export interface BashExecSpec { timeoutMs: number /** Abort signal — implementations kill the command when it fires. */ signal?: AbortSignal | undefined + /** + * Opaque owner token, REQUIRED-but-nullable (mirrors `workdir`/`timeoutMs` + * being required on the resolved spec): {@link BashExecutor.resolve} carries + * the request's `owner` through, defaulting a missing one to `undefined`. A + * required field makes a forgotten owner a VISIBLE `undefined` rather than a + * silently-absent property that yields an unowned (cross-session-readable) + * task. `start()` stores it; `run()` (foreground) ignores it. + */ + owner: OwnerToken | undefined } /** One captured stream: the (possibly truncated) text plus recovery info. */ @@ -69,7 +112,7 @@ export type BashTaskStatus = 'running' | 'completed' | 'killed' /** A tracked background task handle. */ export interface BashTask { - readonly id: string + readonly id: BashTaskId readonly command: string status: BashTaskStatus /** Exit code once finished (null = killed by signal / still running). */ diff --git a/packages/bash/tests/service.spec.ts b/packages/bash/bash/tests/service.spec.ts similarity index 88% rename from packages/bash/tests/service.spec.ts rename to packages/bash/bash/tests/service.spec.ts index 751bd80015..81530843ed 100644 --- a/packages/bash/tests/service.spec.ts +++ b/packages/bash/bash/tests/service.spec.ts @@ -1,11 +1,12 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import { BashExecutor } from '@deepseek-ai/dsh-bash' +import { BashExecutor, BashTaskId, OwnerToken } from '@deepseek-ai/dsh-bash' import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead } from '@deepseek-ai/dsh-bash' /** Minimal concrete executor: records calls, lets tests drive completions. */ class StubExecutor extends BashExecutor { - tasks = new Map() + tasks = new Map() + private owners = new Map() resolve(request: BashExecRequest): BashExecSpec { return { @@ -13,6 +14,7 @@ class StubExecutor extends BashExecutor { workdir: request.workdir ?? '/stub', timeoutMs: request.timeoutMs ?? 1000, ...request.signal ? { signal: request.signal } : {}, + owner: request.owner, } } @@ -30,7 +32,7 @@ class StubExecutor extends BashExecutor { start(spec: BashExecSpec): BashTask { const task: BashTask = { - id: `stub-${this.tasks.size + 1}`, + id: BashTaskId(`stub-${this.tasks.size + 1}`), command: spec.command, status: 'running', exitCode: null, @@ -38,24 +40,29 @@ class StubExecutor extends BashExecutor { done: Promise.resolve(), } this.tasks.set(task.id, task) + this.owners.set(task.id, spec.owner) return task } - get(id: string): BashTask | undefined { + get(id: BashTaskId): BashTask | undefined { return this.tasks.get(id) } + ownerOf(id: BashTaskId): OwnerToken | undefined { + return this.owners.get(id) + } + list(): BashTask[] { return [...this.tasks.values()] } - readOutput(id: string): BashTaskRead { + readOutput(id: BashTaskId): BashTaskRead { const task = this.tasks.get(id) if (!task) throw new Error(`unknown bash task "${id}"`) return { task, delta: '', lossy: false } } - kill(id: string): boolean { + kill(id: BashTaskId): boolean { const task = this.tasks.get(id) if (!task) throw new Error(`unknown bash task "${id}"`) if (task.status !== 'running') return false diff --git a/packages/bash/bash/tsconfig.json b/packages/bash/bash/tsconfig.json new file mode 100644 index 0000000000..342f636170 --- /dev/null +++ b/packages/bash/bash/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../util/brand" + } + ] +} diff --git a/packages/tool-bash/README.md b/packages/bash/tool-bash/README.md similarity index 64% rename from packages/tool-bash/README.md rename to packages/bash/tool-bash/README.md index 33ac71bc05..656a22cdb1 100644 --- a/packages/tool-bash/README.md +++ b/packages/bash/tool-bash/README.md @@ -30,15 +30,15 @@ Result text: stdout, then a `[stderr]` section, then status markers — `[timed ### Task ownership (cross-session isolation) -The owning agent is recorded per task id at spawn and kept for the lifetime of the loaded plugin instance (it is **not** cleared on completion). `bash_output`/`bash_kill` reject a task owned by a *different* agent with `task belongs to another session` (a task started with no agent — a non-loop caller — has no owner and is open to anyone; a call with no `exec.agent` cannot access an owned task). Task ids are global and predictable, so under multi-session ACP this ownership check is the fence that stops one session's agent from reading or killing another session's background task. (`XXX(tool-bash-owner-hmr)`: an independent HMR reload of this plugin starts a fresh map, so a task spawned before the reload becomes un-owned — acceptable as HMR is dev-only and the session boundary is one user's cooperative editor; a durable fix attaches ownership to the executor/task lifetime.) +The owning agent's session token (`session.header.id`) is stamped onto the task at spawn — passed to the executor via `resolve({ …, owner })` and stored ON THE TASK inside the executor (the `dsh-bash` `ownerOf(id)` seam), **not** in a plugin-local map. `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to the caller's token (`session.header.id`) with `!== undefined` semantics and reject a task owned by a *different* session with `task belongs to another session` (a task started with no agent — a non-loop caller — has no owner token and is open to anyone; a call with no `exec.agent` cannot access an owned task). Task ids are global and predictable, so under multi-session ACP this token check is the fence that stops one session's agent from reading or killing another session's background task. Because ownership lives on the task in the executor (disposed with the `dsh-bash` fiber), it **survives an independent `tool-bash` HMR reload** — closing the old plugin-local-map gap where a reload orphaned pre-reload tasks. (The `onTaskDone` listener is still effect-scoped to this plugin's `apply`, so a completion landing during the reload gap still drops its one notice — the pre-existing reload-gap drop — but the ownership fence itself is HMR-proof.) ## UI presentation -These tools own how their calls render in a UI (an editor's tool-call card) via the `dsh-tools` `presentCall`/`presentResult` seam — a UI never special-cases tool names. For `bash`: the **title** is the exact `command` ("ls -la src") and `kind` is `execute` (terminal/run treatment), matching the reference ACP adapters (claude-agent-acp, codex-acp), which both use the bare command as an execute tool's title. The command is ALSO the **rawInput** for non-terminal UIs that render it (an execute-kind card hides rawInput — Zed shows it only for non-terminal tools — so the command must BE the title to be seen). The model-written `description` rides as a **content** text block shown ABOVE the card. (claude-agent-acp DROPS the description in terminal mode and shows only the card; surfacing it as a content block is a deliberate divergence — we keep the human summary visible alongside the card.) The completed output is wrapped in a fenced ` ```console ` block as the no-terminal-capability fallback — a UI-only affordance, so the model-facing result text stays unfenced. A FOREGROUND `bash` run also flags itself as a **terminal** (the neutral `terminal` field: `presentCall` sets a `cwd` from the model `workdir` when given — absolute as-is, relative for the UI bridge to resolve against the session cwd — else leaves it for the bridge to fill from the session cwd; `presentResult` carries the raw output plus the parsed `exitCode`/`signal`) so a capable client (Zed) renders a terminal card with an exit-status pill instead of the text block — see `packages/acp` ("Terminal card"). A `run_in_background` call is NOT a terminal (it returns a task id immediately and never streams a terminal — poll with `bash_output`), and an `isError` result (spawn failure / abort) carries no exit pill (there is no real process exit); both render as the ordinary execute card / fenced text. `bash_output`/`bash_kill` present a task-scoped title ("Read output from background task bash-3" / "Kill background task bash-3") with the task id as rawInput. These methods are pure/display-only (they also run on `session/load` replay), and a malformed/older logged arg shape falls back to a generic presentation rather than throwing. See `packages/tools` ("Tool-owned UI presentation") and `packages/acp` ("Terminal card" / "Tool-call presentation"). +These tools own how their calls render in a UI (an editor's tool-call card) via the `dsh-tools` `presentCall`/`presentResult` seam — a UI never special-cases tool names. For `bash`: the **title** is the exact `command` ("ls -la src") and `kind` is `execute` (terminal/run treatment), matching the reference ACP adapters (claude-agent-acp, codex-acp), which both use the bare command as an execute tool's title. The command is ALSO the **rawInput** for non-terminal UIs that render it (an execute-kind card hides rawInput — Zed shows it only for non-terminal tools — so the command must BE the title to be seen). The model-written `description` rides as a **content** text block shown ABOVE the card. (claude-agent-acp DROPS the description in terminal mode and shows only the card; surfacing it as a content block is a deliberate divergence — we keep the human summary visible alongside the card.) The completed output is wrapped in a fenced ` ```console ` block as the no-terminal-capability fallback — a UI-only affordance, so the model-facing result text stays unfenced. A FOREGROUND `bash` run also flags itself as a **terminal** (the neutral `terminal` field: `presentCall` sets a `cwd` from the model `workdir` when given — absolute as-is, relative for the UI bridge to resolve against the session cwd — else leaves it for the bridge to fill from the session cwd; `presentResult` carries the raw output plus the parsed `exitCode`/`signal`) so a capable client (Zed) renders a terminal card with an exit-status pill instead of the text block — see `packages/ui/acp` ("Terminal card"). A `run_in_background` call is NOT a terminal (it returns a task id immediately and never streams a terminal — poll with `bash_output`), and an `isError` result (spawn failure / abort) carries no exit pill (there is no real process exit); both render as the ordinary execute card / fenced text. `bash_output`/`bash_kill` present a task-scoped title ("Read output from background task bash-3" / "Kill background task bash-3") with the task id as rawInput. These methods are pure/display-only (they also run on `session/load` replay), and a malformed/older logged arg shape falls back to a generic presentation rather than throwing. See `packages/core/tools` ("Tool-owned UI presentation") and `packages/ui/acp` ("Terminal card" / "Tool-call presentation"). ## Background completion notices -When a background task finishes, a short notice is injected into the owning agent's session (`agent.inject()`, source `{kind: 'plugin', plugin: 'tool-bash'}`). Injection is **durable context for the next model request, not a wake-up** — an idle agent stays idle until something sends a message. That's why the tool descriptions tell the model to poll with `bash_output`. +When a background task finishes, a short notice is injected into the owning agent's session (`agent.inject()`, source `{kind: 'plugin', plugin: 'tool-bash'}`). The owning agent is found by its session token: the listener reads `ctx.bash.ownerOf(task.id)` and scans `ctx.get('agents')?.list()` for an agent whose `session.header.id` matches (read via `ctx.get` — `onTaskDone` runs on the bash fiber, a foreign fiber, so the `ctx.agents` proxy would throw). If no live agent carries that token — e.g. the owning session disconnected and its agent was disposed while the task ran on — the notice is dropped cleanly. Injection is **durable context for the next model request, not a wake-up** — an idle agent stays idle until something sends a message. That's why the tool descriptions tell the model to poll with `bash_output`. ## Permissions diff --git a/packages/tool-bash/package.json b/packages/bash/tool-bash/package.json similarity index 87% rename from packages/tool-bash/package.json rename to packages/bash/tool-bash/package.json index aaf4fde4cc..f9092fabb6 100644 --- a/packages/tool-bash/package.json +++ b/packages/bash/tool-bash/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/tool-bash/src/index.ts b/packages/bash/tool-bash/src/index.ts similarity index 81% rename from packages/tool-bash/src/index.ts rename to packages/bash/tool-bash/src/index.ts index 2cdc28b1f0..9ad1f9a17c 100644 --- a/packages/tool-bash/src/index.ts +++ b/packages/bash/tool-bash/src/index.ts @@ -11,22 +11,24 @@ * message, which is why the tool descriptions tell the model to poll with * `bash_output`. * - * Task ownership: the owning agent is recorded per task id at spawn and kept - * for the lifetime of THIS plugin instance (it is NOT cleared on task - * completion — a finished task must stay un-readable / un-killable by a - * different agent). `bash_output`/`bash_kill` reject a task owned by a DIFFERENT - * agent (a task with no recorded owner is open to anyone). Task ids are global - * and predictable (`bash-1`, …); under multi-session ACP (RFC 011) this - * ownership check is the fence that stops one session's agent from reading or - * killing another session's background task. + * Task ownership: a background task's OWNER is an opaque token — the owning + * agent's `session.header.id` — passed to the executor at spawn + * (`resolve({ …, owner })`) and stored ON THE TASK inside the executor + * (`@deepseek-ai/dsh-bash`'s `ownerOf(id)` seam), NOT in a plugin-local map. + * `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to the caller's token + * and reject a task owned by a DIFFERENT session (`owner !== undefined && owner + * !== caller`); an unowned task (no token — started by a non-agent caller) is + * open to anyone. Task ids are global and predictable (`bash-1`, …); under + * multi-session ACP (RFC 011) this token check is the fence that stops one + * session's agent from reading or killing another session's background task. * - * XXX(tool-bash-owner-hmr): the ownership map is per-plugin-instance, so an - * independent HMR reload of `tool-bash` (without reloading `dsh-bash`) starts a - * fresh map and a task spawned before the reload becomes un-owned (open to any - * caller). This is acceptable today — HMR is dev-only, the ACP session boundary - * is one user's cooperative editor (not an adversarial trust boundary), and the - * executor's own disposal kills its tasks — but a durable fix would attach - * ownership to the executor/task lifetime via a `dsh-bash` seam. + * Storing the token on the task in the EXECUTOR (disposed with the `dsh-bash` + * fiber), rather than in this plugin, is what makes ownership survive a + * `tool-bash` HMR reload — a reload that reset a plugin-local map would orphan + * a task spawned before it. (The `onTaskDone` listener is still effect-scoped + * to this plugin's `apply`, so a + * completion landing during the reload gap still drops its one notice — the + * pre-existing reload-gap drop — but the ownership fence itself is HMR-proof.) * * TODO(permissions): commands run with the executor's full authority. The * permission/sandbox seam is the `tools/execute` waterfall (veto/ask) plus @@ -41,6 +43,7 @@ import { isAbsolute, resolve as resolvePath } from 'node:path' import { defineTool } from '@deepseek-ai/dsh-tools' import type { ToolCallPresentation, ToolResult, ToolResultPresentation } from '@deepseek-ai/dsh-tools' import type { Agent } from '@deepseek-ai/dsh-agent' +import { BashTaskId, OwnerToken } from '@deepseek-ai/dsh-bash' import type { BashRunResult, BashTask, CollectedOutput } from '@deepseek-ai/dsh-bash' export const name = 'tool-bash' @@ -77,11 +80,11 @@ function validateBashArgs(args: { * SchemaSpec validation (the arg-validation RFC); only the non-empty constraint, which the * DSL can't express, is left to check here. */ -function validateTaskId(value: string): string { +function validateTaskId(value: string): BashTaskId { if (value.length === 0) { throw new Error(`invalid task_id: expected a string, got ${JSON.stringify(value)}`) } - return value + return BashTaskId(value) } /** Append the truncation notice (with the full-output spill path) to a stream's text. */ @@ -268,33 +271,48 @@ function statusLine(task: BashTask): string { } export function apply(ctx: Context): void { - // Owning agent per background task id, recorded at spawn. Kept for the - // lifetime of THIS plugin instance (NOT cleared on completion): a completed - // task must stay un-readable / un-killable by a DIFFERENT agent, so the - // ownership record outlives the task. Under multi-session ACP (RFC 011) this - // is the isolation fence — one session's agent must never read or kill - // another session's background task. A task with no recorded owner (started by - // a non-loop caller, `exec.agent` absent) is unowned and accessible to anyone. - // An independent `tool-bash` HMR reload resets this map — see the - // XXX(tool-bash-owner-hmr) note in the module doc. - const taskOwner = new Map() + /** + * The caller's owner TOKEN — the owning agent's `session.header.id`, or + * `undefined` for a non-agent caller. Read `session.header.id` (NOT + * `session.id`): every other subsystem keys off the header id (the ACP bridge, + * both persistence backends), and the sibling `resolveWorkdir` already reads + * `session.header.cwd`, so using `session.id` here would be the asymmetry smell + * the conventions flag. The two are equal in production, but the header is the + * canonical identity. + */ + const callerToken = (exec: { agent?: Agent }): OwnerToken | undefined => + exec.agent ? OwnerToken(exec.agent.session.header.id) : undefined /** - * Authorize a `bash_output`/`bash_kill` call against a task's owner. Rejects - * when the task has a recorded owner and the caller is not that exact agent — - * including the conservative no-agent case (`exec.agent` absent cannot prove - * ownership of an owned task). An unowned task (no record) is allowed. + * Authorize a `bash_output`/`bash_kill` call against the task's stored owner + * token. Rejects when the task HAS an owner and it differs from the caller's + * token — using `!== undefined` semantics, NOT truthiness, so an empty-string + * token is still a real owner (never treated as unowned). An unowned task + * (`ownerOf` returns `undefined`) is allowed; a truly unknown id is also + * `undefined` here and then fails loudly at the subsequent + * `readOutput`/`kill` ("unknown bash task"). The conservative no-agent caller + * (`callerToken` undefined) cannot match an owned task and is rejected. */ - const assertTaskAccess = (taskId: string, exec: { agent?: Agent }): void => { - const owner = taskOwner.get(taskId) - if (owner !== undefined && owner !== exec.agent) { + const assertTaskAccess = (taskId: BashTaskId, exec: { agent?: Agent }): void => { + const owner = ctx.bash.ownerOf(taskId) + if (owner !== undefined && owner !== callerToken(exec)) { throw new Error(`task ${taskId} belongs to another session`) } } // Background completion → inject a notice into the owning agent's session. + // Find the live agent by its session id token via the agent registry, read + // opportunistically with `ctx.get('agents')` (NOT `ctx.agents`/static inject): + // this listener runs from `task.done.then` on the bash fiber — a foreign + // fiber — where the `ctx.agents` property proxy would throw through the + // traceable shadow; `ctx.get(name)` is the topology-independent lookup. No + // registry mounted (`undefined`) → drop the notice. Match on + // `agent.session.header.id`, NOT the registry key: a config agent's id differs + // from its session id, and the owner token IS the session id. ctx.bash.onTaskDone((task) => { - const agent = taskOwner.get(task.id) + const ownerToken = ctx.bash.ownerOf(task.id) + if (ownerToken === undefined) return + const agent = ctx.get('agents')?.list().find(a => OwnerToken(a.session.header.id) === ownerToken) if (!agent) return try { agent.inject( @@ -303,7 +321,7 @@ export function apply(ctx: Context): void { ) } catch (error: unknown) { // The ONE expected failure: the agent was disposed between task - // completion and this injection (LoopAgent.inject throws + // completion and this injection (ReactLoopAgent.inject throws // `agent "" is disposed`). That race is benign — drop the notice. // Anything else is a real bug and must surface, not be swallowed. if (error instanceof Error && error.message.includes('is disposed')) return @@ -348,8 +366,11 @@ export function apply(ctx: Context): void { ...exec.signal ? { signal: exec.signal } : {}, } if (args.run_in_background === true) { - const task = ctx.bash.start(ctx.bash.resolve(request)) - if (exec.agent) taskOwner.set(task.id, exec.agent) + // Stamp the owner token (the agent's session id) onto the spec so the + // executor stores it on the task — the isolation fence for bash_output/ + // bash_kill. Foreground runs pass no owner (they finish inline; nothing + // to fence). + const task = ctx.bash.start(ctx.bash.resolve({ ...request, owner: callerToken(exec) })) return [{ type: 'text', text: `started background task ${task.id}` }] } const result = await ctx.bash.run(ctx.bash.resolve(request)) diff --git a/packages/tool-bash/tests/integration.spec.ts b/packages/bash/tool-bash/tests/integration.spec.ts similarity index 90% rename from packages/tool-bash/tests/integration.spec.ts rename to packages/bash/tool-bash/tests/integration.spec.ts index 2683f7ec55..a67809ffee 100644 --- a/packages/tool-bash/tests/integration.spec.ts +++ b/packages/bash/tool-bash/tests/integration.spec.ts @@ -5,11 +5,12 @@ import SessionStore from '@deepseek-ai/dsh-session' import type { SessionEvent } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import AgentRegistry from '@deepseek-ai/dsh-agent' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' +import { BashTaskId } from '@deepseek-ai/dsh-bash' import * as ToolBash from '@deepseek-ai/dsh-tool-bash' -import { MockAdapter, textResponse, toolCallResponse } from '../../agent-loop/tests/mock-adapter.ts' +import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts' /** * Full-loop integration: a scripted mock model drives the REAL bash tool @@ -30,7 +31,7 @@ async function harness(adapter: MockAdapter) { return ctx } -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { @@ -41,7 +42,7 @@ function waitForIdle(ctx: Context, agent: LoopAgent): Promise { }) } -function events(agent: LoopAgent): SessionEvent[] { +function events(agent: ReactLoopAgent): SessionEvent[] { return [...agent.session.events] } @@ -73,7 +74,7 @@ describe('bash tool through the agent loop', () => { textResponse('The command printed integration-ok.'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('it-fg', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('it-fg'), { model: 'mock' }) agent.send([{ type: 'text', text: 'run echo integration-ok' }]) await waitForIdle(ctx, agent) @@ -105,7 +106,7 @@ describe('bash tool through the agent loop', () => { textResponse('It failed with code 9.'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('it-exit', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('it-exit'), { model: 'mock' }) agent.send([{ type: 'text', text: 'run exit 9' }]) await waitForIdle(ctx, agent) @@ -126,7 +127,7 @@ describe('bash tool through the agent loop', () => { let taskId = '' const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('it-bg', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('it-bg'), { model: 'mock' }) // Intercept the first tool result to capture the generated task id, then // rewrite the second scripted call's arguments to use it. @@ -147,7 +148,7 @@ describe('bash tool through the agent loop', () => { await waitForIdle(ctx, agent) // Wait for the background task itself (completion may race turn end). - const task = ctx.bash.get(taskId) + const task = ctx.bash.get(BashTaskId(taskId)) if (!task) throw new Error(`task ${taskId} not registered`) await task.done diff --git a/packages/tool-bash/tests/tools.spec.ts b/packages/bash/tool-bash/tests/tools.spec.ts similarity index 79% rename from packages/tool-bash/tests/tools.spec.ts rename to packages/bash/tool-bash/tests/tools.spec.ts index 1e75d7fb24..9de079fd33 100644 --- a/packages/tool-bash/tests/tools.spec.ts +++ b/packages/bash/tool-bash/tests/tools.spec.ts @@ -4,10 +4,12 @@ import { join } from 'node:path' import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import { CallId } from '@deepseek-ai/dsh-llm' -import { BashExecutor } from '@deepseek-ai/dsh-bash' -import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead } from '@deepseek-ai/dsh-bash' +import { BashExecutor, BashTaskId } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead, OwnerToken } from '@deepseek-ai/dsh-bash' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' import * as ToolBash from '@deepseek-ai/dsh-tool-bash' import { renderResult } from '@deepseek-ai/dsh-tool-bash' @@ -18,12 +20,43 @@ async function setup() { const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) await ctx.plugin(LocalBashExecutor, { timeoutMs: 10_000 }) ;(ctx.bash as LocalBashExecutor).internals = { spillDir, graceMs: 200 } await ctx.plugin(ToolBash) return ctx } +/** + * Build a fake {@link Agent} whose session token is `sessionId`, REGISTER it in + * `ctx.agents` (the completion-notice path finds the owning agent by scanning + * the registry for a matching `session.header.id`), and return it. The returned + * agent is also passed to `execute` as `exec.agent` so it owns the spawned task. + * The registration disposer is tracked so {@link unregisterFakeAgents} can drop + * it (simulating the owning session disconnecting before a task completes). + */ +const fakeAgentDisposers = new Map void)[]>() +function registerFakeAgent(ctx: Context, sessionId: string, inject: (...args: unknown[]) => void): Agent { + // The registry KEY (agent.id) is deliberately DIFFERENT from the session + // token (session.header.id) — a config agent has `agentId !== sessionId`. The + // owner token IS the session id, so the notice path must find the agent by + // `session.header.id`, NOT the registry key. Using distinct values here makes + // the test fail if a regression matched on the wrong field (a same-value fake + // would pass either way — the "hits the line but not the scenario" trap). + const agent = { id: `agent-${sessionId}`, inject, session: { header: { version: 0, id: sessionId, createdAt: 0 } } } as unknown as Agent + const dispose = ctx.agents.register(agent) + const list = fakeAgentDisposers.get(ctx) ?? [] + list.push(dispose) + fakeAgentDisposers.set(ctx, list) + return agent +} + +/** Unregister every fake agent in this ctx (simulate the owning session disconnecting). */ +function unregisterFakeAgents(ctx: Context): void { + for (const dispose of fakeAgentDisposers.get(ctx) ?? []) dispose() + fakeAgentDisposers.delete(ctx) +} + let callCounter = 0 function call(ctx: Context, name: string, args: unknown) { return ctx.tools.execute({ callId: CallId(`call-${++callCounter}`), name, arguments: args }) @@ -33,9 +66,26 @@ function text(result: { content: { type: string; text?: string }[] }): string { return result.content.filter(block => block.type === 'text').map(block => block.text).join('') } +async function callUntilText( + ctx: Context, + name: string, + args: unknown, + expected: string, + timeoutMs = 5_000, +): Promise>> { + const deadline = Date.now() + timeoutMs + let last: Awaited> | undefined + while (Date.now() < deadline) { + last = await call(ctx, name, args) + if (text(last).includes(expected)) return last + await new Promise(resolve => setTimeout(resolve, 20)) + } + throw new Error(`${name} output did not include ${JSON.stringify(expected)}; last text was ${JSON.stringify(last !== undefined ? text(last) : '')}`) +} + class LossyReadBashExecutor extends BashExecutor { private readonly task: BashTask = { - id: 'bash-lossy', + id: BashTaskId('bash-lossy'), command: 'fake', status: 'running', exitCode: null, @@ -49,6 +99,7 @@ class LossyReadBashExecutor extends BashExecutor { workdir: request.workdir ?? process.cwd(), timeoutMs: request.timeoutMs ?? 0, ...request.signal ? { signal: request.signal } : {}, + owner: request.owner, } } @@ -60,15 +111,19 @@ class LossyReadBashExecutor extends BashExecutor { return this.task } - get(id: string): BashTask | undefined { + get(id: BashTaskId): BashTask | undefined { return id === this.task.id ? this.task : undefined } + ownerOf(): OwnerToken | undefined { + return undefined + } + list(): BashTask[] { return [this.task] } - readOutput(id: string): BashTaskRead { + readOutput(id: BashTaskId): BashTaskRead { if (id !== this.task.id) throw new Error(`unknown bash task "${id}"`) return { task: this.task, delta: 'tail', lossy: true } } @@ -240,11 +295,10 @@ describe('background tools', () => { it('bash_output polls incrementally and reports status', async () => { const ctx = await setup() - const started = await call(ctx, 'bash', { command: 'echo first; sleep 0.3; echo second', description: 'test command', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const started = await call(ctx, 'bash', { command: 'echo first; sleep 1; echo second', description: 'test command', run_in_background: true }) + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) - await new Promise(resolve => setTimeout(resolve, 150)) - const first = await call(ctx, 'bash_output', { task_id: id }) + const first = await callUntilText(ctx, 'bash_output', { task_id: id }, 'first') expect(text(first)).toContain('first') expect(text(first)).toContain('[status: running]') @@ -267,7 +321,7 @@ describe('background tools', () => { await ctx.plugin(ToolBash) const started = await call(ctx, 'bash', { command: 'for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', description: 'test command', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) await ctx.bash.get(id)!.done const read = await call(ctx, 'bash_output', { task_id: id }) expect(text(read)).toContain('[some output was dropped from memory; full output: ') @@ -287,7 +341,7 @@ describe('background tools', () => { it('bash_kill stops a running task; repeat reports already-finished', async () => { const ctx = await setup() const started = await call(ctx, 'bash', { command: 'sleep 60', description: 'test command', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) const killed = await call(ctx, 'bash_kill', { task_id: id }) expect(text(killed)).toBe(`killed background task ${id}`) @@ -320,10 +374,13 @@ describe('background tools', () => { expect(text(result)).toMatch(pattern) }) - it('injects a completion notice into the owning agent', async () => { + it('injects a completion notice into the owning agent (found via the registry by session token)', async () => { const ctx = await setup() const inject = vi.fn() - const agent = { inject, session: { header: { version: 1, id: 'bg', createdAt: 0 } } } as unknown as import('@deepseek-ai/dsh-agent').Agent + // The notice path looks the agent up in ctx.agents by its session token, so + // the agent must be REGISTERED (not merely passed to execute). Mount a + // registry and register a fake whose session.header.id IS the owner token. + const agent = registerFakeAgent(ctx, 'bg', inject) const started = await ctx.tools.execute({ callId: CallId('call-bg'), @@ -331,7 +388,7 @@ describe('background tools', () => { arguments: { command: 'true', description: 'test command', run_in_background: true }, agent, }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) await ctx.bash.get(id)!.done expect(inject).toHaveBeenCalledTimes(1) @@ -346,10 +403,7 @@ describe('background tools', () => { it('swallows ONLY the disposed-agent inject error', async () => { const ctx = await setup() - const agent = { - inject: () => { throw new Error('agent "x" is disposed') }, - session: { header: { version: 1, id: 'bg', createdAt: 0 } }, - } as unknown as import('@deepseek-ai/dsh-agent').Agent + const agent = registerFakeAgent(ctx, 'bg', () => { throw new Error('agent "x" is disposed') }) const started = await ctx.tools.execute({ callId: CallId('call-bg2'), @@ -357,7 +411,7 @@ describe('background tools', () => { arguments: { command: 'true', description: 'test command', run_in_background: true }, agent, }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) await expect(ctx.bash.get(id)!.done).resolves.toBeUndefined() }) @@ -368,10 +422,7 @@ describe('background tools', () => { // the listener itself must have thrown rather than silently eaten it. const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => undefined) try { - const agent = { - inject: () => { throw new Error('unexpected inject bug') }, - session: { header: { version: 1, id: 'bg', createdAt: 0 } }, - } as unknown as import('@deepseek-ai/dsh-agent').Agent + const agent = registerFakeAgent(ctx, 'bg', () => { throw new Error('unexpected inject bug') }) const started = await ctx.tools.execute({ callId: CallId('call-bg3'), @@ -379,7 +430,7 @@ describe('background tools', () => { arguments: { command: 'true', description: 'test command', run_in_background: true }, agent, }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) await ctx.bash.get(id)!.done // notifyTaskDone caught and logged the rethrown error. expect(errorSpy).toHaveBeenCalled() @@ -390,10 +441,32 @@ describe('background tools', () => { } }) + it('drops the notice cleanly when the owning agent is gone from the registry by completion', async () => { + // A bash task (owned by the host-scoped bash-local fiber) can OUTLIVE its + // per-session agent — e.g. the ACP session disconnects and its AgentHandle + // disposes while the background task is still running. The owner token is + // still on the task, but no live agent carries it anymore, so the registry + // lookup finds nothing and the notice is dropped (no throw). + const ctx = await setup() + const inject = vi.fn() + const agent = registerFakeAgent(ctx, 'bg', inject) + const started = await ctx.tools.execute({ + callId: CallId('call-bg4'), + name: 'bash', + arguments: { command: 'true', description: 'test command', run_in_background: true }, + agent, + }) + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) + // Unregister the agent BEFORE the task completes (simulate disconnect). + unregisterFakeAgents(ctx) + await expect(ctx.bash.get(id)!.done).resolves.toBeUndefined() + expect(inject).not.toHaveBeenCalled() + }) + it('does not notify when no agent owned the task', async () => { const ctx = await setup() const started = await call(ctx, 'bash', { command: 'true', description: 'test command', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) await expect(ctx.bash.get(id)!.done).resolves.toBeUndefined() }) }) @@ -403,18 +476,23 @@ describe('background task ownership (cross-session isolation)', () => { function callAs(ctx: Context, agent: import('@deepseek-ai/dsh-agent').Agent | undefined, name: string, args: unknown) { return ctx.tools.execute({ callId: CallId(`own-${++callCounter}`), name, arguments: args, ...agent ? { agent } : {} }) } - // Distinct identities — ownership is by agent object identity, not id. - const fakeAgent = () => ({ inject: () => undefined, session: { header: { version: 1, id: 'bg', createdAt: 0 } } }) as unknown as import('@deepseek-ai/dsh-agent').Agent + // Ownership is by TOKEN (session.header.id), NOT agent object identity — so + // each agent needs a DISTINCT session id, else every fake yields the same + // token and the isolation tests pass for the wrong reason (all tasks owned by + // the same token). The impl reads `session.header.id`, so the fakes MUST carry + // it. + const fakeAgent = (sessionId: string) => + ({ inject: () => undefined, session: { header: { version: 0, id: sessionId, createdAt: 0 } } }) as unknown as import('@deepseek-ai/dsh-agent').Agent - it('rejects bash_output/bash_kill for a task owned by a DIFFERENT agent', async () => { + it('rejects bash_output/bash_kill for a task owned by a DIFFERENT session token', async () => { const ctx = await setup() - const a = fakeAgent() - const b = fakeAgent() + const a = fakeAgent('sess-a') + const b = fakeAgent('sess-b') // Agent A starts a long-running background task. const started = await callAs(ctx, a, 'bash', { command: 'sleep 60', description: 'bg', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) - // Agent B cannot read or kill A's task. + // Agent B (a different session token) cannot read or kill A's task. const readByB = await callAs(ctx, b, 'bash_output', { task_id: id }) expect(readByB.isError).toBe(true) expect(text(readByB)).toMatch(/belongs to another session/) @@ -428,12 +506,26 @@ describe('background task ownership (cross-session isolation)', () => { expect(text(killByA)).toBe(`killed background task ${id}`) }) + it('a DIFFERENT Agent object with the SAME session token may access the task (ownership is by token, not object identity)', async () => { + // Ownership fences by session.header.id, NOT Agent object identity. Two + // distinct Agent objects sharing one session token (e.g. an agent re-created + // on the same session) are the SAME owner. + const ctx = await setup() + const a1 = fakeAgent('sess-shared') + const a2 = fakeAgent('sess-shared') // distinct object, same token + const started = await callAs(ctx, a1, 'bash', { command: 'sleep 60', description: 'bg', run_in_background: true }) + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) + const readByA2 = await callAs(ctx, a2, 'bash_output', { task_id: id }) + expect(readByA2.isError).toBe(false) + await callAs(ctx, a1, 'bash_kill', { task_id: id }) // cleanup + }) + it('the no-agent (non-loop) caller cannot access an owned task', async () => { const ctx = await setup() - const a = fakeAgent() + const a = fakeAgent('sess-a') const started = await callAs(ctx, a, 'bash', { command: 'sleep 60', description: 'bg', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! - // A call with no exec.agent cannot prove ownership of an owned task. + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) + // A call with no exec.agent has no token → cannot prove ownership of an owned task. const read = await callAs(ctx, undefined, 'bash_output', { task_id: id }) expect(read.isError).toBe(true) expect(text(read)).toMatch(/belongs to another session/) @@ -442,22 +534,22 @@ describe('background task ownership (cross-session isolation)', () => { it('an UNOWNED task (started with no agent) is accessible to anyone', async () => { const ctx = await setup() - // Started by a non-loop caller (no exec.agent) → no recorded owner. + // Started by a non-loop caller (no exec.agent) → no owner token recorded. const started = await callAs(ctx, undefined, 'bash', { command: 'sleep 60', description: 'bg', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) // Any agent (and the no-agent caller) may read/kill it. - const read = await callAs(ctx, fakeAgent(), 'bash_output', { task_id: id }) + const read = await callAs(ctx, fakeAgent('sess-x'), 'bash_output', { task_id: id }) expect(read.isError).toBe(false) const killed = await callAs(ctx, undefined, 'bash_kill', { task_id: id }) expect(killed.isError).toBe(false) }) - it('the owner can still access its task AFTER it completes (owner record persists)', async () => { + it('the owner can still access its task AFTER it completes (owner token persists on the task)', async () => { const ctx = await setup() - const a = fakeAgent() - const b = fakeAgent() + const a = fakeAgent('sess-a') + const b = fakeAgent('sess-b') const started = await callAs(ctx, a, 'bash', { command: 'echo done', description: 'bg', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) await ctx.bash.get(id)!.done // Completion does NOT clear ownership: B is still rejected, A still allowed. const readByB = await callAs(ctx, b, 'bash_output', { task_id: id }) @@ -467,12 +559,12 @@ describe('background task ownership (cross-session isolation)', () => { expect(readByA.isError).toBe(false) }) - it('documents the HMR caveat: an independent tool-bash reload resets ownership', async () => { - // The ownership map is per-plugin-instance (XXX(tool-bash-owner-hmr)). When - // ONLY tool-bash is reloaded (bash/executor + task survive), the new instance - // has an empty map, so the previously-owned task becomes unowned (open). This - // test pins that documented behavior — a regression here (e.g. an accidental - // global map) would change it. + it('ownership SURVIVES an independent tool-bash HMR reload (token lives on the executor)', async () => { + // The owner token lives on the TASK inside the executor (dsh-bash fiber), NOT + // in a tool-bash plugin-local map. So reloading ONLY tool-bash (executor + + // task survive) preserves ownership. This is the regression guard: a + // plugin-local map would make B accessible after reload, and this test would + // catch it. const ctx = new Context() await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) @@ -480,21 +572,23 @@ describe('background task ownership (cross-session isolation)', () => { ;(ctx.bash as LocalBashExecutor).internals = { spillDir, graceMs: 200 } const fiber = await ctx.plugin(ToolBash) - const a = fakeAgent() - const b = fakeAgent() + const a = fakeAgent('sess-a') + const b = fakeAgent('sess-b') const started = await callAs(ctx, a, 'bash', { command: 'sleep 60', description: 'bg', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) // Before reload: B is rejected (A owns it). expect((await callAs(ctx, b, 'bash_output', { task_id: id })).isError).toBe(true) - // Reload ONLY tool-bash; the executor and its running task survive. + // Reload ONLY tool-bash; the executor and its running task (with its owner + // token) survive. await fiber.dispose() await ctx.plugin(ToolBash) expect(ctx.bash.get(id)?.status).toBe('running') + expect(ctx.bash.ownerOf(id)).toBe('sess-a') - // After reload the fresh map has no owner → B can now access it (the caveat). - expect((await callAs(ctx, b, 'bash_output', { task_id: id })).isError).toBe(false) - await callAs(ctx, b, 'bash_kill', { task_id: id }) // cleanup + // After reload, ownership is INTACT → B is STILL rejected. + expect((await callAs(ctx, b, 'bash_output', { task_id: id })).isError).toBe(true) + await callAs(ctx, a, 'bash_kill', { task_id: id }) // cleanup }) }) @@ -504,7 +598,7 @@ describe('session-cwd routing (per-session workdir)', () => { } // An agent whose session header carries a cwd (what session/new records). const agentInCwd = (cwd: string) => - ({ inject: () => undefined, session: { header: { version: 1, id: 'c', createdAt: 0, cwd } } }) as unknown as import('@deepseek-ai/dsh-agent').Agent + ({ inject: () => undefined, session: { header: { version: 0, id: 'c', createdAt: 0, cwd } } }) as unknown as import('@deepseek-ai/dsh-agent').Agent it('defaults bash to the agent\'s session cwd (not the server launch dir)', async () => { const ctx = await setup() @@ -596,7 +690,7 @@ describe('status lines', () => { it('reports kills without a recorded signal (executor raced process exit)', async () => { const ctx = await setup() const started = await call(ctx, 'bash', { command: 'sleep 60', description: 'test command', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) const task = ctx.bash.get(id)! await call(ctx, 'bash_kill', { task_id: id }) @@ -610,7 +704,7 @@ describe('status lines', () => { it('reports completed tasks with a null exit code as exit 0', async () => { const ctx = await setup() const started = await call(ctx, 'bash', { command: 'true', description: 'test command', run_in_background: true }) - const id = /task (bash-\d+)/.exec(text(started))![1]! + const id = BashTaskId(/task (bash-\d+)/.exec(text(started))![1]!) const task = ctx.bash.get(id)! await task.done // Defensive: completed tasks always carry an exit code in practice; the diff --git a/packages/bash/tool-bash/tsconfig.json b/packages/bash/tool-bash/tsconfig.json new file mode 100644 index 0000000000..89b10bfea8 --- /dev/null +++ b/packages/bash/tool-bash/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/tools" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../bash/bash" + } + ] +} diff --git a/packages/bash/tsconfig.json b/packages/bash/tsconfig.json deleted file mode 100644 index 2617271c44..0000000000 --- a/packages/bash/tsconfig.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" } - ] -} diff --git a/packages/core/README.md b/packages/core/README.md new file mode 100644 index 0000000000..8d8805471a --- /dev/null +++ b/packages/core/README.md @@ -0,0 +1,16 @@ +# core/ — product API spine + +The packages every harness build is assembled from: the session log, the system-prompt assembly, the tool registry, the agent vocabulary, and the one concrete loop that drives them. These are **product** packages — the stable surface plugins and consumers build against. + +| Package | Role | ctx key | +|---|---|---| +| `session/` | Event-sourced session log + in-memory store | `ctx.sessions` | +| `system-prompt/` | Prompt-section + tool-schema assembly registry | `ctx.systemPrompt` | +| `tools/` | Tool registry + `tools/execute` waterfall | `ctx.tools` | +| `agent/` | Agent interface, registry, `agent/*` event vocabulary | `ctx.agents` | +| `agent-loop/` | The concrete loop plugin: `ReactLoopAgent` + the loop driver | `ctx.agentLoop` | +| `agent-core/` | Bundle plugin: the providerless/executor-less/UI-less spine as code | (loads the spine) | + +`agent-loop` is the one concrete implementation of the `agent` seam and lives here because it is the harness's default product loop; everything else in `core/` is interface/vocabulary. Plugins depend on the `agent` vocabulary, never on `agent-loop` directly, so the loop stays swappable. + +`agent-core` is the composition counterpart: one bundle plugin that loads the whole providerless spine (`timer` + `llm` + sessions + system-prompt + tools + agents + invariants + `tool-bash` + `agent-loop`) and forwards `agent-loop`'s `agents` list as its own config. App packages (`ui/stdio-agent`, `ui/acp-agent`) consume it and add only a front door; a leaf adds only the swappable backends. It lives in `core/` because it composes exclusively `core/` + interface packages and ships no provider, executor, or UI of its own. diff --git a/packages/core/agent-core/README.md b/packages/core/agent-core/README.md new file mode 100644 index 0000000000..28a3592ac6 --- /dev/null +++ b/packages/core/agent-core/README.md @@ -0,0 +1,44 @@ +# @deepseek-ai/dsh-agent-core + +The **providerless, executor-less, UI-less agent spine** as ONE Cordis bundle plugin. It loads the fixed set of services every harness agent needs and forwards the loop's `agents` list as its own config — so an app package composes a working agent by adding only a front door and the swappable backends. + +This is the package to read to see **the whole plugin tree at once** — the teaching role the inlined `echo-agent` `cordis.yml` used to play before the spine moved behind this bundle. + +## The tree it loads + +`apply(ctx, config)` mounts each of these as a child of the bundle fiber: + +``` +@cordisjs/plugin-timer timer service (writes nothing to stdout) +@deepseek-ai/dsh-llm abstract LLM service + content-block vocabulary +@deepseek-ai/dsh-session event-sourced session log + store +@deepseek-ai/dsh-system-prompt prompt-section + tool-schema assembly +@deepseek-ai/dsh-tools tool registry + tools/execute waterfall +@deepseek-ai/dsh-agent agent registry + agent/* event vocabulary +@deepseek-ai/dsh-invariants dev-mode event-contract assertions +@deepseek-ai/dsh-tool-bash the model-facing bash/bash_output/bash_kill schemas +@deepseek-ai/dsh-agent-loop THE concrete loop (gets the forwarded `agents`) +``` + +## What it deliberately leaves OUTSIDE the bundle + +The spine is everything COMMON to every front door. The swappable and front-door-coupled pieces stay out, picked by whatever loads the bundle: + +- **the LLM adapter** — the bundle ships the abstract `llm` service; the leaf registers a concrete adapter on `ctx.llm` (`llm-deepseek`, `llm-pi-ai`, `llm-replay`). +- **the bash executor** — the bundle ships `tool-bash` (the consumer schema); the leaf provides `ctx.bash` (`bash-local` or a sandboxed impl). +- **presentation + per-app infra** — the stdio UI / ACP bridge, a console logger, `hmr`. These form the coupled "front-door cluster" that the app packages ([`dsh-stdio-agent`](../../ui/stdio-agent/README.md), [`dsh-acp-agent`](../../ui/acp-agent/README.md)) bake in. `timer` is in the spine (common to both, stdout-silent); a console logger is NOT (it writes to stdout, which the ACP bridge reserves for JSON-RPC). + +This is the [interface/implementation/consumer seam](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md) raised to the composition level: the bundle owns the shared spine, the leaf owns the backends, the app package owns the front door. + +## Config + +```ts +import type { Config } from '@deepseek-ai/dsh-agent-core' +// Config === AgentLoop.Config — the `agents` list, default []. +``` + +The bundle FORWARDS `agent-loop`'s `agents` list as its own (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`). Forwarding the list is exactly why the loop can live in the shared spine even though the apps disagree on which agents to pre-create. + +## Why a code bundle, not a shared YAML include + +A YAML include can dedupe the config, but it cannot OWN a `bin`, and it can only *describe* the front-door coupling in a comment and trust each leaf to obey. Moving the spine into a package, and the front-door cluster into the app packages, means the default leaf for an ACP server has no logger entry to copy wrong — "the ACP app never logs to stdout" stops being a prose warning a leaf must remember and becomes the app package's default shape (a leaf can still add a sibling logger, so the rule stays documented — but it has nothing to get wrong by default). Services register in the root store keyed by their isolate symbol, so a child loaded here is visible to the bundle's siblings (the leaf's adapter and executor) exactly as a nested `plugin-include` subtree's services were — cordis gates every read on `inject`, never on load order. diff --git a/packages/core/agent-core/package.json b/packages/core/agent-core/package.json new file mode 100644 index 0000000000..a70ee30e71 --- /dev/null +++ b/packages/core/agent-core/package.json @@ -0,0 +1,48 @@ +{ + "name": "@deepseek-ai/dsh-agent-core", + "description": "The providerless/executor-less/UI-less agent spine as one Cordis bundle plugin (timer + llm + sessions + system-prompt + tools + agents + invariants + tool-bash + agent-loop)", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@cordisjs/plugin-timer": "^1.1.2", + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-agent-loop": "^0.0.1", + "@deepseek-ai/dsh-invariants": "^0.0.1", + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "@deepseek-ai/dsh-system-prompt": "^0.0.1", + "@deepseek-ai/dsh-tool-bash": "^0.0.1", + "@deepseek-ai/dsh-tools": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@cordisjs/plugin-timer": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-loop": "workspace:^", + "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", + "@deepseek-ai/dsh-tool-bash": "workspace:^", + "@deepseek-ai/dsh-tools": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/core/agent-core/src/index.ts b/packages/core/agent-core/src/index.ts new file mode 100644 index 0000000000..ad3f5d8c46 --- /dev/null +++ b/packages/core/agent-core/src/index.ts @@ -0,0 +1,88 @@ +/** + * The providerless, executor-less, UI-less agent spine as ONE bundle plugin. + * + * Loads the fixed set of services every harness agent needs — `timer`, the LLM + * service, the session store, system-prompt assembly, the tool registry, the + * agent registry, the dev-mode invariants, the model-facing `bash` tool + * schemas, and the concrete `agent-loop` — and forwards the loop's `agents` + * list as its OWN config (default `[]`), so each app supplies its own + * pre-created agents. + * + * It is deliberately NOT the whole app: the swappable choices stay OUTSIDE the + * bundle, picked by whatever loads it. + * - the LLM ADAPTER (`llm-deepseek`/`llm-pi-ai`/`llm-replay`) — the bundle + * ships the abstract `llm` service + `tool-bash` consumer schema; the leaf + * registers a concrete adapter on `ctx.llm`. + * - the bash EXECUTOR (`bash-local` or a sandboxed impl) — the bundle ships + * the `bash` tool consumer; the leaf provides `ctx.bash`. + * - the PRESENTATION (stdio UI / ACP bridge / a logger) and the per-app infra + * (a console logger, `hmr`) — these are the coupled "front-door cluster" the + * app packages ({@link @deepseek-ai/dsh-stdio-agent}, + * {@link @deepseek-ai/dsh-acp-agent}) bake in, NOT the shared spine. + * + * This is the interface/implementation/consumer seam at the composition level: + * the bundle owns the shared spine, the leaf owns the backends, the app package + * owns the front door. `timer` is in the spine (common to every front door — it + * writes nothing to stdout); the console logger is NOT (it writes to stdout, + * which the ACP bridge reserves for its JSON-RPC channel). + * + * Services register in the root store keyed by their isolate symbol, so a child + * loaded here via `ctx.plugin(...)` is visible to the bundle's SIBLINGS (the + * leaf's adapter and executor) exactly as a nested `plugin-include` subtree's + * services were before this bundle existed — cordis gates every read on + * `inject`, never on load order, so the fixed child set resolves regardless of + * which entry loads first. + * + * Plugin export shape: named `name`/`Config`/`apply`, NO default export — the + * cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray + * default would collapse the module to the bare `apply` function and drop the + * `Config` schema (see docs/postmortem/0001). The keyless Loader-path smokes in + * the app packages guard this end-to-end. + * + * @module @deepseek-ai/dsh-agent-core + */ + +import type { Context } from 'cordis' +import Timer from '@cordisjs/plugin-timer' +import LlmService from '@deepseek-ai/dsh-llm' +import SessionStore from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import * as invariants from '@deepseek-ai/dsh-invariants' +import * as toolBash from '@deepseek-ai/dsh-tool-bash' +import AgentLoop, { type Config as AgentLoopConfig } from '@deepseek-ai/dsh-agent-loop' + +export const name = 'agent-core' + +/** + * Bundle config: the agent-loop `agents` list, forwarded verbatim. Default `[]` + * — an app that pre-creates no agents (the ACP bridge creates them on demand at + * `session/new`) simply omits it; an app that needs a pre-created `main` (the + * stdio chat) supplies one. This IS {@link AgentLoopConfig}, so the schema and + * the forwarded shape can never drift. + */ +export type Config = AgentLoopConfig + +/** Forward the loop's own schema so validation + defaulting stay identical. */ +export const Config = AgentLoop.Config + +/** + * Load the spine. Each `ctx.plugin(...)` mounts one child of the bundle fiber; + * `agent-loop` receives the forwarded `agents` list. Load order is irrelevant + * (cordis pends each fiber on its `inject` until the services it needs exist), + * but the listing mirrors the dependency layering for readability: the LLM + * vocabulary and core registries first, then the dev tripwire and the bash tool + * consumer, then the loop that drives them. + */ +export function apply(ctx: Context, config: Config): void { + ctx.plugin(Timer) + ctx.plugin(LlmService) + ctx.plugin(SessionStore) + ctx.plugin(SystemPrompt) + ctx.plugin(ToolRegistry) + ctx.plugin(AgentRegistry) + ctx.plugin(invariants) + ctx.plugin(toolBash) + ctx.plugin(AgentLoop, { agents: config.agents }) +} diff --git a/packages/core/agent-core/tests/agent-core.spec.ts b/packages/core/agent-core/tests/agent-core.spec.ts new file mode 100644 index 0000000000..67f5d88532 --- /dev/null +++ b/packages/core/agent-core/tests/agent-core.spec.ts @@ -0,0 +1,79 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' +import * as agentCore from '../src/index.ts' +import { AgentId } from '@deepseek-ai/dsh-agent' + +/** + * Unit coverage for the @deepseek-ai/dsh-agent-core bundle: mounting it brings + * up the whole providerless spine in one `ctx.plugin`, and the forwarded + * `agents` config reaches the loop (default `[]`, or a pre-created agent). + * + * The bundle is exercised through `ctx.plugin(agentCore, …)` — the NAMESPACE + * import, the same shape the Loader builds from `unwrapExports`. The real + * Loader-path guard (export shape, `unwrapExports`) is the app packages' keyless + * bin smokes; here we assert the composition + config forwarding. + */ +async function mount(config?: agentCore.Config): Promise { + const ctx = new Context() + await ctx.plugin(agentCore, config) + // The bundle mounts its children inside apply() (not awaited there); let their + // fibers settle so the spine services and any pre-created agent are ready. + await new Promise(resolve => setTimeout(resolve, 50)) + return ctx +} + +describe('dsh-agent-core bundle', () => { + it('brings up the full providerless spine', async () => { + const ctx = await mount() + // One service from each layer of the spine proves the children loaded. + expect(ctx.get('timer')).toBeDefined() + expect(ctx.get('llm')).toBeDefined() + expect(ctx.get('sessions')).toBeDefined() + expect(ctx.get('systemPrompt')).toBeDefined() + expect(ctx.get('tools')).toBeDefined() + expect(ctx.get('agents')).toBeDefined() + expect(ctx.get('agentLoop')).toBeDefined() + await ctx.fiber.dispose() + }) + + it('defaults the agents list to empty (no pre-created agents)', async () => { + const ctx = await mount() + expect(ctx.get('agents')?.get(AgentId('main'))).toBeUndefined() + await ctx.fiber.dispose() + }) + + it('forwards a pre-created agent to the loop', async () => { + const ctx = await mount({ + agents: [{ id: AgentId('main'), model: 'mock', systemPrompt: 'hi' }], + }) + expect(ctx.get('agents')?.get(AgentId('main'))).toBeDefined() + await ctx.fiber.dispose() + }) + + it('re-exports the loop config schema as its own', () => { + expect(agentCore.Config).toBeDefined() + expect(agentCore.name).toBe('agent-core') + }) + + it('has the namespace-plugin export shape (no stray default) so the Loader keeps name/Config/apply', () => { + // Postmortem 0001 guard: a stray `export default apply` makes the Loader's + // `unwrapExports` (`exports.default ?? exports`) collapse the module to the + // bare `apply` function, DROPPING the named `name`/`Config`. This package has + // no `inject` export (it mounts children that carry their own), so that + // collapse would NOT crash at load — the plugin would boot but silently lose + // its config schema. This bundle is also never Loader-unwrapped by any smoke + // (the apps import it directly; the mount test namespace-mounts it), so this + // is its ONLY export-shape guard. Assert directly AND through the real + // `unwrapExports` so adding `export default` to src/index.ts fails here. + expect('default' in agentCore).toBe(false) + expect(typeof agentCore.apply).toBe('function') + + const loader = Object.create(Loader.prototype) as Loader + const unwrapped = loader.unwrapExports(agentCore) as Record + expect(unwrapped).toBe(agentCore) + expect(unwrapped.name).toBe('agent-core') + expect(unwrapped.Config).toBeDefined() + expect(typeof unwrapped.apply).toBe('function') + }) +}) diff --git a/packages/core/agent-core/tsconfig.json b/packages/core/agent-core/tsconfig.json new file mode 100644 index 0000000000..83bf06c586 --- /dev/null +++ b/packages/core/agent-core/tsconfig.json @@ -0,0 +1,42 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/timer" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + }, + { + "path": "../../core/system-prompt" + }, + { + "path": "../../core/tools" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../core/agent-loop" + }, + { + "path": "../../support/invariants" + }, + { + "path": "../../bash/tool-bash" + } + ] +} diff --git a/packages/agent-loop/README.md b/packages/core/agent-loop/README.md similarity index 51% rename from packages/agent-loop/README.md rename to packages/core/agent-loop/README.md index fbfc745808..4892b357bf 100644 --- a/packages/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -1,6 +1,6 @@ # dsh-agent-loop -THE concrete agent plugin: `LoopAgent` and the loop driver. Implements the `Agent` interface and drives the session/turn/step lifecycle. +THE concrete agent plugin: `ReactLoopAgent` and the loop driver. Implements the `Agent` interface and drives the session/turn/step lifecycle. This is the only package in the harness that contains concrete loop logic. Everything else is an abstract service or a plugin against extension seams — new behavior goes into plugins, not here. @@ -8,12 +8,14 @@ This is the only package in the harness that contains concrete loop logic. Every ### Public API -- `ctx.agentLoop.create(id: string, options?: AgentOptions): LoopAgent` — config-driven create: an agent on a fresh per-run session id `${id}-session-` (no cwd). Used for `cordis.yml`-configured agents. The per-run uuid avoids colliding with the on-disk log a prior run materialized once a durable persistence backend is loaded; each run is a new session (a deliberate demo simplification — a real resume-or-create policy is a TODO). Disposed with the calling fiber. +- `ctx.agentLoop.create(id: string, options?: AgentOptions): ReactLoopAgent` — config-driven create: an agent on a fresh per-run session id `${id}-session-` (no cwd). Used for `cordis.yml`-configured agents. The per-run uuid avoids colliding with the on-disk log a prior run materialized once a durable persistence backend is loaded; each run is a new session (a deliberate demo simplification — a real resume-or-create policy is a TODO). Disposed with the calling fiber. `AgentLoop` also implements the `AgentFactory` seam and registers itself via `ctx.agents.setFactory(this)`, so plugins create/resume agents through `ctx.agents` (the interface): -- `ctx.agents.create({ agentId, sessionId, meta?, agentOptions? })` — programmatic create on a caller-supplied `sessionId` (e.g. an ACP-generated id), NOT `${id}-session`. -- `ctx.agents.resume({ agentId, resumeSessionId, agentOptions? })` — load a persisted session via `ctx.sessionPersistence` ([session persistence](../../docs/rfc/implemented/2026-06-14-session-persistence.md)) and resume an agent on it. The live session id is the resumed id; turn numbering and derived history continue from the loaded log. Requires a session-persistence backend (NOT hard-injected — non-persistent demos still work; `resume` rejects with a clear error when persistence is absent). +- `ctx.agents.create({ agentId, sessionId, meta?, agentOptions? }): AgentHandle` — programmatic create on a caller-supplied `sessionId` (e.g. an ACP-generated id), NOT `${id}-session`. Returns an [`AgentHandle`](../agent/README.md) — the owner disposes it to tear down exactly this agent (stop loop + await quiescence + unregister + remove session). +- `ctx.agents.resume({ agentId, resumeSessionId, agentOptions? }): Promise` — load a persisted session via `ctx.sessionPersistence` ([session persistence](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md)) and resume an agent on it. The live session id is the resumed id; turn numbering and derived history continue from the loaded log. Requires a session-persistence backend (NOT hard-injected — non-persistent demos still work; `resume` rejects with a clear error when persistence is absent). Returns an `AgentHandle`. + +The config-driven `ctx.agentLoop.create()` path keeps its agent owned by the loop fiber (it discards the handle) — only the programmatic factory callers (the ACP bridge) hold a handle and own per-agent teardown. ### Injected services @@ -35,7 +37,7 @@ Agents listed in config are auto-created at startup. ### Classes -- `LoopAgent` — the concrete `Agent` implementation. Owns the inbox (`Inbox`), the per-step `AbortController`, and the loop driver. Everything observable happens through session events and the `agent/*` event taxonomy. +- `ReactLoopAgent` — the concrete `Agent` implementation. Owns the inbox (`Inbox`), the per-step `AbortController`, and the loop driver. Everything observable happens through session events and the `agent/*` event taxonomy. - `Inbox` — per-agent queued + steering FIFOs (`enqueue`, `steer`, `drainQueued`, `drainSteering`, `waitForQueued`). ### Loop lifecycle (`loop.ts`) @@ -66,6 +68,8 @@ forever: Error containment: a throwing plugin ends the **turn**, never the loop. Dispose mid-turn emits `agent/status('disposed')` and ends with reason `disposed`. A step that hits the model's output-token ceiling makes the turn end `max-tokens` (the rule: any `max-tokens` step in the turn surfaces as `max-tokens`; `disposed`/`aborted`/`error` still take precedence) — distinct from a clean `completed` stop. +Cancellation: `agent.cancel()` is the single public stop primitive — it clears the queued + steering FIFOs, aborts the in-flight step, and drives a turn-scoped marker the driver checks at every point a turn could start or continue (right after the idle wait, after the `running` flip, before each step, and at the continuation gate) so a turn about to start is dropped. A cancelled turn ends `aborted`; a queued-but-not-started prompt never runs and cannot be batched into the cancelled turn. The marker is reset once per loop iteration, so a cancel governs exactly one turn and never leaks onto a later prompt. (The loop still aborts its own per-step `AbortController` directly on disposal and from `cancel()`; that controller is loop-internal, not a public verb.) + ### What is NOT here Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy: diff --git a/packages/agent-loop/package.json b/packages/core/agent-loop/package.json similarity index 88% rename from packages/agent-loop/package.json rename to packages/core/agent-loop/package.json index b9744eab2a..6e92adb6ab 100644 --- a/packages/agent-loop/package.json +++ b/packages/core/agent-loop/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/agent-loop/src/agent.ts b/packages/core/agent-loop/src/agent.ts similarity index 75% rename from packages/agent-loop/src/agent.ts rename to packages/core/agent-loop/src/agent.ts index 96a1158c66..402a9a8416 100644 --- a/packages/agent-loop/src/agent.ts +++ b/packages/core/agent-loop/src/agent.ts @@ -1,5 +1,5 @@ /** - * The concrete Agent implementation: LoopAgent plus its inbox. Everything + * The concrete Agent implementation: ReactLoopAgent plus its inbox. Everything * observable happens through session events and the agent/* event taxonomy — * plugins never need this class. * @@ -21,11 +21,30 @@ import { isTurnOpen, lastTurnNumber, runLoop } from './loop.ts' * the loop driver. Everything observable happens through session events and * the agent/* event taxonomy — plugins never need this class. */ -export class LoopAgent implements Agent { +export class ReactLoopAgent implements Agent { readonly inbox = new Inbox() private _status: AgentStatus = 'idle' private currentAbort: AbortController | undefined + /** + * Turn-scoped cancel marker, set by {@link cancel} and read/cleared by the + * driver loop (via the LoopHandle) at every point a turn could start or + * continue. Armed ONLY when there is something to cancel (a running turn, an + * in-flight step, or queued/steering work), so an idle no-op cancel cannot + * leave it set to wrongly drop a later prompt. + */ + private cancelRequested = false + /** + * The resolved reason for the pending {@link cancel} (`reason ?? 'cancelled'`), + * read by the driver loop's marker branches so a turn dropped in a + * marker-only window (pre-step / continuation, where no `AbortController` + * carries the reason) ends with the SAME `{kind:'aborted', reason}` the + * mid-step abort path produces from `abort.signal.reason`. Without this the + * caller's `cancel(reason)` would be silently replaced by the literal + * 'cancelled' whenever the cancel landed outside a running step — making the + * logged reason race-dependent and the public `reason?` param half-effective. + */ + private cancelReason = 'cancelled' private disposed: Promise private resolveDisposed!: () => void /** Resolves when the driver loop has fully exited (tests/disposal). */ @@ -172,8 +191,32 @@ export class LoopAgent implements Agent { } } - abort(reason?: string): void { - this.currentAbort?.abort(reason ?? 'aborted') + cancel(reason?: string): void { + // Arm-gate: only mark a cancellation when there is actually work to cancel — + // a running turn, an in-flight step, or queued/steering work. An idle cancel + // with nothing pending is a true no-op; arming the marker then would wrongly + // drop the NEXT legitimate prompt (the marker is consumed only at the loop's + // turn-decision points, which an idle parked loop does not reach until woken + // by a real send()). Note the gate canNOT be `status === 'running'` alone: + // the pre-step window (a send() queued but the loop not yet flipped to + // running) has status `idle` with `hasQueued` true, and the marker exists + // precisely to cover it. + if (this._status === 'running' || this.currentAbort !== undefined || this.inbox.hasQueued || this.inbox.hasSteering) { + this.cancelRequested = true + // Capture the resolved reason for the marker-only windows (pre-step / + // continuation). The mid-step path reads it from abort.signal.reason + // below; the marker path reads it via the LoopHandle's cancelReason(). + this.cancelReason = reason ?? 'cancelled' + } + // Drop all pending queued + steering work (un-started prompts never run; the + // cancelled turn's steering is not re-enqueued). Cleared directly even when + // the loop is parked in waitForQueued — there is no turn to stop and nothing + // left for the parked loop to run, so no wake is needed. + this.inbox.clear() + // Interrupt an in-flight step immediately (the running turn observes the + // abort and ends `aborted`). The marker covers the windows where no step is + // running (pre-step, continuation). + this.currentAbort?.abort(reason ?? 'cancelled') } /** @@ -185,8 +228,10 @@ export class LoopAgent implements Agent { * internal waiter (see {@link idleWaiters}) released on the next * running→idle/disposed transition, resolving on `idle` directly (the turn * fully ended) or chaining {@link done} on `disposed` (wait for the loop to - * actually exit). Implements the {@link Agent.whenIdle} contract used by - * teardown (`abort()` then `await whenIdle()`). + * actually exit). Implements the {@link Agent.whenIdle} contract: a non-owner + * quiescence-observation hook, distinct from teardown (a lifecycle owner stops + * and unregisters via `AgentHandle.dispose()`, which awaits {@link done} + * directly, not through this). */ whenIdle(): Promise { if (this._status === 'disposed') return this.done @@ -218,6 +263,16 @@ export class LoopAgent implements Agent { setAbort: controller => void (this.currentAbort = controller), disposed: this.disposed, isDisposed: () => this._status === 'disposed', + isCancelled: () => this.cancelRequested, + cancelReason: () => this.cancelReason, + clearCancel: () => { this.cancelRequested = false }, + // Settle whenIdle() waiters WITHOUT a status transition — the pre-step + // cancel-skip path drops the about-to-run turn and re-parks without ever + // flipping running→idle, so a waiter registered in the pre-step window + // (status idle, hasQueued was true) would otherwise hang. This emits no + // agent/status, so an ACP agent/status listener never sees a spurious idle + // that would resolve a freshly-queued prompt as cancelled. + settleIdle: () => { this.settleIdleWaiters() }, }) // The disposer must be infallible: it runs inside the fiber's LIFO // disposal chain, where a throw would skip later disposers (e.g. the diff --git a/packages/agent-loop/src/inbox.ts b/packages/core/agent-loop/src/inbox.ts similarity index 84% rename from packages/agent-loop/src/inbox.ts rename to packages/core/agent-loop/src/inbox.ts index 7aabad0166..a7b2e64e2c 100644 --- a/packages/agent-loop/src/inbox.ts +++ b/packages/core/agent-loop/src/inbox.ts @@ -52,6 +52,16 @@ export class Inbox { return this.steeringMessages.splice(0) } + /** + * Discard all pending messages (queued + steering) without delivering them — + * used by `cancel()`, which drops un-started work rather than draining it into + * a turn. Unlike `drainQueued`/`drainSteering`, the messages are thrown away. + */ + clear(): void { + this.queuedMessages.length = 0 + this.steeringMessages.length = 0 + } + /** Wait until a queued message arrives or `cancel` resolves. */ waitForQueued(cancel: Promise): Promise { if (this.hasQueued) return Promise.resolve() diff --git a/packages/agent-loop/src/index.ts b/packages/core/agent-loop/src/index.ts similarity index 58% rename from packages/agent-loop/src/index.ts rename to packages/core/agent-loop/src/index.ts index 71170db49a..90d641eeeb 100644 --- a/packages/agent-loop/src/index.ts +++ b/packages/core/agent-loop/src/index.ts @@ -1,5 +1,5 @@ /** - * THE concrete agent plugin: creates LoopAgents, runs their loops, and + * THE concrete agent plugin: creates ReactLoopAgents, runs their loops, and * registers them in ctx.agents. Deliberately thin — every behavior beyond * "call the model, run the tools, repeat" belongs to plugins on the event * taxonomy. @@ -10,17 +10,16 @@ import { Context, Service } from 'cordis' import { randomUUID } from 'node:crypto' import z from 'schemastery' -import { AgentId } from '@deepseek-ai/dsh-agent' -import type { Agent, AgentFactory, AgentOptions, CreateAgentOptions, ResumeAgentOptions } from '@deepseek-ai/dsh-agent' +import type { AgentFactory, AgentHandle, AgentId, AgentOptions, CreateAgentOptions, ResumeAgentOptions } from '@deepseek-ai/dsh-agent' import type {} from '@deepseek-ai/dsh-llm' import { SessionId } from '@deepseek-ai/dsh-session' import type { Session } from '@deepseek-ai/dsh-session' import type {} from '@deepseek-ai/dsh-system-prompt' import type {} from '@deepseek-ai/dsh-tools' import type { SessionPersistence } from '@deepseek-ai/dsh-session-persistence' -import { LoopAgent } from './agent.ts' +import { ReactLoopAgent } from './agent.ts' -export { LoopAgent } from './agent.ts' +export { ReactLoopAgent } from './agent.ts' export { Inbox, type InboxMessage } from './inbox.ts' export { runLoop } from './loop.ts' @@ -33,7 +32,7 @@ declare module 'cordis' { export interface Config { /** Agents created from configuration at startup. */ agents: (AgentOptions & { - id: string + id: AgentId /** * If set, the config agent RESUMES this persisted session id instead of * starting a fresh `${id}-session-`. Sourced from an env var in @@ -42,13 +41,17 @@ export interface Config { * `dsh-session-persistence` backend; the resume is deferred until that * service is available (via `ctx.inject`) and the loaded session's events * seed the live session so history continues. + * + * The schema accepts a plain string at runtime (cordis.yml values are + * untyped); the brand is compile-time only — the config format is the + * boundary where an id enters, so the TYPE declares the brand here. */ - resumeSessionId?: string + resumeSessionId?: SessionId })[] } /** - * The agent-loop plugin (`ctx.agentLoop`): creates {@link LoopAgent}s, runs + * The agent-loop plugin (`ctx.agentLoop`): creates {@link ReactLoopAgent}s, runs * their loops, and registers them in `ctx.agents`. Also implements the * {@link AgentFactory} seam, so plugins create/resume agents through * `ctx.agents` (the interface) without depending on this concrete package. @@ -60,14 +63,19 @@ export interface Config { export class AgentLoop extends Service implements AgentFactory { static inject = ['agents', 'sessions', 'llm', 'tools', 'systemPrompt'] - static Config: z = z.object({ + // The schema validates plain strings (cordis.yml config values are untyped at + // runtime); the {@link Config} TYPE declares the branded `id`/`resumeSessionId` + // because the config format is the boundary where an id enters. The brand is a + // zero-cost compile-time cast, so the runtime schema stays string-based and we + // assert the branded view once here — the single schema boundary. + static Config = z.object({ agents: z.array(z.object({ id: z.string().required(), model: z.string(), systemPrompt: z.string(), resumeSessionId: z.string(), })).default([]), - }) + }) as unknown as z constructor(ctx: Context, public config: Config) { super(ctx, 'agentLoop') @@ -118,25 +126,31 @@ export class AgentLoop extends Service implements AgentFactory { * fork seeds the new Session with the parent's event log, spawn starts * fresh; the child is returned as a regular Agent handle. */ - create(id: string, options: AgentOptions = {}): LoopAgent { + create(id: AgentId, options: AgentOptions = {}): ReactLoopAgent { this.assertAgentIdFree(id) - const session = this.ctx.sessions.create(`${id}-session-${randomUUID()}`, { meta: {} }) - return this.start(AgentId(id), options, session) + // Config/programmatic path: prepare the session and let start() fold its + // lifecycle into the agent's composite effect (so a fiber unload tears the + // session + agent down as one ordered chain, capturing the loop's closing + // flush). The whole effect is owned by THIS fiber; no AgentHandle is needed. + const session = this.ctx.sessions.prepare(SessionId(`${id}-session-${randomUUID()}`), { meta: {} }) + const { agent } = this.start(id, options, session) + return agent } /** * Programmatic factory create ({@link AgentFactory}): an agent on a * caller-supplied `sessionId` (NOT `${id}-session`), with optional session * metadata (validated `cwd`, lineage). The ACP bridge uses this so the - * client-generated session id becomes the live/persisted session id. + * client-generated session id becomes the live/persisted session id. Returns + * an {@link AgentHandle} the owner disposes to tear down exactly this agent. */ - createAgent(options: CreateAgentOptions): Agent { - // Check the agent id BEFORE creating the session: register() would reject a - // duplicate id only AFTER sessions.create(), leaving an orphaned live - // session (and lazy persistence state) that blocks reuse of that id. + createAgent(options: CreateAgentOptions): AgentHandle { + // Check the agent id BEFORE preparing the session: register() would reject a + // duplicate id only AFTER the session enters the store, leaving an orphaned + // live session (and lazy persistence state) that blocks reuse of that id. this.assertAgentIdFree(options.agentId) - const session = this.ctx.sessions.create(options.sessionId, { meta: options.meta ?? {} }) - return this.start(AgentId(options.agentId), options.agentOptions ?? {}, session) + const session = this.ctx.sessions.prepare(options.sessionId, { meta: options.meta ?? {} }) + return this.startOwned(options.agentId, options.agentOptions ?? {}, session) } /** @@ -151,7 +165,7 @@ export class AgentLoop extends Service implements AgentFactory { * forever) — callers that need resume (ACP) inject `sessionPersistence`, so * by the time this runs the service exists. */ - async resume(options: ResumeAgentOptions): Promise { + async resume(options: ResumeAgentOptions): Promise { // Read the service through `ctx.get('sessionPersistence')` — a direct // global-store lookup keyed by the isolate symbol — NOT // `this.ctx.sessionPersistence`. AgentLoop deliberately does NOT inject @@ -183,20 +197,21 @@ export class AgentLoop extends Service implements AgentFactory { * sessions store + registry are still read through `this.ctx` (both are in * AgentLoop's static inject, so they resolve fine). */ - private async resumeWith(persistence: SessionPersistence, options: ResumeAgentOptions): Promise { + private async resumeWith(persistence: SessionPersistence, options: ResumeAgentOptions): Promise { this.assertAgentIdFree(options.agentId) - const { meta, events } = await persistence.load(SessionId(options.resumeSessionId)) + const { meta, events } = await persistence.load(options.resumeSessionId) // Re-check the agent id AFTER the await: the pre-load check above can go // stale while load() is pending (a concurrent resume/create may register the - // same id). Re-checking immediately before sessions.create() keeps the + // same id). Re-checking immediately before prepare()/start keeps the // "no orphaned session on a duplicate id" guarantee under concurrency. this.assertAgentIdFree(options.agentId) // Reconstruct the live session with the FULL persisted header (createdAt, // cwd, lineage) so resume preserves identity, not just the cwd. The seed // events make lastTurnNumber/deriveMessages continue; the backend already // has state (cursor) from the load above, so onCreated is a no-op and the - // seed is not re-persisted. - const session = this.ctx.sessions.create(options.resumeSessionId, { + // seed is not re-persisted. prepare() (not create()) so the session + // lifecycle folds into the agent's composite effect (ordered teardown). + const session = this.ctx.sessions.prepare(options.resumeSessionId, { seed: events, meta: { createdAt: meta.createdAt, @@ -204,31 +219,81 @@ export class AgentLoop extends Service implements AgentFactory { ...meta.parentSession !== undefined ? { parentSession: meta.parentSession } : {}, }, }) - return this.start(AgentId(options.agentId), options.agentOptions ?? {}, session) + return this.startOwned(options.agentId, options.agentOptions ?? {}, session) } /** - * Reject a duplicate agent id BEFORE any session is created, so a failed - * factory call never leaves an orphaned live session (and lazy persistence - * state) behind. `register()` enforces the same uniqueness, but only after - * `sessions.create()` has already run. + * Reject a duplicate agent id BEFORE the session is entered into the store, so + * a failed factory call never leaves an orphaned live session (and lazy + * persistence state) behind. `register()` enforces the same uniqueness, but + * only after the session has already entered the store. */ - private assertAgentIdFree(id: string): void { + private assertAgentIdFree(id: AgentId): void { if (this.ctx.agents.get(id) !== undefined) { throw new Error(`agent "${id}" is already registered`) } } - /** Shared: construct a LoopAgent, register it, and start its loop (LIFO). */ - private start(id: AgentId, options: AgentOptions, session: Session): LoopAgent { - const agent = new LoopAgent(this.ctx, id, options, session) - // Generator effect: stop and unregister are independent disposables - // (LIFO), so a throwing stop() cannot leak the registry entry. - this.ctx.effect(function* (this: AgentLoop) { + /** + * Shared: construct a ReactLoopAgent over a PREPARED (not-yet-entered) + * session, then build the ONE composite effect that owns the whole agent + * lifecycle — session entry, registry registration, and the loop. Keeping all + * three in a SINGLE effect (not sibling effects) is load-bearing: a fiber + * unload disposes sibling effects CONCURRENTLY (`Promise.all`), which would + * race the session detach against the loop's closing flush and drop the + * closing `turn/end`. Inside one effect the disposers run as an ORDERED LIFO + * chain — the runtime awaits each disposer's returned promise before the next: + * + * yield session-detach (disposed LAST — detach onAppend + remove entry) + * yield register (disposed 2nd — unregister) + * yield stop-and-drain (disposed FIRST — request loop stop, await agent.done) + * + * So on teardown: the loop is stopped and AWAITED to exit (its final + * `session/flush` + `turn/end` fire through the still-attached `onAppend`), + * THEN the agent is unregistered, THEN the session is detached — capturing the + * closing events before detach, whether the trigger is the handle's `dispose()` + * OR a fiber unload. Rollback safety: each yield runs before the next mutation, + * so a throwing `session/created`/`agent/created` listener unwinds the + * already-yielded disposers instead of leaking. + * + * Returns the agent plus the composite effect's disposer (`disposeAgent`). + */ + private start(id: AgentId, options: AgentOptions, session: Session): { agent: ReactLoopAgent; disposeAgent: () => Promise } { + const agent = new ReactLoopAgent(this.ctx, id, options, session) + const dispose = this.ctx.effect(function* (this: AgentLoop) { + yield this.ctx.sessions.enter(session) + this.ctx.sessions.announce(session) yield this.ctx.agents.register(agent) - yield agent.start() + const stop = agent.start() + // Disposed FIRST (LIFO): request loop stop (sync), then AWAIT the loop's + // actual exit so its closing flush lands while onAppend (yielded above, + // disposed later) is still attached. + yield async () => { stop(); await agent.done } }.bind(this), 'agentLoop.start()') - return agent + return { agent, disposeAgent: async () => { await dispose() } } + } + + /** + * Build an {@link AgentHandle} for a PREPARED session + a fresh agent. The + * handle's `dispose()` runs the composite effect's disposer (see + * {@link start}) — which stops the loop, awaits its exit (final flush + * captured), unregisters the agent, and detaches the session, in that order. + * The same composite effect is what a fiber unload disposes, so both teardown + * triggers honor the ordering identically. + * + * `dispose()` is MEMOIZED: the underlying cordis effect disposer is + * single-shot (a second call returns immediately because the effect's epoch is + * already cleared, NOT awaiting the in-flight teardown), so concurrent/repeated + * `dispose()` calls would otherwise resolve before the first call's + * `await agent.done` + final flush completed. Memoizing the promise makes every + * caller observe the SAME quiescence boundary, honoring the + * `AgentHandle.dispose(): Promise` contract (mirrors the ACP `quiesce()` + * helper). + */ + private startOwned(id: AgentId, options: AgentOptions, session: Session): AgentHandle { + const { agent, disposeAgent } = this.start(id, options, session) + let disposing: Promise | undefined + return { agent, dispose: () => (disposing ??= disposeAgent()) } } } diff --git a/packages/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts similarity index 73% rename from packages/agent-loop/src/loop.ts rename to packages/core/agent-loop/src/loop.ts index 359329e062..8d19c464fd 100644 --- a/packages/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -13,7 +13,7 @@ import { BlockAssembler, HarnessError } from '@deepseek-ai/dsh-llm' import type { Session, TurnEndReason, TurnTrigger } from '@deepseek-ai/dsh-session' import { renderPrompt } from '@deepseek-ai/dsh-system-prompt' import type {} from '@deepseek-ai/dsh-tools' -import type { LoopAgent } from './agent.ts' +import type { ReactLoopAgent } from './agent.ts' /** An Error with an optional machine-readable code (e.g., from LlmError or a throwing plugin). */ type CodedError = Error & { code?: string } @@ -38,8 +38,8 @@ function toError(error: unknown): CodedError { * caller's try/catch), OR end the stream with a finish-error/aborted chunk * (the only option for adapters that can't throw mid-stream, e.g. * library-backed ones). This translates the latter into a thrown step error - * so the turn ends error/aborted with a logged `error` event, never as a - * normal `completed` assistant message. + * so the turn ends error/aborted (the failure recorded on `turn/end.reason`), + * never as a normal `completed` assistant message. * * `FinishReason` is merge-extensible (plugins/adapters can add `kind`s), so * the switch handles the known terminal-failure kinds and treats every other @@ -98,7 +98,7 @@ function stepFinishReason(finish: FinishReason): TurnEndReason | undefined { /** * Ambient handles the loop driver receives from the agent. Decouples the - * pure function `runLoop` from the mutable LoopAgent fields, making the + * pure function `runLoop` from the mutable ReactLoopAgent fields, making the * loop testable without a real agent. */ export interface LoopHandle { @@ -107,6 +107,34 @@ export interface LoopHandle { /** Resolves when the agent is disposed — unblocks the idle wait. */ disposed: Promise isDisposed(): boolean + /** + * Whether a `cancel()` is pending for the current turn. The driver checks this + * at every decision point where a turn could start or continue (right after + * the idle wait, after the `running` flip, before each step, and at the + * continuation gate) and drops the about-to-run / continuing turn. Reset once + * per loop iteration via {@link clearCancel} after the turn returns, so the + * marker governs exactly one cancellation and never leaks to a later prompt. + */ + isCancelled(): boolean + /** + * The resolved reason for the pending cancel (`reason ?? 'cancelled'`), read + * by the marker branches (pre-step / continuation) so a turn dropped where no + * `AbortController` carries the reason still records the caller's + * `cancel(reason)` value — matching the mid-step abort path. Only meaningful + * when {@link isCancelled} is true. + */ + cancelReason(): string + /** Clear the cancel marker (called once per iteration after the turn returns). */ + clearCancel(): void + /** + * Settle pending `whenIdle()` waiters WITHOUT a status transition. Used by the + * pre-step cancel-skip path: it drops the about-to-run turn and re-parks at the + * idle wait, so no `running→idle` transition fires to settle a `whenIdle()` + * waiter that was registered in the pre-step window — this settles it directly + * (it emits no `agent/status`, so an ACP `agent/status` listener never sees a + * spurious idle that would resolve a freshly-queued prompt as cancelled). + */ + settleIdle(): void } /** @@ -126,7 +154,7 @@ export interface LoopHandle { * stream ctx.llm.stream(req) ⟵ waterfall llm/stream (raw chunks) * session('assistant/chunk'); emit agent/stream-chunk * msg = waterfall agent/step-result ⟵ BEFORE the log append, so the - * session('assistant/message','usage') session records what actually ran + * session('assistant/message' {content, usage?}) session records what actually ran * each tool-call in msg (sequential, abort-checked): * session('tool/call'); ctx.tools.execute() ⟵ waterfall tools/execute * session('tool/result') @@ -141,14 +169,56 @@ export interface LoopHandle { * idle (emit agent/status) unless more queued * ``` */ -export async function runLoop(ctx: Context, agent: LoopAgent, handle: LoopHandle): Promise { +export async function runLoop(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle): Promise { const { session } = agent while (!handle.isDisposed()) { await agent.inbox.waitForQueued(handle.disposed) if (handle.isDisposed()) break + // Pre-step cancel (window 1): a `cancel()` landed after a `send()` woke the + // idle wait but before we flip to `running`. The cancelled queued/steering + // work is already cleared by `cancel()`. Clear the marker, then: + // - if NOTHING new is queued, drop the about-to-run turn and re-park, + // settling any `whenIdle()` waiter DIRECTLY (no running→idle transition + // fires here to settle it) and WITHOUT emitting `agent/status` (an ACP + // listener must not see a spurious idle that resolves a freshly-queued + // prompt as cancelled); + // - if a NEW prompt was queued AFTER the cancel (a send() that raced in + // before the loop resumed), the marker was for the cancelled work only — + // fall through and run the new prompt's turn. Do NOT settle waiters here: + // a whenIdle() waiter must wait for that new turn's running→idle, not + // resolve before it runs (the quiescence contract). + if (handle.isCancelled()) { + handle.clearCancel() + if (!agent.inbox.hasQueued) { + handle.settleIdle() + continue + } + } + handle.setStatus('running') + + // Pre-step cancel (window 2): `setStatus('running')` emits `agent/status` + // SYNCHRONOUSLY, so a `running` listener can `cancel()` in the gap between the + // check above and `runTurn`. Mirror window 1: clear the marker, then + // - if NOTHING new is queued, drop the about-to-run turn and transition + // back to `idle` (`running` was already emitted, so a real idle + // transition balances the status AND settles `whenIdle()` waiters); + // - if a NEW prompt was queued AFTER the cancel (a `running` listener that + // cancels then sends), the marker was for the cancelled work only — fall + // through and run the new prompt's turn (status is already `running`), so + // a `whenIdle()` waiter resolves on THAT turn's running→idle, not before + // it runs. Settling here would resolve quiescence while the replacement + // is still queued and unrun (the same early-resolve race window 1 fixes). + if (handle.isCancelled()) { + handle.clearCancel() + if (!agent.inbox.hasQueued) { + handle.setStatus('idle') + continue + } + } + // Re-derive the turn number from the log each iteration (do NOT keep a local // counter): an idle `agent.inject()` can append its own one-shot turn while // the loop waits above, so the next real turn must continue from whatever @@ -170,8 +240,18 @@ export async function runLoop(ctx: Context, agent: LoopAgent, handle: LoopHandle } catch { /* contained: a throwing agent/error listener must not kill the driver */ } } + // Reset the cancel marker UNCONDITIONALLY here, after the turn returns and + // before the next iteration's idle wait. NOT gated on the idle transition + // below: a `send()` that lands during the cancelled turn's flush window makes + // `hasQueued` true at the `setStatus('idle')` guard, so an idle-gated reset + // would never fire and the stale marker would wrongly drop that next prompt's + // turn. Resetting per iteration scopes the marker to exactly the turn that was + // cancelled. + handle.clearCancel() + // Steering that arrived too late to join this turn (turn-end listeners, - // flush) becomes a queued message — it must never be stranded. + // flush) becomes a queued message — it must never be stranded. (A cancelled + // turn already cleared its steering, so there is nothing to re-enqueue.) for (const message of agent.inbox.drainSteering()) { agent.inbox.enqueue(message) } @@ -180,7 +260,7 @@ export async function runLoop(ctx: Context, agent: LoopAgent, handle: LoopHandle } } -async function runTurn(ctx: Context, agent: LoopAgent, handle: LoopHandle, turn: number): Promise { +async function runTurn(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle, turn: number): Promise { const { session } = agent // --- Pre-turn. A throw here (the invariant guard) is owed NO turn/end — @@ -233,41 +313,31 @@ async function runTurn(ctx: Context, agent: LoopAgent, handle: LoopHandle, turn: return false } - // Record a step/turn failure exactly once: append the single `error` event - // (only while the turn is still open — see below), set the error reason, and - // emit agent/error (contained — trap: a throwing agent/error listener must not - // re-escape and strand the turn). Disposal and abort set `reason` directly - // without calling this (no `error` event for those — they are not failures). + // Record a step/turn failure exactly once: set the error reason (carrying the + // failing `step` — the durable failure lives entirely on turn/end.reason, there + // is no separate session error event) and emit agent/error (contained — trap: a + // throwing agent/error listener must not re-escape and strand the turn). + // Disposal and abort set `reason` directly without calling this (they are not + // failures). const failTurn = (err: CodedError): void => { if (errorReported) return errorReported = true - // Only append the session `error` INSIDE the turn (before turn/end). If the - // turn has already ended — the only way here is a throwing agent/turn-end - // listener after closeTurn(true) already appended turn/end — appending now - // would land the error AFTER the last turn/end, where the persistence - // backend treats it as a crash tail and drops it on resume (the turn-enclosure RFC). In - // that case report via agent/error + the logger only; the turn is balanced. + // Set the error reason ONLY while the turn is still open — closeTurn appends + // turn/end with it. If the turn has already ended (the only way here: a + // throwing agent/turn-end listener after closeTurn(true) already appended + // turn/end), the reason can no longer affect the durable log, so log the late + // throw directly instead — otherwise the listener exception would vanish. if (!turnEnded) { - // Set `reason` BEFORE the append: Session.append pushes the error event - // before notifying session/event listeners, so a throwing listener would - // otherwise leave `reason` unset (and closeTurn would record the wrong - // reason / the outer catch would skip closeTurn). The append is contained - // — the error event is already in the log either way; a throwing listener - // must not abort finalization. - reason = { kind: 'error', ...errorData(err) } - try { - session.append('error', { turn, step, ...errorData(err) }) - } catch (appendError: unknown) { - ctx.logger.warn(`agent "${agent.id}": session/event listener threw on the error event at turn ${turn}: ${toError(appendError).message}`) - } + reason = { kind: 'error', step, ...errorData(err) } } else { ctx.logger.warn(`agent "${agent.id}": agent/turn-end listener threw after turn ${turn} closed: ${err.message}`) } try { ctx.emit('agent/error', agent, turn, step, err) } catch { - // contained: the error is already logged; a throwing agent/error - // listener must not prevent the turn from closing. + // contained: the error is already captured (on `reason`, or via the logger + // above); a throwing agent/error listener must not prevent the turn from + // closing. } } @@ -323,6 +393,20 @@ async function runTurn(ctx: Context, agent: LoopAgent, handle: LoopHandle, turn: const abort = new AbortController() handle.setAbort(abort) + // Cancel landing in the step-start window: a synchronous `agent/turn-start` + // or `agent/step-start` listener (both fire before this point) can have + // called `cancel()`, and `runStep` would otherwise run a full extra step + // with no AbortController having observed it. Check the marker AFTER + // setAbort (so the next-iteration drain sees a clean controller) and before + // `runStep`: drop the step, end the turn `aborted`. closeStep balances the + // already-appended step/start. + if (handle.isCancelled()) { + handle.setAbort(undefined) + reason = { kind: 'aborted', reason: handle.cancelReason() } + closeStep() + break + } + let stepOutcome: { hadToolCalls: boolean; finish: FinishReason } | { error: Error } try { stepOutcome = await runStep(ctx, agent, turn, step, abort.signal) @@ -341,7 +425,7 @@ async function runTurn(ctx: Context, agent: LoopAgent, handle: LoopHandle, turn: if (handle.isDisposed()) { reason = { kind: 'disposed' } } else if (abort.signal.aborted) { - /* v8 ignore next -- abort.signal.reason always set by agent.abort() which provides a default */ + /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ reason = { kind: 'aborted', reason: String(abort.signal.reason ?? 'aborted') } } else { failTurn(error) @@ -382,6 +466,16 @@ async function runTurn(ctx: Context, agent: LoopAgent, handle: LoopHandle, turn: // next iteration's drain records it. if (!shouldContinue && agent.inbox.hasSteering) shouldContinue = true + // A cancel that landed during the continuation window — after the step's + // AbortController was cleared (setAbort(undefined)) but before the next + // step starts — has no controller to observe it, so the turn-scoped marker + // ends the turn here. cancel() also cleared the steering FIFO, so the + // override above did not re-arm continuation. + if (handle.isCancelled()) { + reason = { kind: 'aborted', reason: handle.cancelReason() } + break + } + if (!shouldContinue || handle.isDisposed()) { /* v8 ignore next -- disposal during continuation-decision window is a narrow race; error-path disposal is covered elsewhere */ if (handle.isDisposed()) reason = { kind: 'disposed' } @@ -446,7 +540,7 @@ async function runTurn(ctx: Context, agent: LoopAgent, handle: LoopHandle, turn: } /** Drain the steering queue into the session. Returns whether any arrived. */ -function drainSteering(ctx: Context, agent: LoopAgent, turn: number): boolean { +function drainSteering(ctx: Context, agent: ReactLoopAgent, turn: number): boolean { const messages = agent.inbox.drainSteering() for (const message of messages) { agent.session.append('steering/message', { turn, content: message.content, source: message.source }) @@ -458,7 +552,7 @@ function drainSteering(ctx: Context, agent: LoopAgent, turn: number): boolean { /** One step: assemble request → stream model → record → execute tools. */ async function runStep( ctx: Context, - agent: LoopAgent, + agent: ReactLoopAgent, turn: number, step: number, signal: AbortSignal, @@ -486,7 +580,7 @@ async function runStep( // --- Model call (streaming-first; raw chunks are the replay record) --- const assembler = new BlockAssembler() for await (const chunk of ctx.llm.stream(request)) { - /* v8 ignore next -- signal.reason always set by agent.abort() which provides a default */ + /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) session.append('assistant/chunk', { turn, step, chunk }) ctx.emit('agent/stream-chunk', agent, turn, step, chunk) @@ -504,11 +598,14 @@ async function runStep( if (assembler.finish.kind === 'max-tokens') { let message: Message = withoutToolCalls(assembler.message()) message = withoutToolCalls(await ctx.waterfall('agent/step-result', agent, turn, step, message, () => Promise.resolve(message))) - if (message.content.length > 0) { - session.append('assistant/message', { turn, step, content: message.content }) - } - if (assembler.usage) { - session.append('usage', { turn, step, usage: assembler.usage }) + // Fire the assistant/message when there is content OR usage: a max-tokens + // step can be cut off with empty content but still carry token accounting, + // and assistant/message is the only host for usage (there is no standalone + // usage event). An empty-content assistant/message is skipped by + // deriveMessages(), so hosting usage on it never injects a spurious assistant + // turn into derived history. + if (message.content.length > 0 || assembler.usage) { + session.append('assistant/message', { turn, step, content: message.content, ...(assembler.usage ? { usage: assembler.usage } : {}) }) } return { hadToolCalls: false, finish: assembler.finish } } @@ -519,9 +616,13 @@ async function runStep( let message: Message = assembler.message() message = await ctx.waterfall('agent/step-result', agent, turn, step, message, () => Promise.resolve(message)) - session.append('assistant/message', { turn, step, content: message.content }) - if (assembler.usage) { - session.append('usage', { turn, step, usage: assembler.usage }) + // Same content-or-usage guard as the max-tokens branch: a step that finishes + // with neither assembled content nor usage (e.g. a bare `stop` finish that + // streamed nothing) records no assistant/message — an empty-content message + // exists only to host usage, and deriveMessages() skips it either way, so + // appending one with no usage would be a pure trace-only row. + if (message.content.length > 0 || assembler.usage) { + session.append('assistant/message', { turn, step, content: message.content, ...(assembler.usage ? { usage: assembler.usage } : {}) }) } // --- Tool execution (sequential; parallel execution is a TODO) --- @@ -529,7 +630,7 @@ async function runStep( // isError results, so abort is re-checked around every call here. const toolCalls = message.content.filter(block => block.type === 'tool-call') for (const call of toolCalls) { - /* v8 ignore next -- signal.reason always set by agent.abort() which provides a default */ + /* v8 ignore next -- signal.reason always set: cancel()/disposal provide a default */ if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) session.append('tool/call', { turn, step, callId: call.id, name: call.name, arguments: call.arguments }) let parsedArguments: unknown @@ -560,7 +661,7 @@ async function runStep( }) // signal CAN flip during the await above (abort() inside a tool); // the analyzer can't see through the await boundary. - /* v8 ignore start -- signal.reason default unreachable via agent.abort() */ + /* v8 ignore start -- signal.reason default unreachable: cancel()/disposal always set it */ // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition if (signal.aborted) throw new Error(String(signal.reason ?? 'aborted')) /* v8 ignore stop */ diff --git a/packages/agent-loop/tests/agent.spec.ts b/packages/core/agent-loop/tests/agent.spec.ts similarity index 86% rename from packages/agent-loop/tests/agent.spec.ts rename to packages/core/agent-loop/tests/agent.spec.ts index 4fd7237061..ed163bf4c2 100644 --- a/packages/agent-loop/tests/agent.spec.ts +++ b/packages/core/agent-loop/tests/agent.spec.ts @@ -2,11 +2,11 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import { AgentId } from '@deepseek-ai/dsh-agent' import LlmService from '@deepseek-ai/dsh-llm' -import SessionStore from '@deepseek-ai/dsh-session' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' import AgentRegistry from '@deepseek-ai/dsh-agent' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { MockAdapter, textResponse } from './mock-adapter.ts' async function harness(adapter: MockAdapter) { @@ -21,7 +21,7 @@ async function harness(adapter: MockAdapter) { return ctx } -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { @@ -32,7 +32,7 @@ function waitForIdle(ctx: Context, agent: LoopAgent): Promise { }) } -function waitForStatus(ctx: Context, agent: LoopAgent, expected: LoopAgent['status']): Promise { +function waitForStatus(ctx: Context, agent: ReactLoopAgent, expected: ReactLoopAgent['status']): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === expected) { @@ -43,17 +43,17 @@ function waitForStatus(ctx: Context, agent: LoopAgent, expected: LoopAgent['stat }) } -function send(agent: LoopAgent, text: string) { +function send(agent: ReactLoopAgent, text: string) { agent.send([{ type: 'text', text }]) } -describe('LoopAgent', () => { +describe('ReactLoopAgent', () => { it('send() throws after disposal', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) @@ -66,9 +66,9 @@ describe('LoopAgent', () => { it('steer() throws after disposal', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) @@ -81,9 +81,9 @@ describe('LoopAgent', () => { it('inject() throws after disposal', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) @@ -96,7 +96,7 @@ describe('LoopAgent', () => { it('inject() decides enclosure from the LOG (open turn), not agent status', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // Simulate an OPEN turn in the log while the agent is idle (status is not a // reliable open-turn signal). inject must append into that open turn, NOT @@ -122,7 +122,7 @@ describe('LoopAgent', () => { // A persistence-like listener whose flush rejects. ctx.on('session/flush', () => { throw new Error('disk gone') }) const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // inject() is synchronous and fires a fire-and-forget flush; a rejecting // flush must be contained (logged), never thrown into the caller. @@ -135,7 +135,7 @@ describe('LoopAgent', () => { it('idle inject() closes its one-shot turn AND still checkpoints even if the append throws', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let flushes = 0 ctx.on('session/flush', () => { flushes += 1 }) @@ -155,7 +155,7 @@ describe('LoopAgent', () => { it('idle inject() still checkpoints when a listener throws on the synthetic turn/end', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let flushes = 0 ctx.on('session/flush', () => { flushes += 1 }) // A session/event listener that throws on the synthetic turn/end. Append @@ -180,7 +180,7 @@ describe('LoopAgent', () => { // A non-Error rejection exercises the String() normalization branch. ctx.on('session/flush', () => { throw 'disk gone' }) const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const errors: { turn: number; step: number; message: string }[] = [] ctx.on('agent/error', (_a, turn, step, error) => void errors.push({ turn, step, message: error.message })) @@ -199,7 +199,7 @@ describe('LoopAgent', () => { it('idle inject() with a non-serializable source opens no turn (nothing to close)', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // A non-serializable source makes the turn/start append throw BEFORE the // event is pushed (Session.append validates before push), so NO turn opens. @@ -214,7 +214,7 @@ describe('LoopAgent', () => { it('steer() when idle falls through to send() and starts a turn', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // steer while idle delegates to send agent.steer([{ type: 'text', text: 'steer idle' }], { source: { kind: 'plugin', plugin: 'test' } }) @@ -226,12 +226,12 @@ describe('LoopAgent', () => { }) it('disposer is idempotent (double-stop)', async () => { - // Create a bare LoopAgent and call start() directly to get the disposer. + // Create a bare ReactLoopAgent and call start() directly to get the disposer. // Then call it twice — the second call hits the early-return branch. const ctx = new Context() await ctx.plugin(SessionStore) - const session = ctx.sessions.create('test') - const agent = new LoopAgent(ctx, AgentId('bare'), { model: 'mock' }, session) + const session = ctx.sessions.create(SessionId('test')) + const agent = new ReactLoopAgent(ctx, AgentId('bare'), { model: 'mock' }, session) // Start the loop to get the disposer; the agent waits for messages // (idle, never-resolving cancel), so it will stay idle. @@ -249,7 +249,7 @@ describe('LoopAgent', () => { it('setting the same status does not emit agent/status again', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const statuses: string[] = [] ctx.on('agent/status', (subject, status) => { @@ -268,7 +268,7 @@ describe('LoopAgent', () => { it('whenIdle() resolves immediately when the agent is not running', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // Fresh agent is idle — whenIdle() takes the not-running fast path and // resolves without subscribing. await must not hang. @@ -279,7 +279,7 @@ describe('LoopAgent', () => { it('whenIdle() waits for queued work that has not flipped status yet', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'queued') let settled = false @@ -288,7 +288,7 @@ describe('LoopAgent', () => { expect(settled).toBe(false) await waitForStatus(ctx, agent, 'running') - agent.abort('done') + agent.cancel('done') await idle expect(settled).toBe(true) expect(agent.status).toBe('idle') @@ -297,8 +297,8 @@ describe('LoopAgent', () => { it('whenIdle() awaits the running→idle transition, ignoring other subjects/running events', async () => { const adapter = new MockAdapter([textResponse('ok'), textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) - const other = ctx.agentLoop.create('a2', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + const other = ctx.agentLoop.create(AgentId('a2'), { model: 'mock' }) // Drive `agent` into `running`, then await whenIdle() — it subscribes to // agent/status and resolves on the first transition out of running. @@ -323,7 +323,7 @@ describe('LoopAgent', () => { it('whenIdle() subscribed while running resolves via done when the agent is then disposed', async () => { // Covers the waiter's disposed arm: whenIdle() queues an internal waiter // while running (not the fast path), then the disposer settles it and chains - // `done` (loop exit), not an eager resolve. A bare LoopAgent + direct + // `done` (loop exit), not an eager resolve. A bare ReactLoopAgent + direct // start() disposer keeps the emit synchronous. const ctx = new Context() await ctx.plugin(LlmService) @@ -333,8 +333,8 @@ describe('LoopAgent', () => { await ctx.plugin(AgentRegistry) const adapter = new MockAdapter(['hang']) ctx.llm.registerAdapter(['mock'], adapter) - const session = ctx.sessions.create('bare') - const agent = new LoopAgent(ctx, AgentId('bare'), { model: 'mock' }, session) + const session = ctx.sessions.create(SessionId('bare')) + const agent = new ReactLoopAgent(ctx, AgentId('bare'), { model: 'mock' }, session) const dispose = agent.start() agent.send([{ type: 'text', text: 'go' }]) await new Promise(r => setTimeout(r, 30)) @@ -355,9 +355,9 @@ describe('LoopAgent', () => { // it. Regression for the round-3 whenIdle finding. const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) @@ -376,9 +376,9 @@ describe('LoopAgent', () => { // only after `done` — i.e. the loop has actually exited. const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) @@ -399,7 +399,7 @@ describe('LoopAgent', () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.on('agent/status', (_subject, status) => { if (status === 'running') throw new Error('bad running listener') }) @@ -417,7 +417,7 @@ describe('LoopAgent', () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.on('agent/status', (_subject, status) => { if (status === 'idle') throw new Error('bad idle listener') }) @@ -430,20 +430,4 @@ describe('LoopAgent', () => { expect(warn).toHaveBeenCalledWith(expect.stringContaining('agent/status listener threw on idle')) warn.mockRestore() }) - - it('abort() resolves reason to "aborted" when no reason provided', async () => { - const adapter = new MockAdapter(['hang']) - const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) - - const reasons: { kind: string; reason?: string }[] = [] - ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) - - send(agent, 'go') - await new Promise(r => setTimeout(r, 30)) - agent.abort() // no reason string - await waitForIdle(ctx, agent) - - expect(reasons[0]).toMatchObject({ kind: 'aborted', reason: 'aborted' }) - }) }) diff --git a/packages/core/agent-loop/tests/cancel.spec.ts b/packages/core/agent-loop/tests/cancel.spec.ts new file mode 100644 index 0000000000..9cdaa1973b --- /dev/null +++ b/packages/core/agent-loop/tests/cancel.spec.ts @@ -0,0 +1,338 @@ +/** + * Tests for the queue-aware `Agent.cancel()` primitive. `cancel()` is the + * broad verb — it clears queued + steering work, aborts an in-flight step, and + * drops a turn about to start — whereas a bare step abort (the loop's private + * `AbortController`) kills only the current step and leaves the queue intact. + * These tests exercise every window where a cancel can land (idle, pre-step, + * mid-step, continuation) and the marker's arm/reset rules that keep a cancel + * from leaking to a later prompt or hanging `whenIdle()`. + * + * @module dsh-agent-loop/tests/cancel + */ + +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import LlmService from '@deepseek-ai/dsh-llm' +import SessionStore, { TurnEndReason } from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' +import { MockAdapter, textResponse } from './mock-adapter.ts' + +async function harness(adapter: MockAdapter) { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + ctx.llm.registerAdapter(['mock'], adapter) + return ctx +} + +function send(agent: ReactLoopAgent, text: string) { + agent.send([{ type: 'text', text }]) +} + +/** Resolve on the agent's next idle transition (event-based, not status poll). */ +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { + return new Promise((resolve) => { + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'idle') { dispose(); resolve() } + }) + }) +} + +/** All user-message texts recorded in the log (to assert what actually ran). */ +function userTexts(agent: ReactLoopAgent): string[] { + return agent.session.events + .filter(e => e.type === 'user/message') + .flatMap(e => e.type === 'user/message' ? e.data.content : []) + .flatMap(b => b.type === 'text' ? [b.text] : []) +} + +describe('Agent.cancel()', () => { + it('cancel() on an idle agent with nothing queued is a no-op; the next prompt runs (F2 leak guard)', async () => { + const adapter = new MockAdapter([textResponse('reply')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + // The loop is parked at the idle wait with nothing queued. A cancel here must + // NOT arm the marker — otherwise the next legitimate prompt would be dropped. + agent.cancel('nothing to cancel') + + send(agent, 'real prompt') + await waitForIdle(ctx, agent) + + // The prompt ran: its user message is in the log and one turn completed. + expect(userTexts(agent)).toEqual(['real prompt']) + expect(agent.session.events.some(e => e.type === 'turn/end')).toBe(true) + }) + + it('pre-step cancel drops the about-to-start turn (no turn is opened)', async () => { + const adapter = new MockAdapter([textResponse('should not run')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + // send() queues synchronously (status still idle, loop microtask not yet + // resumed). Cancel in that pre-step window: the queued turn must not run. + send(agent, 'drop me') + agent.cancel('pre-step') + + // Give the loop a chance to wake and process the cancel. + await new Promise(r => setTimeout(r, 30)) + + // No turn was opened — the queued prompt was dropped, never recorded. + expect(userTexts(agent)).toEqual([]) + expect(agent.session.events.some(e => e.type === 'turn/start')).toBe(false) + expect(agent.status).toBe('idle') + }) + + it('a whenIdle() waiter registered BEFORE a pre-step cancel resolves (F1 hang guard)', async () => { + const adapter = new MockAdapter([textResponse('x')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + // Queue work, then register a whenIdle() waiter while in the pre-step window + // (status idle, hasQueued true) — it does NOT take the fast path. Then cancel. + // The skip path must settle this waiter directly (no running→idle transition + // ever fires), or it would hang forever. + send(agent, 'q') + const idle = agent.whenIdle() + agent.cancel('pre-step') + + // Must resolve (not hang). A timeout makes the failure a clear test failure. + await Promise.race([ + idle, + new Promise((_r, reject) => setTimeout(() => { reject(new Error('whenIdle hung after pre-step cancel')) }, 1000)), + ]) + expect(agent.status).toBe('idle') + }) + + it('cancel() mid-step aborts the in-flight model call; the turn ends aborted', async () => { + const adapter = new MockAdapter(['hang']) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + const reasons: TurnEndReason[] = [] + ctx.on('agent/turn-end', (_a, _t, reason) => void reasons.push(reason)) + + send(agent, 'go') + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('running') + agent.cancel('mid-step') + await waitForIdle(ctx, agent) + + expect(reasons).toEqual([{ kind: 'aborted', reason: 'mid-step' }]) + }) + + it('cancel() with no reason defaults to "cancelled" when aborting an in-flight step', async () => { + const adapter = new MockAdapter(['hang']) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + const reasons: TurnEndReason[] = [] + ctx.on('agent/turn-end', (_a, _t, reason) => void reasons.push(reason)) + + send(agent, 'go') + await new Promise(r => setTimeout(r, 30)) + agent.cancel() // no reason → default 'cancelled' + await waitForIdle(ctx, agent) + + expect(reasons).toEqual([{ kind: 'aborted', reason: 'cancelled' }]) + }) + + it('a prompt sent AFTER a cancelled turn settles runs normally (marker reset)', async () => { + const adapter = new MockAdapter(['hang', textResponse('second reply')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + // First turn hangs; cancel it mid-step. + send(agent, 'first') + await new Promise(r => setTimeout(r, 30)) + agent.cancel('cancel first') + await waitForIdle(ctx, agent) + + // The marker must have been reset after the cancelled turn — a fresh prompt + // runs to completion rather than being dropped by a stale marker. + send(agent, 'second') + await waitForIdle(ctx, agent) + + expect(userTexts(agent)).toContain('second') + // The second turn completed (its reply was streamed). + const reasons = agent.session.events.filter(e => e.type === 'turn/end') + expect(reasons.length).toBe(2) + }) + + it('cancel from a synchronous agent/turn-start listener drops the step (step-start window)', async () => { + const adapter = new MockAdapter([textResponse('should not stream')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + // A turn-start listener fires BEFORE any AbortController is installed for the + // step. Cancelling there must still drop the step (the turn-scoped marker, + // not the step AbortController, is what catches this) — no model step runs. + let streamed = false + ctx.on('agent/stream-chunk', () => { streamed = true }) + const dispose = ctx.on('agent/turn-start', (subject) => { + if (subject === agent) agent.cancel('from turn-start') + }) + + const reasons: TurnEndReason[] = [] + ctx.on('agent/turn-end', (_a, _t, reason) => void reasons.push(reason)) + + send(agent, 'go') + await waitForIdle(ctx, agent) + dispose() + + // No step streamed (the model never ran), and the turn ended aborted with + // the CALLER's reason — the marker carries `cancel(reason)` through even + // though no AbortController observed it in this window. + expect(streamed).toBe(false) + expect(reasons).toEqual([{ kind: 'aborted', reason: 'from turn-start' }]) + }) + + it('cancel during the continuation window ends the turn aborted and runs no further step', async () => { + // A continuation-waterfall listener cancels DURING the continuation decision + // (the finished step's AbortController is already cleared), and votes to + // continue — but the turn-scoped marker checked right after must end the turn + // `aborted` and run NO second step. + const adapter = new MockAdapter([textResponse('one'), textResponse('two')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + let steps = 0 + ctx.on('agent/step-start', () => { steps += 1 }) + const reasons: TurnEndReason[] = [] + ctx.on('agent/turn-end', (_a, _t, reason) => void reasons.push(reason)) + + let continued = false + ctx.on('agent/turn-continuation', async (subject, _turn, _default, next) => { + if (subject === agent && !continued) { + continued = true + agent.cancel('from continuation') + return true // vote to continue — the post-waterfall marker check must override + } + return next() + }) + + send(agent, 'go') + await waitForIdle(ctx, agent) + + // Only ONE step ran (the second was cancelled in the continuation window), + // and the turn ended aborted with the CALLER's reason (carried by the + // marker, since the finished step's AbortController was already cleared). + expect(steps).toBe(1) + expect(reasons).toEqual([{ kind: 'aborted', reason: 'from continuation' }]) + }) + + it('cancel from a synchronous agent/status(running) listener drops the turn (window 2)', async () => { + const adapter = new MockAdapter([textResponse('should not run')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + // setStatus('running') emits agent/status SYNCHRONOUSLY, so a running + // listener can cancel in the gap between the loop's pre-step check and + // runTurn. The second check (after the running flip) must drop the turn — + // runTurn would otherwise throw on the now-empty queue. + let streamed = false + ctx.on('agent/stream-chunk', () => { streamed = true }) + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'running') agent.cancel('from running listener') + }) + + send(agent, 'go') + await waitForIdle(ctx, agent) + dispose() + + // No turn opened, no step streamed, and a later prompt still runs (the marker + // was reset). + expect(streamed).toBe(false) + expect(agent.session.events.some(e => e.type === 'turn/start')).toBe(false) + }) + + it('window 2: whenIdle() does NOT resolve early when a running listener cancels then queues replacement work', async () => { + // The window-1 early-resolve race has a window-2 twin: a synchronous + // agent/status('running') listener cancels the about-to-run turn AND queues a + // replacement. window 2 must NOT settle waiters (via setStatus('idle')) while + // the replacement is still queued-and-unrun — it must fall through and run it, + // so whenIdle() resolves on the replacement turn's running→idle, not before. + const adapter = new MockAdapter([textResponse('A reply'), textResponse('B reply')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + let replaced = false + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject !== agent || status !== 'running' || replaced) return + replaced = true + agent.cancel('drop A') + send(agent, 'B') + }) + + send(agent, 'A') + const idle = agent.whenIdle() + await idle + dispose() + + // whenIdle() resolved only AFTER B's turn ran: B's user message + a turn/end + // are in the log, and A was dropped. + expect(userTexts(agent)).toContain('B') + expect(userTexts(agent)).not.toContain('A') + expect(agent.session.events.some(e => e.type === 'turn/end')).toBe(true) + }) + + it('whenIdle() does NOT resolve early when a new prompt is queued during a pre-step cancel', async () => { + // The subtle race: a whenIdle() waiter is registered for prompt A; cancel() + // clears A; prompt B is queued BEFORE the loop resumes from the idle wait. + // The window-1 cancel branch must NOT settle the waiter while B is still + // queued-and-unrun — whenIdle() must wait for B's turn to actually run and + // settle (the quiescence contract), not resolve before B's first event. + const adapter = new MockAdapter([textResponse('A reply'), textResponse('B reply')]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + send(agent, 'A') // queues A (status still idle, loop microtask pending) + const idle = agent.whenIdle() // registers a waiter (idle + hasQueued → no fast path) + agent.cancel('drop A') // arms marker, clears A + send(agent, 'B') // B races in before the loop resumes + + // whenIdle() must resolve only AFTER B's turn fully ran — by which point B's + // user message and a turn/end are in the log. (Before the fix it resolved + // immediately, with zero events, then B ran afterward.) + await idle + expect(userTexts(agent)).toContain('B') + expect(agent.session.events.some(e => e.type === 'turn/end')).toBe(true) + // A was dropped (never ran); only B's turn is recorded. + expect(userTexts(agent)).not.toContain('A') + }) + + it("cancel clears the turn's steering — it is not re-enqueued as a fresh turn", async () => { + const adapter = new MockAdapter(['hang']) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + send(agent, 'go') + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('running') + // Steer (joins the running turn's steering FIFO), then cancel: the steering + // must be dropped, NOT re-enqueued as a new queued turn. + agent.steer([{ type: 'text', text: 'steer text' }]) + agent.cancel('cancel with steering') + await waitForIdle(ctx, agent) + + // After the cancelled turn settles, the agent is idle with NO follow-up turn + // started from the dropped steering. + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('idle') + const turnStarts = agent.session.events.filter(e => e.type === 'turn/start') + expect(turnStarts.length).toBe(1) // only the original (cancelled) turn + // The steering text was dropped — it never reached the log. + const flat = agent.session.events + .filter(e => e.type === 'steering/message') + .flatMap(e => e.type === 'steering/message' ? e.data.content : []) + .flatMap(b => b.type === 'text' ? [b.text] : []) + expect(flat).not.toContain('steer text') + }) +}) diff --git a/packages/agent-loop/tests/config-session-id.spec.ts b/packages/core/agent-loop/tests/config-session-id.spec.ts similarity index 82% rename from packages/agent-loop/tests/config-session-id.spec.ts rename to packages/core/agent-loop/tests/config-session-id.spec.ts index 52f0223564..8cf5bd81f8 100644 --- a/packages/agent-loop/tests/config-session-id.spec.ts +++ b/packages/core/agent-loop/tests/config-session-id.spec.ts @@ -4,18 +4,18 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import LlmService from '@deepseek-ai/dsh-llm' -import SessionStore from '@deepseek-ai/dsh-session' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import AgentRegistry from '@deepseek-ai/dsh-agent' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { MockAdapter, textResponse } from './mock-adapter.ts' const dirs: string[] = [] afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { dispose(); resolve() } @@ -35,10 +35,10 @@ describe('config-driven session id', () => { await ctx1.plugin(SystemPrompt) await ctx1.plugin(ToolRegistry) await ctx1.plugin(AgentRegistry) - await ctx1.plugin(AgentLoop, { agents: [{ id: 'cfg', model: 'mock', systemPrompt: '' }] }) + await ctx1.plugin(AgentLoop, { agents: [{ id: AgentId('cfg'), model: 'mock', systemPrompt: '' }] }) await ctx1.plugin(SessionPersistenceJsonl, { root }) ctx1.llm.registerAdapter(['mock'], new MockAdapter([textResponse('cfg')])) - const a1 = ctx1.agents.get('cfg') as LoopAgent + const a1 = ctx1.agents.get(AgentId('cfg')) as ReactLoopAgent expect(a1.session.id).toMatch(idPattern) a1.send([{ type: 'text', text: 'q' }], { source: { kind: 'user' } }) await waitForIdle(ctx1, a1) @@ -52,10 +52,10 @@ describe('config-driven session id', () => { await ctx2.plugin(SystemPrompt) await ctx2.plugin(ToolRegistry) await ctx2.plugin(AgentRegistry) - await ctx2.plugin(AgentLoop, { agents: [{ id: 'cfg', model: 'mock', systemPrompt: '' }] }) + await ctx2.plugin(AgentLoop, { agents: [{ id: AgentId('cfg'), model: 'mock', systemPrompt: '' }] }) await ctx2.plugin(SessionPersistenceJsonl, { root }) ctx2.llm.registerAdapter(['mock'], new MockAdapter([textResponse('cfg2')])) - const a2 = ctx2.agents.get('cfg') as LoopAgent + const a2 = ctx2.agents.get(AgentId('cfg')) as ReactLoopAgent expect(a2.session.id).toMatch(idPattern) expect(a2.session.id).not.toBe(a1.session.id) a2.send([{ type: 'text', text: 'q2' }], { source: { kind: 'user' } }) @@ -78,7 +78,7 @@ describe('config-driven session id', () => { await ctx1.plugin(AgentLoop, { agents: [] }) await ctx1.plugin(SessionPersistenceJsonl, { root }) ctx1.llm.registerAdapter(['mock'], new MockAdapter([textResponse('first')])) - const a1 = ctx1.agents.create({ agentId: 'main', sessionId: 'sticky-1' }) as LoopAgent + const a1 = ctx1.agents.create({ agentId: AgentId('main'), sessionId: SessionId('sticky-1') }).agent as ReactLoopAgent a1.send([{ type: 'text', text: 'remember me' }], { source: { kind: 'user' } }) await waitForIdle(ctx1, a1) await ctx1.fiber.dispose() @@ -92,15 +92,15 @@ describe('config-driven session id', () => { await ctx2.plugin(SystemPrompt) await ctx2.plugin(ToolRegistry) await ctx2.plugin(AgentRegistry) - await ctx2.plugin(AgentLoop, { agents: [{ id: 'main', model: 'mock', systemPrompt: '', resumeSessionId: 'sticky-1' }] }) + await ctx2.plugin(AgentLoop, { agents: [{ id: AgentId('main'), model: 'mock', systemPrompt: '', resumeSessionId: SessionId('sticky-1') }] }) await ctx2.plugin(SessionPersistenceJsonl, { root }) ctx2.llm.registerAdapter(['mock'], new MockAdapter([textResponse('second')])) // The deferred resume runs on a microtask after the backend is available. - let resumed: LoopAgent | undefined + let resumed: ReactLoopAgent | undefined for (let i = 0; i < 50 && !resumed; i++) { await new Promise(r => setTimeout(r, 5)) - resumed = ctx2.agents.get('main') as LoopAgent | undefined + resumed = ctx2.agents.get(AgentId('main')) as ReactLoopAgent | undefined } expect(resumed).toBeDefined() // The live session id IS the resumed id (NOT a fresh ${id}-session-), @@ -120,7 +120,7 @@ describe('config-driven session id', () => { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(AgentRegistry) - await ctx.plugin(AgentLoop, { agents: [{ id: 'main', model: 'mock', systemPrompt: '', resumeSessionId: 'does-not-exist' }] }) + await ctx.plugin(AgentLoop, { agents: [{ id: AgentId('main'), model: 'mock', systemPrompt: '', resumeSessionId: SessionId('does-not-exist') }] }) const warn = vi.spyOn((ctx.agentLoop as unknown as { ctx: { logger: { warn: (...a: unknown[]) => void } } }).ctx.logger, 'warn') .mockImplementation(() => undefined) await ctx.plugin(SessionPersistenceJsonl, { root }) @@ -129,7 +129,7 @@ describe('config-driven session id', () => { // The deferred resume fails (no such session on disk). It must be contained: // a warning is logged, no 'main' agent is registered, and the app stays up. await new Promise(r => setTimeout(r, 200)) - expect(ctx.agents.get('main')).toBeUndefined() + expect(ctx.agents.get(AgentId('main'))).toBeUndefined() expect(warn).toHaveBeenCalledWith(expect.stringContaining('config-driven resume of "does-not-exist" failed')) warn.mockRestore() await ctx.fiber.dispose() diff --git a/packages/agent-loop/tests/coverage-edges.spec.ts b/packages/core/agent-loop/tests/coverage-edges.spec.ts similarity index 86% rename from packages/agent-loop/tests/coverage-edges.spec.ts rename to packages/core/agent-loop/tests/coverage-edges.spec.ts index 542df6faef..3eefbf6986 100644 --- a/packages/agent-loop/tests/coverage-edges.spec.ts +++ b/packages/core/agent-loop/tests/coverage-edges.spec.ts @@ -4,8 +4,8 @@ import LlmService, { CallId, LlmError, StreamChunk } from '@deepseek-ai/dsh-llm' import SessionStore, { TurnEndReason } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' -import AgentRegistry from '@deepseek-ai/dsh-agent' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { MockAdapter, textResponse, toolCallResponse } from './mock-adapter.ts' async function harness(adapter: MockAdapter) { @@ -20,7 +20,7 @@ async function harness(adapter: MockAdapter) { return ctx } -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { @@ -31,7 +31,7 @@ function waitForIdle(ctx: Context, agent: LoopAgent): Promise { }) } -function send(agent: LoopAgent, text: string) { +function send(agent: ReactLoopAgent, text: string) { agent.send([{ type: 'text', text }]) } @@ -43,7 +43,7 @@ describe('turn boundary listener throws (handled in-turn, loop survives)', () => // The second turn should proceed normally and consume the first script entry. const adapter = new MockAdapter([textResponse('turn 2')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threwOnce = false ctx.on('agent/turn-start', () => { @@ -73,7 +73,7 @@ describe('turn boundary listener throws (handled in-turn, loop survives)', () => it('a throwing agent/turn-end listener surfaces via agent/error and the loop survives', async () => { const adapter = new MockAdapter([textResponse('turn 1'), textResponse('turn 2')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threwOnce = false ctx.on('agent/turn-end', () => { @@ -107,7 +107,7 @@ describe('turn boundary listener throws (handled in-turn, loop survives)', () => // driver survives. This is the ONLY path that reaches the backstop. const adapter = new MockAdapter([textResponse('turn 2')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const errors: { turn: number; step: number; message: string }[] = [] ctx.on('agent/error', (_a, turn, step, error) => void errors.push({ turn, step, message: error.message })) @@ -149,7 +149,7 @@ describe('tool JSON parse', () => { return [{ type: 'text', text: typeof args === 'string' ? `raw: ${args}` : JSON.stringify(args) }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'use tool') await waitForIdle(ctx, agent) @@ -182,7 +182,7 @@ describe('tool JSON parse', () => { return [{ type: 'text', text: 'ran with empty args' }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'use tool') await waitForIdle(ctx, agent) @@ -195,7 +195,7 @@ describe('toError normalization', () => { it('normalizes non-Error throws from turn-start listeners via toError', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threwOnce = false ctx.on('agent/turn-start', () => { @@ -213,15 +213,15 @@ describe('toError normalization', () => { expect(errors).toHaveLength(1) expect(errors[0]!.message).toBe('naked string error') // A non-Error throw is wrapped in a HarnessError with code UNKNOWN, so the - // session error event carries a routable code instead of degrading. - const errorEvent = agent.session.events.find(e => e.type === 'error') - expect(errorEvent?.type === 'error' && errorEvent.data.code).toBe('UNKNOWN') + // turn-end error reason carries a routable code instead of degrading. + const turnEnd = agent.session.events.find(e => e.type === 'turn/end') + expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason.kind === 'error' && turnEnd.data.reason.code).toBe('UNKNOWN') }) it('normalizes non-Error throws from agent/request waterfall via inline toError in runStep catch', async () => { const adapter = new MockAdapter([textResponse('irrelevant')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threwOnce = false ctx.on('agent/request', async (_agent, _turn, _step, _options, _next) => { @@ -240,8 +240,8 @@ describe('toError normalization', () => { expect(errors).toHaveLength(1) // String() of { code: 500 } is '[object Object]' expect(errors[0]!.message).toBe('[object Object]') - const errorEvent = agent.session.events.find(e => e.type === 'error') - expect(errorEvent?.type === 'error' && errorEvent.data.code).toBe('UNKNOWN') + const turnEnd = agent.session.events.find(e => e.type === 'turn/end') + expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason.kind === 'error' && turnEnd.data.reason.code).toBe('UNKNOWN') }) }) @@ -249,7 +249,7 @@ describe('coded error data emission', () => { it('errorData includes code when a coded error (LlmError) is thrown from a plugin', async () => { const adapter = new MockAdapter([textResponse('turn 1')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threwOnce = false ctx.on('agent/request', async (_agent, _turn, _step, _options, next) => { @@ -268,11 +268,11 @@ describe('coded error data emission', () => { expect(errors).toHaveLength(1) expect(errors[0]!.message).toBe('server overloaded') - // session error event includes the code - const errorEvent = agent.session.events.find(e => e.type === 'error') - expect(errorEvent).toBeDefined() - if (errorEvent!.type === 'error') { - expect(errorEvent!.data.code).toBe('RATE_LIMIT') + // turn-end error reason includes the code + const turnEnd = agent.session.events.find(e => e.type === 'turn/end') + expect(turnEnd).toBeDefined() + if (turnEnd?.type === 'turn/end' && turnEnd.data.reason.kind === 'error') { + expect(turnEnd.data.reason.code).toBe('RATE_LIMIT') } }) }) @@ -281,9 +281,9 @@ describe('disposed vs aborted branching', () => { it('handles dispose during model streaming producing reason "disposed"', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) const reasons: TurnEndReason[] = [] @@ -311,7 +311,7 @@ describe('structured tool error propagation (the runtime-validation RFC, part 2) textResponse('done'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.tools.register(defineTool({ name: 'boom', description: 'always fails', diff --git a/packages/agent-loop/tests/inbox.spec.ts b/packages/core/agent-loop/tests/inbox.spec.ts similarity index 100% rename from packages/agent-loop/tests/inbox.spec.ts rename to packages/core/agent-loop/tests/inbox.spec.ts diff --git a/packages/agent-loop/tests/loop.spec.ts b/packages/core/agent-loop/tests/loop.spec.ts similarity index 77% rename from packages/agent-loop/tests/loop.spec.ts rename to packages/core/agent-loop/tests/loop.spec.ts index c4f09a24bf..d018eff7a2 100644 --- a/packages/agent-loop/tests/loop.spec.ts +++ b/packages/core/agent-loop/tests/loop.spec.ts @@ -1,11 +1,11 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import LlmService, { CallId, StreamChunk } from '@deepseek-ai/dsh-llm' -import SessionStore, { TurnEndReason } from '@deepseek-ai/dsh-session' +import SessionStore, { SessionId, TurnEndReason } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' -import AgentRegistry from '@deepseek-ai/dsh-agent' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { MockAdapter, maxTokensResponse, textResponse, toolCallResponse } from './mock-adapter.ts' async function harness(adapter: MockAdapter) { @@ -25,7 +25,7 @@ async function harness(adapter: MockAdapter) { * invoke this right after send(), when the loop hasn't woken yet (status is * still 'idle' synchronously), so polling the current status would lie. */ -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { @@ -36,7 +36,7 @@ function waitForIdle(ctx: Context, agent: LoopAgent): Promise { }) } -function send(agent: LoopAgent, text: string) { +function send(agent: ReactLoopAgent, text: string) { agent.send([{ type: 'text', text }]) } @@ -44,7 +44,7 @@ describe('agent loop', () => { it('runs a simple turn: queued message → model → idle, with ordered events', async () => { const adapter = new MockAdapter([textResponse('hello there')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const order: string[] = [] for (const name of ['agent/turn-start', 'agent/step-start', 'agent/step-end', 'agent/turn-end'] as const) { @@ -58,11 +58,13 @@ describe('agent loop', () => { const types = agent.session.events.map(e => e.type) // turn/start opens the turn, THEN the queued user message is recorded inside - // it (every event is turn-enclosed), then assembled message + usage. + // it (every event is turn-enclosed), then the assembled message (carrying the + // step's usage). expect(types[0]).toBe('turn/start') expect(types[1]).toBe('user/message') expect(types).toContain('assistant/message') - expect(types).toContain('usage') + const assistantMessage = agent.session.events.find(e => e.type === 'assistant/message') + expect(assistantMessage?.type === 'assistant/message' && assistantMessage.data.usage).toEqual({ inputTokens: 10, outputTokens: 'hello there'.length }) expect(types.at(-1)).toBe('turn/end') // derived history: user + assistant @@ -85,7 +87,7 @@ describe('agent loop', () => { return [{ type: 'text', text: `echo: ${args.text}` }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'use the tool') await waitForIdle(ctx, agent) @@ -120,7 +122,7 @@ describe('agent loop', () => { return [] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock', systemPrompt: 'Agent-specific suffix.' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock', systemPrompt: 'Agent-specific suffix.' }) send(agent, 'hi') await waitForIdle(ctx, agent) @@ -133,7 +135,7 @@ describe('agent loop', () => { it('records raw chunks for replay and emits agent/stream-chunk', async () => { const adapter = new MockAdapter([textResponse('abc')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const streamed: StreamChunk[] = [] ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => void streamed.push(chunk)) @@ -161,7 +163,7 @@ describe('agent loop', () => { ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.tools.register(defineTool({ name: 'slow', description: '', @@ -193,7 +195,7 @@ describe('agent loop', () => { it('steering while idle behaves like send (starts a turn)', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) agent.steer([{ type: 'text', text: 'hello' }]) await waitForIdle(ctx, agent) @@ -203,7 +205,7 @@ describe('agent loop', () => { it('inject() while idle wraps context in a one-shot turn, visible to the next request', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) agent.inject([{ type: 'text', text: 'file changed: a.ts' }], { source: { kind: 'plugin', plugin: 'watcher' } }) // The idle inject records a self-contained turn (turn/start → context/message @@ -230,7 +232,7 @@ describe('agent loop', () => { textResponse('done'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // A tool that injects mid-execution: at this point the agent is running, so // inject must append the context/message into the ALREADY-open turn rather // than wrap it in its own one-shot turn. @@ -264,7 +266,7 @@ describe('agent loop', () => { textResponse('step 3'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let steps = 0 ctx.on('agent/step-end', () => void steps++) @@ -290,7 +292,7 @@ describe('agent loop', () => { return [{ type: 'text', text: String(args.text) }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.on('agent/turn-continuation', async () => false as const) @@ -306,7 +308,7 @@ describe('agent loop', () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) ctx.llm.registerAdapter(['other-model'], adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.on('agent/request', async (_agent, _turn, _step, options, next) => { options.model = 'other-model' @@ -318,19 +320,19 @@ describe('agent loop', () => { expect(adapter.requests[0]!.model).toBe('other-model') }) - it('abort() mid-stream ends the turn with reason aborted', async () => { + it('cancel() mid-stream ends the turn with reason aborted', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) send(agent, 'go') - // wait until the stream is hanging, then abort + // wait until the stream is hanging, then cancel await new Promise(r => setTimeout(r, 30)) expect(agent.status).toBe('running') - agent.abort('user interrupt') + agent.cancel('user interrupt') await waitForIdle(ctx, agent) expect(reasons).toEqual([{ kind: 'aborted', reason: 'user interrupt' }]) @@ -341,7 +343,7 @@ describe('agent loop', () => { // turn stops by default and ends max-tokens, not completed. const adapter = new MockAdapter([maxTokensResponse('truncat')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) @@ -366,7 +368,7 @@ describe('agent loop', () => { textResponse('second half'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let steps = 0 ctx.on('agent/step-end', () => void steps++) @@ -397,7 +399,7 @@ describe('agent loop', () => { // stop. The per-turn reason must be independent — turn 2 ends completed. const adapter = new MockAdapter([maxTokensResponse('cut'), textResponse('clean')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) @@ -430,7 +432,7 @@ describe('agent loop', () => { return [{ type: 'text', text: 'should not run' }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) @@ -442,6 +444,66 @@ describe('agent loop', () => { expect(agent.session.events.some(e => e.type === 'tool/call')).toBe(false) expect(agent.session.deriveMessages()).toEqual([{ role: 'user', content: [{ type: 'text', text: 'go' }] }]) expect(reasons).toEqual([{ kind: 'max-tokens' }]) + // No-data-loss: a max-tokens step whose only content was a dropped tool call + // has EMPTY assistant content, but its usage must still be represented. It + // rides on an (empty-content) assistant/message — there is no standalone + // usage event — and that empty message is skipped by deriveMessages(), so + // the derived history above is NOT corrupted by a spurious assistant turn. + const assistantMessage = agent.session.events.find(e => e.type === 'assistant/message') + expect(assistantMessage?.type === 'assistant/message' && assistantMessage.data).toEqual({ + turn: 1, step: 1, content: [], usage: { inputTokens: 10, outputTokens: 5 }, + }) + }) + + it('appends no assistant/message for a max-tokens step with empty content and no usage', async () => { + // A max-tokens step truncated to a dropped tool call AND with no usage chunk + // has nothing to record: empty content and no accounting → no assistant/message + // (the empty-content host exists only to carry usage). The turn still ends + // max-tokens. + const callId = CallId('c1') + const adapter = new MockAdapter([[ + { type: 'block-start', index: 0, blockType: 'tool-call' }, + { type: 'tool-call-delta', index: 0, id: callId, name: 'echo', argumentsDelta: '{"text":"x"}' }, + { type: 'block-end', index: 0, block: { type: 'tool-call', id: callId, name: 'echo', arguments: '{"text":"x"}' } }, + { type: 'finish', reason: { kind: 'max-tokens' } }, + ]]) + const ctx = await harness(adapter) + ctx.tools.register(defineTool({ + name: 'echo', + description: '', + parameters: { text: { type: 'string' } }, + async execute() { return [{ type: 'text', text: 'should not run' }] }, + })) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + const reasons: TurnEndReason[] = [] + ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) + + send(agent, 'go') + await waitForIdle(ctx, agent) + + expect(reasons).toEqual([{ kind: 'max-tokens' }]) + expect(agent.session.events.some(e => e.type === 'assistant/message')).toBe(false) + expect(agent.session.deriveMessages()).toEqual([{ role: 'user', content: [{ type: 'text', text: 'go' }] }]) + }) + + it('appends no assistant/message for a normal stop finish with empty content and no usage', async () => { + // A clean `stop` finish that streamed nothing assembled (no blocks) and + // carried no usage chunk has nothing to record: the content-or-usage guard + // on the normal step path suppresses a pure trace-only empty assistant/message. + const adapter = new MockAdapter([[{ type: 'finish', reason: { kind: 'stop' } }]]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) + + const reasons: TurnEndReason[] = [] + ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) + + send(agent, 'go') + await waitForIdle(ctx, agent) + + expect(reasons).toEqual([{ kind: 'completed' }]) + expect(agent.session.events.some(e => e.type === 'assistant/message')).toBe(false) + expect(agent.session.deriveMessages()).toEqual([{ role: 'user', content: [{ type: 'text', text: 'go' }] }]) }) it('keeps safe max-tokens assistant content while dropping truncated tool calls', async () => { @@ -461,7 +523,7 @@ describe('agent loop', () => { expect(message.content).toEqual([{ type: 'text', text: 'partial text' }]) return next() }) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'go') await waitForIdle(ctx, agent) @@ -488,7 +550,7 @@ describe('agent loop', () => { return [{ type: 'text', text: String(args.text) }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threw = false ctx.on('agent/step-end', () => { if (!threw) { threw = true; throw new Error('bad step-end listener') } @@ -505,7 +567,7 @@ describe('agent loop', () => { it('chains queued messages into consecutive turns', async () => { const adapter = new MockAdapter([textResponse('first'), textResponse('second')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const turns: number[] = [] ctx.on('agent/turn-start', (_agent, turn) => void turns.push(turn)) @@ -530,7 +592,7 @@ describe('agent loop', () => { it('awaits session/flush at turn end (persistence checkpoint)', async () => { const adapter = new MockAdapter([textResponse('ok')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let flushed = 0 let flushedBeforeIdle = false @@ -550,7 +612,7 @@ describe('agent loop', () => { it('errors from the model surface as agent/error and end the turn', async () => { const adapter = new MockAdapter([]) // script exhausted → throws const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) const errors: Error[] = [] const reasons: TurnEndReason[] = [] @@ -563,19 +625,22 @@ describe('agent loop', () => { expect(errors).toHaveLength(1) expect(errors[0]!.message).toContain('script exhausted') expect(reasons[0]).toMatchObject({ kind: 'error' }) - expect(agent.session.events.some(e => e.type === 'error')).toBe(true) + // The durable failure lives entirely on turn/end.reason (with the failing + // step), not a standalone error event. + const turnEnd = agent.session.events.find(e => e.type === 'turn/end') + expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason).toMatchObject({ kind: 'error', step: 1 }) }) it('disposing the loop fiber mid-turn stops the loop (HMR safety)', async () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) - expect(ctx.agents.get('scoped')).toBe(agent) + expect(ctx.agents.get(AgentId('scoped'))).toBe(agent) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) expect(agent.status).toBe('running') @@ -584,7 +649,7 @@ describe('agent loop', () => { await agent.done expect(agent.status).toBe('disposed') - expect(ctx.agents.get('scoped')).toBeUndefined() + expect(ctx.agents.get(AgentId('scoped'))).toBeUndefined() expect(() => { send(agent, 'too late') }).toThrow('disposed') }) @@ -597,11 +662,11 @@ describe('agent loop', () => { await ctx.plugin(ToolRegistry) await ctx.plugin(AgentRegistry) await ctx.plugin(AgentLoop, { - agents: [{ id: 'config-agent', model: 'mock', systemPrompt: 'Config prompt' }], + agents: [{ id: AgentId('config-agent'), model: 'mock', systemPrompt: 'Config prompt' }], }) ctx.llm.registerAdapter(['mock'], adapter) - const agent = ctx.agents.get('config-agent')! as LoopAgent + const agent = ctx.agents.get(AgentId('config-agent'))! as ReactLoopAgent expect(agent).toBeDefined() expect(agent.id).toBe('config-agent') expect(agent.options.model).toBe('mock') @@ -626,11 +691,11 @@ describe('agent loop', () => { return [{ type: 'text', text: String(args.text) }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'run') await waitForIdle(ctx, agent) - const replayed = ctx.sessions.create('replayed', { seed: [...agent.session.events] }) + const replayed = ctx.sessions.create(SessionId('replayed'), { seed: [...agent.session.events] }) expect(replayed.deriveMessages()).toEqual(agent.session.deriveMessages()) // event-by-event identity of types expect(replayed.events.map(e => e.type)).toEqual( diff --git a/packages/agent-loop/tests/mock-adapter.ts b/packages/core/agent-loop/tests/mock-adapter.ts similarity index 100% rename from packages/agent-loop/tests/mock-adapter.ts rename to packages/core/agent-loop/tests/mock-adapter.ts diff --git a/packages/agent-loop/tests/properties.spec.ts b/packages/core/agent-loop/tests/properties.spec.ts similarity index 90% rename from packages/agent-loop/tests/properties.spec.ts rename to packages/core/agent-loop/tests/properties.spec.ts index e7df7f9d74..1e603a1cc4 100644 --- a/packages/agent-loop/tests/properties.spec.ts +++ b/packages/core/agent-loop/tests/properties.spec.ts @@ -17,8 +17,8 @@ import { LlmAdapter } from '@deepseek-ai/dsh-llm' import SessionStore from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import AgentRegistry from '@deepseek-ai/dsh-agent' -import AgentLoop, { type LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' +import AgentLoop, { type ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import fc from 'fast-check' /** A never-exhausting adapter: every model call returns the same short reply. */ @@ -47,7 +47,7 @@ async function harness() { } /** Resolve on the agent's next transition to idle (event-based, not polled). */ -function nextIdle(ctx: Context, agent: LoopAgent): Promise { +function nextIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { @@ -60,7 +60,7 @@ function nextIdle(ctx: Context, agent: LoopAgent): Promise { /** Record every status transition for the legal-machine assertion. Returns * the seen list plus a disposer for the listener (per the registry convention). */ -function recordStatus(ctx: Context, agent: LoopAgent): { seen: string[]; dispose: () => void } { +function recordStatus(ctx: Context, agent: ReactLoopAgent): { seen: string[]; dispose: () => void } { const seen: string[] = [] const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent) seen.push(status) @@ -68,13 +68,13 @@ function recordStatus(ctx: Context, agent: LoopAgent): { seen: string[]; dispose return { seen, dispose } } -function userMessageTexts(agent: LoopAgent): string[] { +function userMessageTexts(agent: ReactLoopAgent): string[] { return agent.session.events .filter(e => e.type === 'user/message') .map(e => (e.data as { content: { type: string; text?: string }[] }).content.map(b => b.text ?? '').join('')) } -function turnNumbers(agent: LoopAgent): number[] { +function turnNumbers(agent: ReactLoopAgent): number[] { return agent.session.events .filter(e => e.type === 'turn/start') .map(e => (e.data as { turn: number }).turn) @@ -95,7 +95,7 @@ describe('agent loop scheduling properties', () => { async (texts) => { const ctx = await harness() try { - const agent = ctx.agentLoop.create('a', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a'), { model: 'mock' }) const { seen: trace } = recordStatus(ctx, agent) const idle = nextIdle(ctx, agent) // Send all in one synchronous tick: they queue before the loop wakes. @@ -120,7 +120,7 @@ describe('agent loop scheduling properties', () => { async (texts) => { const ctx = await harness() try { - const agent = ctx.agentLoop.create('a', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a'), { model: 'mock' }) for (const text of texts) { const idle = nextIdle(ctx, agent) agent.send([{ type: 'text', text }]) @@ -145,7 +145,7 @@ describe('agent loop scheduling properties', () => { async (steps) => { const ctx = await harness() try { - const agent = ctx.agentLoop.create('a', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a'), { model: 'mock' }) // Capture an idle waiter before EACH send; the last one is guaranteed // to resolve because the final send always triggers (or joins) a turn // that ends idle. Awaiting an already-resolved waiter is a no-op, so a diff --git a/packages/agent-loop/tests/resume.spec.ts b/packages/core/agent-loop/tests/resume.spec.ts similarity index 83% rename from packages/agent-loop/tests/resume.spec.ts rename to packages/core/agent-loop/tests/resume.spec.ts index 3586dcab3c..6192396cab 100644 --- a/packages/agent-loop/tests/resume.spec.ts +++ b/packages/core/agent-loop/tests/resume.spec.ts @@ -8,9 +8,9 @@ import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' import type { SessionEvent } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' -import AgentRegistry from '@deepseek-ai/dsh-agent' +import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import { MockAdapter, textResponse } from './mock-adapter.ts' const dirs: string[] = [] @@ -31,7 +31,7 @@ async function persistentHarness(adapter: MockAdapter): Promise<{ ctx: Context; return { ctx, root } } -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { dispose(); resolve() } @@ -43,7 +43,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { it('createAgent uses the caller-supplied sessionId (not ${id}-session)', async () => { const adapter = new MockAdapter([textResponse('hi')]) const { ctx } = await persistentHarness(adapter) - const agent = ctx.agents.create({ agentId: 'a1', sessionId: 'custom-session', meta: { cwd: '/w' } }) + const { agent } = ctx.agents.create({ agentId: AgentId('a1'), sessionId: SessionId('custom-session'), meta: { cwd: '/w' } }) expect(agent.session.id).toBe('custom-session') expect(agent.session.header.cwd).toBe('/w') await ctx.fiber.dispose() @@ -52,18 +52,18 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { it('createAgent rejects a duplicate agent id BEFORE creating the session (no orphan)', async () => { const adapter = new MockAdapter([textResponse('hi')]) const { ctx } = await persistentHarness(adapter) - ctx.agents.create({ agentId: 'dup', sessionId: 'sess-a' }) + ctx.agents.create({ agentId: AgentId('dup'), sessionId: SessionId('sess-a') }) // A second create with the SAME agent id but a fresh session id must reject // up front — and must NOT leave an orphaned 'sess-b' session behind. - expect(() => ctx.agents.create({ agentId: 'dup', sessionId: 'sess-b' })).toThrow(/already registered/) - expect(ctx.sessions.get('sess-b')).toBeUndefined() + expect(() => ctx.agents.create({ agentId: AgentId('dup'), sessionId: SessionId('sess-b') })).toThrow(/already registered/) + expect(ctx.sessions.get(SessionId('sess-b'))).toBeUndefined() await ctx.fiber.dispose() }) it('createAgent works without meta (no cwd)', async () => { const adapter = new MockAdapter([textResponse('hi')]) const { ctx } = await persistentHarness(adapter) - const agent = ctx.agents.create({ agentId: 'a-nometa', sessionId: 'nometa-session' }) + const { agent } = ctx.agents.create({ agentId: AgentId('a-nometa'), sessionId: SessionId('nometa-session') }) expect(agent.session.id).toBe('nometa-session') expect(agent.session.header.cwd).toBeUndefined() await ctx.fiber.dispose() @@ -73,7 +73,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { // Lifecycle 1: create a no-cwd session and run a turn. const adapter1 = new MockAdapter([textResponse('a')]) const { ctx: ctx1, root } = await persistentHarness(adapter1) - const a1 = ctx1.agents.create({ agentId: 'm', sessionId: 'nocwd-sess' }) as LoopAgent + const a1 = ctx1.agents.create({ agentId: AgentId('m'), sessionId: SessionId('nocwd-sess') }).agent as ReactLoopAgent a1.send([{ type: 'text', text: 'q' }], { source: { kind: 'user' } }) await waitForIdle(ctx1, a1) await ctx1.fiber.dispose() @@ -89,7 +89,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { await ctx2.plugin(AgentLoop, { agents: [] }) await ctx2.plugin(SessionPersistenceJsonl, { root }) ctx2.llm.registerAdapter(['mock'], adapter2) - const a2 = await ctx2.agents.resume({ agentId: 'm', resumeSessionId: 'nocwd-sess' }) as LoopAgent + const a2 = (await ctx2.agents.resume({ agentId: AgentId('m'), resumeSessionId: SessionId('nocwd-sess') })).agent as ReactLoopAgent expect(a2.session.header.cwd).toBeUndefined() await ctx2.fiber.dispose() }) @@ -104,7 +104,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { ] const adapter1 = new MockAdapter([textResponse('a')]) const { ctx: ctx1, root } = await persistentHarness(adapter1) - const forked = ctx1.sessions.create('forked-sess', { seed, meta: { cwd: '/w', parentSession: SessionId('parent-sess') } }) + const forked = ctx1.sessions.create(SessionId('forked-sess'), { seed, meta: { cwd: '/w', parentSession: SessionId('parent-sess') } }) await ctx1.parallel('session/flush', forked) await ctx1.fiber.dispose() @@ -120,7 +120,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { await ctx2.plugin(AgentLoop, { agents: [] }) await ctx2.plugin(SessionPersistenceJsonl, { root }) ctx2.llm.registerAdapter(['mock'], adapter2) - const a2 = await ctx2.agents.resume({ agentId: 'm', resumeSessionId: 'forked-sess' }) as LoopAgent + const a2 = (await ctx2.agents.resume({ agentId: AgentId('m'), resumeSessionId: SessionId('forked-sess') })).agent as ReactLoopAgent expect(a2.session.header.parentSession).toBe('parent-sess') expect(a2.session.header.cwd).toBe('/w') await ctx2.fiber.dispose() @@ -133,7 +133,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { // disk, since a crash before the next turn would otherwise lose it. const adapter1 = new MockAdapter([textResponse('answer')]) const { ctx: ctx1, root } = await persistentHarness(adapter1) - const a1 = ctx1.agents.create({ agentId: 'm', sessionId: 'inject-sess', meta: { cwd: '/w' } }) as LoopAgent + const a1 = ctx1.agents.create({ agentId: AgentId('m'), sessionId: SessionId('inject-sess'), meta: { cwd: '/w' } }).agent as ReactLoopAgent a1.send([{ type: 'text', text: 'q' }], { source: { kind: 'user' } }) await waitForIdle(ctx1, a1) a1.inject([{ type: 'text', text: 'background task 42 finished' }], { source: { kind: 'plugin', plugin: 'tool-bash' } }) @@ -158,7 +158,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { // drop it on reload (the bug this guards). const adapter1 = new MockAdapter([textResponse('answer')]) const { ctx: ctx1, root } = await persistentHarness(adapter1) - const a1 = ctx1.agents.create({ agentId: 'm', sessionId: 'inject-sess', meta: { cwd: '/w' } }) as LoopAgent + const a1 = ctx1.agents.create({ agentId: AgentId('m'), sessionId: SessionId('inject-sess'), meta: { cwd: '/w' } }).agent as ReactLoopAgent a1.send([{ type: 'text', text: 'q' }], { source: { kind: 'user' } }) await waitForIdle(ctx1, a1) a1.inject([{ type: 'text', text: 'background task 42 finished' }], { source: { kind: 'plugin', plugin: 'tool-bash' } }) @@ -176,7 +176,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { await ctx2.plugin(AgentLoop, { agents: [] }) await ctx2.plugin(SessionPersistenceJsonl, { root }) ctx2.llm.registerAdapter(['mock'], adapter2) - const a2 = await ctx2.agents.resume({ agentId: 'm', resumeSessionId: 'inject-sess' }) as LoopAgent + const a2 = (await ctx2.agents.resume({ agentId: AgentId('m'), resumeSessionId: SessionId('inject-sess') })).agent as ReactLoopAgent const flat = JSON.stringify(a2.session.deriveMessages()) expect(flat).toContain('background task 42 finished') await ctx2.fiber.dispose() @@ -186,7 +186,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { // Lifecycle 1: run one full turn, persisting it. const adapter1 = new MockAdapter([textResponse('first answer')]) const { ctx: ctx1, root } = await persistentHarness(adapter1) - const a1 = ctx1.agents.create({ agentId: 'main', sessionId: 'sess-resume', meta: { cwd: '/w' } }) as LoopAgent + const a1 = ctx1.agents.create({ agentId: AgentId('main'), sessionId: SessionId('sess-resume'), meta: { cwd: '/w' } }).agent as ReactLoopAgent a1.send([{ type: 'text', text: 'first question' }], { source: { kind: 'user' } }) await waitForIdle(ctx1, a1) const events1 = [...a1.session.events] @@ -206,7 +206,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { await ctx2.plugin(SessionPersistenceJsonl, { root }) ctx2.llm.registerAdapter(['mock'], adapter2) - const a2 = await ctx2.agents.resume({ agentId: 'main', resumeSessionId: 'sess-resume' }) as LoopAgent + const a2 = (await ctx2.agents.resume({ agentId: AgentId('main'), resumeSessionId: SessionId('sess-resume') })).agent as ReactLoopAgent // The resumed session carries the prior history… expect(a2.session.id).toBe('sess-resume') expect(a2.session.events.length).toBe(events1.length) @@ -234,7 +234,7 @@ describe('the session-persistence RFC: AgentLoop factory create/resume', () => { await ctx.plugin(AgentRegistry) await ctx.plugin(AgentLoop, { agents: [] }) ctx.llm.registerAdapter(['mock'], adapter) - await expect(ctx.agents.resume({ agentId: 'm', resumeSessionId: 'nope' })) + await expect(ctx.agents.resume({ agentId: AgentId('m'), resumeSessionId: SessionId('nope') })) .rejects.toThrow(/session persistence is not configured/) await ctx.fiber.dispose() }) diff --git a/packages/agent-loop/tests/review-fixes.spec.ts b/packages/core/agent-loop/tests/review-fixes.spec.ts similarity index 79% rename from packages/agent-loop/tests/review-fixes.spec.ts rename to packages/core/agent-loop/tests/review-fixes.spec.ts index 711177f62e..ed0900c4a1 100644 --- a/packages/agent-loop/tests/review-fixes.spec.ts +++ b/packages/core/agent-loop/tests/review-fixes.spec.ts @@ -1,11 +1,11 @@ -import { describe, expect, it } from 'vitest' +import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import LlmService, { CallId, ContentBlock, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm' +import LlmService, { CallId, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm' import SessionStore, { Session, SessionEvent, SessionId, TurnEndReason } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent' -import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import AgentLoop, { ReactLoopAgent } from '@deepseek-ai/dsh-agent-loop' import * as Invariants from '@deepseek-ai/dsh-invariants' import { MockAdapter, textResponse, toolCallResponse } from './mock-adapter.ts' @@ -26,7 +26,7 @@ async function harness(adapter: MockAdapter) { return ctx } -function waitForIdle(ctx: Context, agent: LoopAgent): Promise { +function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise { return new Promise((resolve) => { const dispose = ctx.on('agent/status', (subject, status) => { if (subject === agent && status === 'idle') { @@ -37,7 +37,7 @@ function waitForIdle(ctx: Context, agent: LoopAgent): Promise { }) } -function send(agent: LoopAgent, text: string) { +function send(agent: ReactLoopAgent, text: string) { agent.send([{ type: 'text', text }]) } @@ -55,7 +55,7 @@ describe('HIGH: session log records what agent/step-result actually produced', ( return [{ type: 'text', text: 'ran' }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) // Plugin rewrites the message: replaces the text AND adds a tool call. let rewritten = false @@ -92,7 +92,7 @@ describe('HIGH: session log records what agent/step-result actually produced', ( }) describe('HIGH: abort during tool execution ends the turn', () => { - it('abort() inside a tool prevents both remaining tools and the next model step', async () => { + it('aborting the in-flight step inside a tool prevents both remaining tools and the next model step', async () => { const adapter = new MockAdapter([ // model asks for two tool calls in one step [ @@ -106,14 +106,18 @@ describe('HIGH: abort during tool execution ends the turn', () => { ]) const ctx = await harness(adapter) const executed: string[] = [] - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.tools.register(defineTool({ name: 'aborter', description: '', parameters: {}, async execute() { executed.push('aborter') - agent.abort('user interrupt') + // Fire the in-flight step's AbortController directly (the loop registers + // it on the agent). This is the bare step-abort path — distinct from + // cancel(), which would also clear the inbox; here the subject is the + // loop's response to its running step being aborted mid-tool. + ;(agent as unknown as { currentAbort?: AbortController }).currentAbort?.abort('user interrupt') return [{ type: 'text', text: 'done' }] }, })) @@ -154,7 +158,7 @@ describe('HIGH: steering from late extension points is never stranded', () => { return [{ type: 'text', text: String(args.text) }] }, })) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let steeredOnce = false ctx.on('agent/step-end', () => { @@ -176,7 +180,7 @@ describe('HIGH: steering from late extension points is never stranded', () => { textResponse('continued because of steering'), ]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let steeredOnce = false ctx.on('agent/turn-continuation', async (_agent, _turn, _decision, next) => { @@ -198,7 +202,7 @@ describe('HIGH: steering from late extension points is never stranded', () => { it('steer() from an agent/turn-end listener becomes a queued message for the next turn', async () => { const adapter = new MockAdapter([textResponse('turn 1'), textResponse('turn 2')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let steeredOnce = false ctx.on('agent/turn-end', () => { @@ -223,12 +227,17 @@ describe('HIGH: steering from late extension points is never stranded', () => { it('steering queued during an aborted step is re-delivered, not silently consumed', async () => { const adapter = new MockAdapter(['hang', textResponse('recovered')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'go') await new Promise(r => setTimeout(r, 30)) agent.steer([{ type: 'text', text: 'redirect' }]) - agent.abort('user interrupt') + // Abort ONLY the in-flight step, via its AbortController directly — NOT + // cancel(), which clears the inbox and would drop the queued steering this + // test proves survives a step abort. There is no public step-only abort + // verb (cancel() is the only public stop primitive), so reach the private + // controller the loop registered. + ;(agent as unknown as { currentAbort?: AbortController }).currentAbort?.abort('user interrupt') await waitForIdle(ctx, agent) // a new turn ran with the steering content delivered as a message @@ -241,7 +250,7 @@ describe('HIGH: plugin exceptions are contained', () => { it('a throwing agent/turn-continuation listener ends the turn with an error, loop survives', async () => { const adapter = new MockAdapter([textResponse('one'), textResponse('two')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let threwOnce = false ctx.on('agent/turn-continuation', async (): Promise => { @@ -269,7 +278,7 @@ describe('HIGH: plugin exceptions are contained', () => { it('a rejecting session/flush listener is reported but does not kill the agent', async () => { const adapter = new MockAdapter([textResponse('one'), textResponse('two')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) let rejectedOnce = false ctx.on('session/flush', async () => { @@ -297,9 +306,9 @@ describe('MEDIUM: disposed status is part of the agent/status contract', () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) const statuses: string[] = [] @@ -320,9 +329,9 @@ describe('MEDIUM: disposed status is part of the agent/status contract', () => { const adapter = new MockAdapter(['hang']) const ctx = await harness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('scoped', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('scoped'), { model: 'mock' }) }, { inject: ['agentLoop'] })) ctx.on('agent/status', (_agent, status) => { @@ -335,7 +344,7 @@ describe('MEDIUM: disposed status is part of the agent/status contract', () => { await agent.done // must not hang expect(agent.status).toBe('disposed') - expect(ctx.agents.get('scoped')).toBeUndefined() // unregistered despite the throw + expect(ctx.agents.get(AgentId('scoped'))).toBeUndefined() // unregistered despite the throw }) }) @@ -354,7 +363,7 @@ describe('MEDIUM: misc registry and config fixes', () => { it('an agent without a model fails the step with a clear error (not NO_ADAPTER for "default")', async () => { const adapter = new MockAdapter([textResponse('never')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', {}) // no model + const agent = ctx.agentLoop.create(AgentId('a1'), {}) // no model const errors: Error[] = [] ctx.on('agent/error', (_agent, _turn, _step, error) => void errors.push(error)) @@ -369,7 +378,7 @@ describe('MEDIUM: misc registry and config fixes', () => { it('the agent/request waterfall can supply the model for a model-less agent', async () => { const adapter = new MockAdapter([textResponse('routed')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', {}) // no model — router plugin decides + const agent = ctx.agentLoop.create(AgentId('a1'), {}) // no model — router plugin decides ctx.on('agent/request', async (_agent, _turn, _step, options, next) => { options.model = 'mock' @@ -385,7 +394,7 @@ describe('MEDIUM: misc registry and config fixes', () => { it('agent/queued carries the resolved source; agent/steering carries its source', async () => { const adapter = new MockAdapter([toolCallResponse('c1', 'noop', {}), textResponse('done')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) ctx.tools.register(defineTool({ name: 'noop', description: '', @@ -414,7 +423,7 @@ describe('MEDIUM: turn numbering continues across seeded (forked) sessions', () it('a forked agent continues turn numbers after the seed log', async () => { const first = new MockAdapter([textResponse('turn one')]) const ctx = await harness(first) - const agent = ctx.agentLoop.create('a1', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) send(agent, 'first') await waitForIdle(ctx, agent) @@ -429,8 +438,8 @@ describe('MEDIUM: turn numbering continues across seeded (forked) sessions', () await ctx2.plugin(AgentLoop, { agents: [] }) ctx2.llm.registerAdapter(['mock'], second) - const seeded = ctx2.sessions.create('forked', { seed: [...agent.session.events] }) - const forked = new LoopAgent(ctx2, AgentId('forked-agent'), { model: 'mock' }, seeded) + const seeded = ctx2.sessions.create(SessionId('forked'), { seed: [...agent.session.events] }) + const forked = new ReactLoopAgent(ctx2, AgentId('forked-agent'), { model: 'mock' }, seeded) ctx2.effect(() => forked.start()) const turns: number[] = [] @@ -446,90 +455,6 @@ describe('MEDIUM: turn numbering continues across seeded (forked) sessions', () }) }) -describe('LOW: BlockAssembler and streamBlocks edge cases', () => { - it('ignores deltas arriving after block-end for the same index (malformed stream)', async () => { - const { BlockAssembler } = await import('@deepseek-ai/dsh-llm') - const assembler = new BlockAssembler() - assembler.push({ type: 'block-start', index: 0, blockType: 'text' }) - assembler.push({ type: 'text-delta', index: 0, text: 'good' }) - assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'good' } }) - assembler.push({ type: 'text-delta', index: 0, text: ' straggler' }) - expect(assembler.blocks()).toEqual([{ type: 'text', text: 'good' }]) - }) - - it('assembles tool-call blocks from deltas without block-end', async () => { - const { BlockAssembler } = await import('@deepseek-ai/dsh-llm') - const assembler = new BlockAssembler() - assembler.push({ type: 'tool-call-delta', index: 0, id: CallId('c9'), name: 'echo', argumentsDelta: '{"a"' }) - assembler.push({ type: 'tool-call-delta', index: 0, id: CallId('c9'), argumentsDelta: ':1}' }) - expect(assembler.blocks()).toEqual([ - { type: 'tool-call', id: CallId('c9'), name: 'echo', arguments: '{"a":1}' }, - ]) - }) - - it('streamBlocks flushes delta-only blocks at end of stream (matches generate())', async () => { - const ctx = new Context() - await ctx.plugin(LlmService) - const deltaOnly: StreamChunk[] = [ - { type: 'text-delta', index: 0, text: 'no ' }, - { type: 'text-delta', index: 0, text: 'block-end' }, - { type: 'finish', reason: { kind: 'stop' } }, - ] - ctx.llm.registerAdapter(['m'], new MockAdapter([deltaOnly, deltaOnly])) - - const blocks: ContentBlock[] = [] - for await (const block of ctx.llm.streamBlocks({ model: 'm', messages: [] })) blocks.push(block) - expect(blocks).toEqual([{ type: 'text', text: 'no block-end' }]) - - const generated = await ctx.llm.generate({ model: 'm', messages: [] }) - expect(generated.message.content).toEqual(blocks) - }) - - it('streamBlocks preserves stream order when an open block precedes a closed one', async () => { - const ctx = new Context() - await ctx.plugin(LlmService) - // index 0 never gets block-end (delta-only); index 1 closes mid-stream. - const interleaved: StreamChunk[] = [ - { type: 'text-delta', index: 0, text: 'first, open' }, - { type: 'block-start', index: 1, blockType: 'text' }, - { type: 'text-delta', index: 1, text: 'second, closed' }, - { type: 'block-end', index: 1, block: { type: 'text', text: 'second, closed' } }, - { type: 'finish', reason: { kind: 'stop' } }, - ] - ctx.llm.registerAdapter(['m'], new MockAdapter([interleaved, interleaved])) - - const blocks: ContentBlock[] = [] - for await (const block of ctx.llm.streamBlocks({ model: 'm', messages: [] })) blocks.push(block) - expect(blocks).toEqual([ - { type: 'text', text: 'first, open' }, - { type: 'text', text: 'second, closed' }, - ]) - - // identical to generate()'s assembled order - const generated = await ctx.llm.generate({ model: 'm', messages: [] }) - expect(generated.message.content).toEqual(blocks) - }) - - it('streamBlocks yields closed blocks incrementally once preceding blocks close', async () => { - const ctx = new Context() - await ctx.plugin(LlmService) - const script: StreamChunk[] = [ - { type: 'block-start', index: 0, blockType: 'text' }, - { type: 'text-delta', index: 0, text: 'a' }, - { type: 'block-end', index: 0, block: { type: 'text', text: 'a' } }, - { type: 'block-start', index: 1, blockType: 'text' }, - { type: 'text-delta', index: 1, text: 'b' }, - { type: 'block-end', index: 1, block: { type: 'text', text: 'b' } }, - { type: 'finish', reason: { kind: 'stop' } }, - ] - ctx.llm.registerAdapter(['m'], new MockAdapter([script])) - - const blocks: ContentBlock[] = [] - for await (const block of ctx.llm.streamBlocks({ model: 'm', messages: [] })) blocks.push(block) - expect(blocks).toEqual([{ type: 'text', text: 'a' }, { type: 'text', text: 'b' }]) - }) -}) - describe('LOW: discriminated SessionEvent narrows without casts', () => { it('narrows event.data from event.type', () => { const session = new Session(SessionId('s')) @@ -559,7 +484,7 @@ describe('HIGH: a finish-error stream chunk ends the turn as error, not complete ] const adapter = new MockAdapter([errorStream]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-finish-error', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-finish-error'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) @@ -567,11 +492,13 @@ describe('HIGH: a finish-error stream chunk ends the turn as error, not complete send(agent, 'go') await waitForIdle(ctx, agent) - expect(reasons).toEqual([{ kind: 'error', message: 'provider 401', code: 'AUTH' }]) + expect(reasons).toEqual([{ kind: 'error', step: 1, message: 'provider 401', code: 'AUTH' }]) const events = [...agent.session.events] - expect(events.some(event => event.type === 'error' - && event.data.message === 'provider 401' && event.data.code === 'AUTH')).toBe(true) + // The durable failure lives on turn/end.reason (with the failing step), not + // a standalone error event. + const turnEnd = events.find(event => event.type === 'turn/end') + expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason).toEqual({ kind: 'error', step: 1, message: 'provider 401', code: 'AUTH' }) // Crucially: no assistant/message was logged for the failed step. expect(events.some(event => event.type === 'assistant/message')).toBe(false) }) @@ -582,7 +509,7 @@ describe('HIGH: a finish-error stream chunk ends the turn as error, not complete ] const adapter = new MockAdapter([abortedStream]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-finish-aborted', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-finish-aborted'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) @@ -590,7 +517,7 @@ describe('HIGH: a finish-error stream chunk ends the turn as error, not complete send(agent, 'go') await waitForIdle(ctx, agent) - expect(reasons).toEqual([{ kind: 'error', message: 'model stream aborted', code: 'ABORTED' }]) + expect(reasons).toEqual([{ kind: 'error', step: 1, message: 'model stream aborted', code: 'ABORTED' }]) expect([...agent.session.events].some(event => event.type === 'assistant/message')).toBe(false) }) @@ -600,7 +527,7 @@ describe('HIGH: a finish-error stream chunk ends the turn as error, not complete ] const adapter = new MockAdapter([errorStream]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-finish-error-nocode', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-finish-error-nocode'), { model: 'mock' }) const reasons: TurnEndReason[] = [] ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason)) @@ -608,7 +535,7 @@ describe('HIGH: a finish-error stream chunk ends the turn as error, not complete send(agent, 'go') await waitForIdle(ctx, agent) - expect(reasons).toEqual([{ kind: 'error', message: 'codeless failure' }]) + expect(reasons).toEqual([{ kind: 'error', step: 1, message: 'codeless failure' }]) }) }) @@ -616,7 +543,7 @@ describe('P1-6: step/start is appended before agent/step-start is emitted', () = it('a step-start listener sees the step/start event already in session.events', async () => { const adapter = new MockAdapter([textResponse('done')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-step-order', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-step-order'), { model: 'mock' }) // Capture, at the moment agent/step-start fires, whether the matching // step/start event is already in the log (append-before-emit, the event-sourcing RFC). @@ -660,14 +587,14 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar } /** Count turn/step boundary events for balance assertions. */ - function boundaryCounts(agent: LoopAgent) { + function boundaryCounts(agent: ReactLoopAgent) { const e = [...agent.session.events] return { turnStart: e.filter(x => x.type === 'turn/start').length, turnEnd: e.filter(x => x.type === 'turn/end').length, stepStart: e.filter(x => x.type === 'step/start').length, stepEnd: e.filter(x => x.type === 'step/end').length, - errors: e.filter(x => x.type === 'error').length, + errors: e.filter(x => x.type === 'turn/end' && x.data.reason.kind === 'error').length, lastTurnEnd: e.findLast(x => x.type === 'turn/end'), } } @@ -675,7 +602,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar it('a throwing agent/turn-start listener still closes the turn with exactly one error and one turn/end, no step', async () => { const adapter = new MockAdapter([textResponse('never reached')]) const ctx = await balancedHarness(adapter) - const agent = ctx.agentLoop.create('a-turnstart', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-turnstart'), { model: 'mock' }) let threw = false ctx.on('agent/turn-start', () => { if (!threw) { threw = true; throw new Error('boom turn-start') } }) @@ -686,10 +613,10 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar await waitForIdle(ctx, agent) const c = boundaryCounts(agent) - // turn opened and closed; no step ran; exactly one error logged + emitted. + // turn opened and closed; no step ran; exactly one error turn-end + emitted. expect(c).toMatchObject({ turnStart: 1, turnEnd: 1, stepStart: 0, stepEnd: 0, errors: 1 }) expect(errors.map(e => e.message)).toEqual(['boom turn-start']) - expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toEqual({ kind: 'error', message: 'boom turn-start' }) + expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toEqual({ kind: 'error', step: 0, message: 'boom turn-start' }) // model was never called (we threw before the step's request). expect(adapter.requests).toHaveLength(0) }) @@ -697,7 +624,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar it('a throwing agent/step-start listener closes the open step then the turn (step/end before turn/end)', async () => { const adapter = new MockAdapter([textResponse('never reached')]) const ctx = await balancedHarness(adapter) - const agent = ctx.agentLoop.create('a-stepstart', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-stepstart'), { model: 'mock' }) let threw = false ctx.on('agent/step-start', () => { if (!threw) { threw = true; throw new Error('boom step-start') } }) @@ -726,7 +653,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', message: 'provider 500' } }] const adapter = new MockAdapter([errorStream, textResponse('turn 2 ok')]) const ctx = await balancedHarness(adapter) - const agent = ctx.agentLoop.create('a-errorlistener', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-errorlistener'), { model: 'mock' }) let threw = false ctx.on('agent/error', () => { if (!threw) { threw = true; throw new Error('boom error-listener') } }) @@ -739,7 +666,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar expect(c.turnStart).toBe(1) expect(c.turnEnd).toBe(1) expect(c.stepStart).toBe(c.stepEnd) - expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toMatchObject({ kind: 'error', message: 'provider 500' }) + expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toMatchObject({ kind: 'error', step: 1, message: 'provider 500' }) // loop survives: a second turn runs to completion (invariants oracle would // throw on its turn/start if turn 1 had been left open). @@ -757,9 +684,9 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // balanced with reason disposed (no error event for a disposal). const adapter = new MockAdapter(['hang']) const ctx = await balancedHarness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('a-dispose', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('a-dispose'), { model: 'mock' }) }, { inject: ['agentLoop'] })) const reasons: TurnEndReason[] = [] @@ -776,8 +703,8 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar expect(turnStarts).toBe(1) expect(turnEnds).toBe(1) // balanced — the turn was closed despite disposal expect(reasons).toEqual([{ kind: 'disposed' }]) - // no error event: disposal is not a failure. - expect(e.some(x => x.type === 'error')).toBe(false) + // no error reason: disposal is not a failure. + expect(e.some(x => x.type === 'turn/end' && x.data.reason.kind === 'error')).toBe(false) }) it('preserves reason disposed when the turn-end emit throws during disposal (outer-catch disposed branch)', async () => { @@ -788,9 +715,9 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // throw. This is the only path that exercises that catch sub-branch. const adapter = new MockAdapter(['hang']) const ctx = await balancedHarness(adapter) - let agent!: LoopAgent + let agent!: ReactLoopAgent const fiber = await ctx.plugin(Object.assign((inner: Context) => { - agent = inner.agentLoop.create('a-dispose-emit-throw', { model: 'mock' }) + agent = inner.agentLoop.create(AgentId('a-dispose-emit-throw'), { model: 'mock' }) }, { inject: ['agentLoop'] })) // The FIRST agent/turn-end emit throws (the disposal-driven turn end). @@ -816,9 +743,10 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar expect(e.filter(x => x.type === 'turn/start')).toHaveLength(1) const turnEnd = e.findLast(x => x.type === 'turn/end') expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason).toEqual({ kind: 'disposed' }) - // The throwing turn-end listener is contained: no error event is logged and - // no agent/error is emitted (disposal is not a failure; the throw is swallowed). - expect(e.some(x => x.type === 'error')).toBe(false) + // The throwing turn-end listener is contained: the turn/end carries the + // disposed reason (not an error) and no agent/error is emitted (disposal is + // not a failure; the throw is swallowed). + expect(e.some(x => x.type === 'turn/end' && x.data.reason.kind === 'error')).toBe(false) expect(errorEmits).toHaveLength(0) }) @@ -833,7 +761,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // subscriber.) const adapter = new MockAdapter([textResponse('turn 2')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-preturn', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-preturn'), { model: 'mock' }) let threw = false ctx.on('session/event', (_session, event) => { @@ -872,12 +800,13 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // surfaced via agent/error instead, and the log's last event is turn/end. const adapter = new MockAdapter([textResponse('done'), textResponse('next ok')]) const ctx = await balancedHarness(adapter) - const agent = ctx.agentLoop.create('a-tend', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-tend'), { model: 'mock' }) let threw = false ctx.on('agent/turn-end', () => { if (!threw) { threw = true; throw new Error('boom turn-end') } }) const errors: Error[] = [] ctx.on('agent/error', (_a, _t, _s, error) => void errors.push(error)) + const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) send(agent, 'go') await waitForIdle(ctx, agent) @@ -887,6 +816,9 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar expect(c.errors).toBe(0) // NO session error event (it would be post-turn/end) expect(agent.session.events.at(-1)?.type).toBe('turn/end') // last event is the boundary expect(errors.map(e => e.message)).toEqual(['boom turn-end']) // surfaced via agent/error + // The late throw is also logged directly: failTurn's turn-already-ended + // branch warns so a throwing turn-end listener after turn/end never vanishes. + expect(warn).toHaveBeenCalledWith(expect.stringContaining('agent/turn-end listener threw after turn 1 closed')) // The whole log is loadable (nothing dropped): a fresh replay sees the turn. const replay = new Session(SessionId('replay'), [...agent.session.events]) expect(replay.deriveMessages().map(m => m.role)).toEqual(['user', 'assistant']) @@ -904,7 +836,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // swallowed the throw in the normal (no-tool, no-steering) path. const adapter = new MockAdapter([textResponse('all good'), textResponse('turn 2 ok')]) const ctx = await balancedHarness(adapter) - const agent = ctx.agentLoop.create('a-stepend-throw', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-stepend-throw'), { model: 'mock' }) let threw = false ctx.on('agent/step-end', () => { if (!threw) { threw = true; throw new Error('boom step-end') } }) @@ -915,11 +847,11 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar await waitForIdle(ctx, agent) const c = boundaryCounts(agent) - // step opened and closed; exactly one error; turn balanced; turn ends error. + // step opened and closed; exactly one error turn-end; turn balanced. expect(c).toMatchObject({ turnStart: 1, turnEnd: 1, stepStart: 1, stepEnd: 1, errors: 1 }) expect(errors.map(e => e.message)).toEqual(['boom step-end']) expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason) - .toEqual({ kind: 'error', message: 'boom step-end' }) + .toEqual({ kind: 'error', step: 1, message: 'boom step-end' }) // step/end precedes turn/end (ordering contract) const e = [...agent.session.events] @@ -946,7 +878,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', message: 'provider down' } }] const adapter = new MockAdapter([errorStream, textResponse('turn 2 ok')]) const ctx = await balancedHarness(adapter) - const agent = ctx.agentLoop.create('a-double', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-double'), { model: 'mock' }) let threw = false ctx.on('agent/turn-end', () => { if (!threw) { threw = true; throw new Error('boom turn-end') } }) @@ -957,12 +889,12 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar await waitForIdle(ctx, agent) const c = boundaryCounts(agent) - // exactly one error event + one agent/error emit, despite two failTurn calls. + // exactly one error turn-end + one agent/error emit, despite two failTurn calls. expect(c.errors).toBe(1) expect(errors.map(e => e.message)).toEqual(['provider down']) expect(c.turnStart).toBe(1) expect(c.turnEnd).toBe(1) // single turn/end, balanced - expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toMatchObject({ kind: 'error', message: 'provider down' }) + expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toMatchObject({ kind: 'error', step: 1, message: 'provider down' }) // loop survives the compound failure. send(agent, 'again') @@ -970,42 +902,6 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar expect(boundaryCounts(agent).turnEnd).toBe(2) }) - it('a throwing session/event listener on the error event still closes the turn (finalizer containment)', async () => { - // failTurn appends the `error` event; Session.append pushes it BEFORE - // notifying session/event listeners, so a throwing listener leaves `error` - // in the log but must NOT abort finalization — `reason` is set before the - // append and the throw is contained, so closeTurn(false) still runs and - // turn/end is appended (the turn is balanced, not left open). - // Plain harness (no invariants oracle): the throwing listener is itself a - // session/event subscriber. A finish-error drives the boundary-error path. - const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', message: 'provider down' } }] - const adapter = new MockAdapter([errorStream, textResponse('turn 2 ok')]) - const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-errthrow', { model: 'mock' }) - - let threw = false - ctx.on('session/event', (_s, event) => { - if (!threw && event.type === 'error') { threw = true; throw new Error('boom error-event listener') } - }) - - send(agent, 'go') - await waitForIdle(ctx, agent) - - const e = [...agent.session.events] - // The error event is in the log (pushed before the listener threw)… - expect(e.some(x => x.type === 'error')).toBe(true) - // …and the turn was still closed with the error reason (finalization did not - // abort): the last event is turn/end carrying the error reason. - const last = e.at(-1) - expect(last?.type).toBe('turn/end') - expect(last?.type === 'turn/end' && last.data.reason).toMatchObject({ kind: 'error', message: 'provider down' }) - - // loop survives: a second turn runs normally. - send(agent, 'again') - await waitForIdle(ctx, agent) - expect(adapter.requests).toHaveLength(2) - }) - it('a throwing session/event listener on step/end during finalization still appends turn/end', async () => { // A throwing agent/step-start listener drives the outer catch, which calls // closeStep() during finalization. closeStep appends step/end; a @@ -1014,7 +910,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // throw is contained + surfaced via failTurn, so turn/end is still appended. const adapter = new MockAdapter([textResponse('never reached')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-stependthrow', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-stependthrow'), { model: 'mock' }) // Open a step, then make the agent/step-start emit throw (boundary throw → // outer catch → closeStep during finalization). @@ -1052,7 +948,7 @@ describe('P1-5: a started turn (and any open step) is always closed on a boundar // what throws.) const adapter = new MockAdapter([textResponse('turn 1'), textResponse('turn 2')]) const ctx = await harness(adapter) - const agent = ctx.agentLoop.create('a-turnendappend', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-turnendappend'), { model: 'mock' }) let threw = false ctx.on('session/event', (_s, event) => { @@ -1098,7 +994,7 @@ describe('P1-7: tool/result is logged under the originating call.id, not result. return Promise.resolve({ callId: CallId('wrong-proxy-id'), content: [{ type: 'text', text: 'ok' }], isError: false }) }, { prepend: true }) - const agent = ctx.agentLoop.create('a-callid', { model: 'mock' }) + const agent = ctx.agentLoop.create(AgentId('a-callid'), { model: 'mock' }) send(agent, 'use tool') await waitForIdle(ctx, agent) diff --git a/packages/core/agent-loop/tsconfig.json b/packages/core/agent-loop/tsconfig.json new file mode 100644 index 0000000000..03df67cfc4 --- /dev/null +++ b/packages/core/agent-loop/tsconfig.json @@ -0,0 +1,39 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + }, + { + "path": "../../session-persistence/session-persistence" + }, + { + "path": "../../core/system-prompt" + }, + { + "path": "../../core/tools" + }, + { + "path": "../../core/agent" + } + ] +} diff --git a/packages/agent/README.md b/packages/core/agent/README.md similarity index 63% rename from packages/agent/README.md rename to packages/core/agent/README.md index 4380cd756d..d0ec0ee614 100644 --- a/packages/agent/README.md +++ b/packages/core/agent/README.md @@ -9,7 +9,7 @@ Tracks live agents so UI, hook, and orchestrator plugins can find them without i ### Public API - `ctx.agents.register(agent: Agent): () => void` — record an **already-constructed** agent. Disposed with the calling fiber. -- `ctx.agents.get(id: string): Agent | undefined` +- `ctx.agents.get(id: AgentId): Agent | undefined` - `ctx.agents.list(): Agent[]` #### Factory seam (creation) @@ -17,8 +17,10 @@ Tracks live agents so UI, hook, and orchestrator plugins can find them without i Agent *creation* is provided by whichever plugin implements `AgentFactory` (phase 1: `dsh-agent-loop`), registered via `setFactory`. This keeps creation on the `dsh-agent` interface so consumers (UI, the ACP bridge) program against `ctx.agents` without depending on the concrete loop package. - `ctx.agents.setFactory(factory: AgentFactory): () => void` — register the creation factory (the loop calls this on construction). Throws on a second factory; the slot clears on dispose. -- `ctx.agents.create(options: CreateAgentOptions): Agent` — construct, start, AND register a new agent on a caller-supplied `sessionId` (with optional `meta.cwd`). Distinct from `register` (which only records). Throws if no factory is registered. -- `ctx.agents.resume(options: ResumeAgentOptions): Promise` — load a persisted session ([session persistence](../../docs/rfc/implemented/2026-06-14-session-persistence.md)) and resume an agent on it. Async; rejects if no factory is registered, or if the factory finds session persistence unconfigured. +- `ctx.agents.create(options: CreateAgentOptions): AgentHandle` — construct, start, AND register a new agent on a caller-supplied `sessionId` (with optional `meta.cwd`). Distinct from `register` (which only records). Throws if no factory is registered. +- `ctx.agents.resume(options: ResumeAgentOptions): Promise` — load a persisted session ([session persistence](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md)) and resume an agent on it. Async; rejects if no factory is registered, or if the factory finds session persistence unconfigured. + +`AgentHandle = { agent: Agent; dispose(): Promise }`. The disposer is a **capability** — only the holder can tear this agent down. `dispose()` stops the loop, `await`s its exit (quiescence — NOT just the `disposed` status flip), unregisters the agent, and removes its session from the store, in an order that captures the loop's final `session/flush` before the session is detached. `ctx.agents.get(id)` still returns a bare `Agent` — the handle is only for the OWNER that created it. The ACP bridge is the production consumer (one handle per session, disposed on disconnect/teardown); config-created agents are owned by the loop fiber and never need a handle. ### Events @@ -53,9 +55,9 @@ The handle every plugin programs against: - `agent.send(content, options?)` — queue a message; starts a turn when idle - `agent.steer(content, options?)` — steer a running turn (inject between steps); behaves like `send` when idle -- `agent.inject(content, options?)` — inject in-session context (context/message event); the next request sees it. Does not run the model. While a turn is open it joins that turn; while idle it is wrapped in a one-shot `injection` turn so every event stays turn-enclosed ([the turn-enclosure invariant](../../docs/rfc/implemented/2026-06-15-turn-enclosure-invariant.md)) -- `agent.abort(reason?)` — abort the in-flight step -- `agent.whenIdle()` — resolve once the agent reaches quiescence after settling out of `running` (idle → immediately; disposed → awaits the loop exit), the signal a teardown awaits (`abort()` then `await whenIdle()`). Observes the transition without disposing the agent. +- `agent.inject(content, options?)` — inject in-session context (context/message event); the next request sees it. Does not run the model. While a turn is open it joins that turn; while idle it is wrapped in a one-shot `injection` turn so every event stays turn-enclosed ([the turn-enclosure invariant](../../../docs/rfc/implemented/architecture/2026-06-15-turn-enclosure-invariant.md)) +- `agent.cancel(reason?)` — cancel ALL pending work: clears the queued + steering FIFOs, aborts the in-flight step, and drops a turn about to start (the pre-step window) so a queued-but-not-started prompt never runs. A UI/ACP `session/cancel` maps to this. The single public stop primitive. Idle with nothing pending → a safe no-op. +- `agent.whenIdle()` — resolve once the agent reaches quiescence after settling out of `running` (idle → immediately; disposed → awaits the loop exit). A non-owner's quiescence-observation hook: it observes the work settling WITHOUT tearing the agent down. Teardown is separate — a lifecycle owner stops and unregisters via `AgentHandle.dispose()`, which awaits the loop exit directly. - `agent.session`, `agent.status`, `agent.options`, `agent.id` ### Extension points diff --git a/packages/agent/package.json b/packages/core/agent/package.json similarity index 74% rename from packages/agent/package.json rename to packages/core/agent/package.json index a215f35fb3..fa6946b8cf 100644 --- a/packages/agent/package.json +++ b/packages/core/agent/package.json @@ -5,26 +5,30 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "cordis": "^4.0.0-rc.6" diff --git a/packages/agent/src/index.ts b/packages/core/agent/src/index.ts similarity index 65% rename from packages/agent/src/index.ts rename to packages/core/agent/src/index.ts index c9181081a3..158946178c 100644 --- a/packages/agent/src/index.ts +++ b/packages/core/agent/src/index.ts @@ -7,7 +7,7 @@ import { Context, Service } from 'cordis' import type { SessionId } from '@deepseek-ai/dsh-session' -import type { Agent, AgentOptions } from './types.ts' +import type { Agent, AgentId, AgentOptions } from './types.ts' export * from './types.ts' @@ -26,9 +26,9 @@ declare module 'cordis' { */ export interface CreateAgentOptions { /** The agent's id (the registry handle). */ - agentId: string + agentId: AgentId /** The live session's id (NOT derived from agentId). */ - sessionId: string + sessionId: SessionId /** * Session creation metadata: validated absolute `cwd` and `parentSession` * fork lineage. Mirrors the `cwd`/`parentSession` fields of @@ -47,13 +47,30 @@ export interface CreateAgentOptions { */ export interface ResumeAgentOptions { /** The agent's id (the registry handle). */ - agentId: string + agentId: AgentId /** The persisted session id to load and resume on. */ - resumeSessionId: string + resumeSessionId: SessionId /** Per-agent options (model, system prompt). */ agentOptions?: AgentOptions } +/** + * An owned agent plus its disposer, returned by {@link AgentRegistry.create} / + * {@link AgentRegistry.resume}. The disposer is a CAPABILITY: only the holder + * can tear this agent down. `dispose()` unregisters the agent, stops its loop, + * awaits the loop's exit (quiescence — NOT just the `disposed` status flip), and + * removes the agent's session from the store, in an order that captures the + * loop's final `session/flush` before the session is detached. + * + * `ctx.agents.get(id)` still returns a bare {@link Agent} — the handle is only + * for the OWNER that created it. Config-created agents (the loop's own startup) + * are owned by the loop fiber and never need a handle. + */ +export interface AgentHandle { + agent: Agent + dispose(): Promise +} + /** * The agent-creation factory the loop implementation provides to the registry * via {@link AgentRegistry.setFactory}. Kept on the `dsh-agent` interface so @@ -61,14 +78,18 @@ export interface ResumeAgentOptions { * depending on the concrete `dsh-agent-loop` package. */ export interface AgentFactory { - /** Create, start, and register a new agent on a caller-supplied session id. */ - createAgent(options: CreateAgentOptions): Agent + /** + * Create, start, and register a new agent on a caller-supplied session id. + * Returns an {@link AgentHandle} — the owner disposes it to tear down exactly + * this agent (unregister + stop loop + await quiescence + remove session). + */ + createAgent(options: CreateAgentOptions): AgentHandle /** * Load a persisted session and resume an agent on it. Async because it awaits * `ctx.sessionPersistence.load`; must be called after that service exists - * (consumers inject `sessionPersistence`). + * (consumers inject `sessionPersistence`). Returns an {@link AgentHandle}. */ - resume(options: ResumeAgentOptions): Promise + resume(options: ResumeAgentOptions): Promise } /** Thrown when create/resume is called before an agent factory is registered. */ @@ -82,7 +103,7 @@ const NO_FACTORY_MESSAGE = 'no agent factory registered (load an agent-loop plug * {@link setFactory}. */ export class AgentRegistry extends Service { - private store = new Map() + private store = new Map() private factory: AgentFactory | undefined constructor(ctx: Context) { @@ -107,9 +128,10 @@ export class AgentRegistry extends Service { * Create, start, and register a new agent through the registered factory. * Distinct from {@link register} (which records an already-constructed * agent): this constructs the agent and its session. Throws if no factory is - * registered. + * registered. Returns an {@link AgentHandle} — the owner disposes it to tear + * down exactly this agent. */ - create(options: CreateAgentOptions): Agent { + create(options: CreateAgentOptions): AgentHandle { if (this.factory === undefined) throw new Error(NO_FACTORY_MESSAGE) return this.factory.createAgent(options) } @@ -117,9 +139,9 @@ export class AgentRegistry extends Service { /** * Load a persisted session and resume an agent on it through the registered * factory. Rejects if no factory is registered; the factory rejects if - * session persistence is not configured. + * session persistence is not configured. Returns an {@link AgentHandle}. */ - async resume(options: ResumeAgentOptions): Promise { + async resume(options: ResumeAgentOptions): Promise { if (this.factory === undefined) throw new Error(NO_FACTORY_MESSAGE) return this.factory.resume(options) } @@ -142,7 +164,22 @@ export class AgentRegistry extends Service { // The duplicate throw above fires before any mutation — it leaks nothing. yield () => { this.store.delete(agent.id) - this.ctx.emit('agent/disposed', agent) + // CONTAIN a throwing `agent/disposed` listener: this disposer runs as + // one link in the owning fiber/effect's disposal chain, and Cordis + // chains later disposers with `task.then(next)` — so an UNCAUGHT throw + // here rejects the chain and SKIPS every later disposer. When this + // registration shares a composite effect with a session (the agent + // factory's `AgentLoop.start`, where the session-detach disposer runs + // AFTER this one), a swallowed-less throw would strand the session in + // the store with `onAppend` attached — a leak AND a durability hole. + // The store entry is already removed above (the useful state), so + // logging the listener bug and continuing is correct (mirrors the + // guarded `agent/status` emit in dsh-agent-loop's ReactLoopAgent). + try { + this.ctx.emit('agent/disposed', agent) + } catch (error: unknown) { + this.ctx.logger.warn(`agent "${agent.id}": agent/disposed listener threw: ${String(error)}`) + } } this.ctx.emit('agent/created', agent) }.bind(this), 'agents.register()') @@ -151,7 +188,7 @@ export class AgentRegistry extends Service { return () => void dispose() } - get(id: string): Agent | undefined { + get(id: AgentId): Agent | undefined { return this.store.get(id) } diff --git a/packages/agent/src/types.ts b/packages/core/agent/src/types.ts similarity index 60% rename from packages/agent/src/types.ts rename to packages/core/agent/src/types.ts index 56805a4a23..6d27bf7256 100644 --- a/packages/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -9,7 +9,8 @@ * @module @deepseek-ai/dsh-agent/types */ -import type { Branded, ContentBlock, GenerateOptions, Message, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm' +import type { Branded } from '@deepseek-ai/dsh-brand' +import type { ContentBlock, GenerateOptions, Message, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm' /** Identifies one live agent in the registry. */ export type AgentId = Branded<'AgentId'> @@ -40,7 +41,7 @@ export type AgentStatus = 'idle' | 'running' | 'disposed' /** * The agent handle — the surface every plugin (UI, hooks, orchestrators) * programs against. The concrete implementation lives in - * `@deepseek-ai/dsh-agent-loop` (class `LoopAgent`); nothing outside the loop + * `@deepseek-ai/dsh-agent-loop` (class `ReactLoopAgent`); nothing outside the loop * package should depend on the implementation. */ export interface Agent { @@ -78,17 +79,35 @@ export interface Agent { */ inject(content: ContentBlock[], options?: SendOptions): void - /** Abort the in-flight step (if any); the turn ends with reason 'aborted'. */ - abort(reason?: string): void + /** + * Cancel ALL pending work for the agent. `cancel()`: + * + * - clears the queued FIFO (un-started prompts never run) and the steering + * FIFO (steering for the cancelled turn is dropped, not re-enqueued); + * - aborts the in-flight step if one is running (the turn ends `aborted`); + * - drops a turn that is about to start (a `cancel()` landing in the + * pre-step window — after a `send()` queued but before the loop flips to + * `running`, or after `running` is emitted but before the first step) so + * that queued prompt does not run and cannot be batched into the cancelled + * turn. + * + * After `cancel()`, `whenIdle()` resolves on the post-cancel quiescent state. + * `cancel()` on an idle agent with nothing queued or running is a safe no-op + * — it does NOT arm anything that would drop a later legitimate prompt. + */ + cancel(reason?: string): void /** * Resolve once the agent has reached quiescence after settling out of - * `running`, or immediately if it is already idle with no queued work. The - * quiescence signal a teardown awaits: `agent.abort()` then - * `await agent.whenIdle()` guarantees queued/running work has fully stopped - * before the caller proceeds (a closing ACP connection, a disposing UI - * plugin), rather than returning while the driver is still streaming or about - * to start a queued turn. + * `running`, or immediately if it is already idle with no queued work. A + * non-owner's quiescence-observation hook: a consumer that does NOT own the + * agent's lifecycle awaits this to proceed only after queued/running work has + * fully stopped, rather than returning while the driver is still streaming or + * about to start a queued turn — without itself tearing the agent down. (A + * lifecycle OWNER does not need it: `AgentHandle.dispose()` already awaits the + * loop-exit promise directly as part of stopping and unregistering. So this is + * for a non-owning observer — e.g. a test awaiting a turn to settle, or a + * monitor — that wants the settle signal but must not dispose the agent.) * * "Quiescence", not merely "status changed": a disposed agent emits * `agent/status('disposed')` from inside its disposer, BEFORE the driver loop @@ -96,10 +115,6 @@ export interface Agent { * to actually exit (the implementation chains the loop-exit promise), not just * observe the status flip. A mid-step disposal that never reaches `idle` still * unblocks the await this way. - * - * Distinct from disposal: `whenIdle()` observes the transition WITHOUT tearing - * the agent down. A consumer that owns the agent's lifecycle disposes it - * separately. */ whenIdle(): Promise @@ -113,48 +128,94 @@ export interface Agent { declare module 'cordis' { interface Events { // ---- lifecycle (emit) ---- - /** An agent was registered. */ + /** + * An agent was registered in the {@link AgentRegistry} and is ready to + * receive messages. + * @mode emit + */ 'agent/created'(agent: Agent): void - /** An agent was disposed. */ + /** + * An agent was disposed and removed from the registry; its fiber and any + * in-flight turn have been torn down. + * @mode emit + */ 'agent/disposed'(agent: Agent): void - /** Agent status changed (idle/running/disposed). */ + /** + * Agent status changed (`idle` ⇄ `running`, or → `disposed`). Drive + * lifecycle off this transition, never off a status you just requested — + * `send()` does not flip status to `running` before it returns. + * @mode emit + */ 'agent/status'(agent: Agent, status: AgentStatus): void /** * A message entered the agent's inbox (queued or steering). `source` is * the resolved source (defaults applied), not the caller's raw options. + * @mode emit */ 'agent/queued'(agent: Agent, content: ContentBlock[], info: { source: MessageSource; steering: boolean }): void // ---- turn/step boundaries (emit) ---- + /** + * A turn began. `turn` is the 1-based turn number within the session. + * @mode emit + */ 'agent/turn-start'(agent: Agent, turn: number): void + /** + * A turn ended. `reason` distinguishes a clean stop from a truncated or + * aborted one (`completed` | `aborted` | `error` | `disposed` | `max-tokens`). + * @mode emit + */ 'agent/turn-end'(agent: Agent, turn: number, reason: TurnEndReason): void + /** + * A step (one model call plus its tool dispatch) began. `step` is 1-based + * within the turn; a turn runs one or more steps. + * @mode emit + */ 'agent/step-start'(agent: Agent, turn: number, step: number): void + /** + * A step ended. + * @mode emit + */ 'agent/step-end'(agent: Agent, turn: number, step: number): void // ---- interception seams (waterfall) ---- /** - * Waterfall: mutate the fully-assembled GenerateOptions before the model - * call (hooks, compaction, model switching, tool filtering, …). + * Waterfall: mutate the fully-assembled {@link GenerateOptions} before the + * model call (hooks, compaction, model switching, tool filtering, …). Call + * `next()` to delegate, or return without it to short-circuit. + * @mode waterfall */ 'agent/request'(agent: Agent, turn: number, step: number, options: GenerateOptions, next: () => Promise): Promise /** - * Waterfall: post-process the assembled assistant message before tool - * dispatch (validation, content rewriting, …). + * Waterfall: post-process the assembled assistant {@link Message} before + * tool dispatch (validation, content rewriting, …). + * @mode waterfall */ 'agent/step-result'(agent: Agent, turn: number, step: number, message: Message, next: () => Promise): Promise /** * Waterfall: override the turn-continuation decision. The default * (computed by the loop) is `hadToolCalls || steeringInjected`. Listeners * can force-continue (/goal, /loop) or force-stop (budget guards). + * @mode waterfall */ 'agent/turn-continuation'(agent: Agent, turn: number, defaultDecision: boolean, next: () => Promise): Promise // ---- streaming + tool notifications (emit) ---- - /** A raw stream chunk arrived (token-level UI/log feed). */ + /** + * A raw {@link StreamChunk} arrived from the model (token-level UI/log feed). + * @mode emit + */ 'agent/stream-chunk'(agent: Agent, turn: number, step: number, chunk: StreamChunk): void - /** Steering content was injected into a running turn. */ + /** + * Steering content was injected into a running turn. + * @mode emit + */ 'agent/steering'(agent: Agent, turn: number, content: ContentBlock[], source: MessageSource): void - /** A step or turn errored. */ + /** + * A step or turn errored. The loop reports a failure here (plus the logger) + * even when the error has no in-turn position for a session `error` event. + * @mode emit + */ 'agent/error'(agent: Agent, turn: number, step: number, error: Error): void } } diff --git a/packages/agent/tests/agent.spec.ts b/packages/core/agent/tests/agent.spec.ts similarity index 70% rename from packages/agent/tests/agent.spec.ts rename to packages/core/agent/tests/agent.spec.ts index 1994c785a6..c344cd2a6f 100644 --- a/packages/agent/tests/agent.spec.ts +++ b/packages/core/agent/tests/agent.spec.ts @@ -13,7 +13,7 @@ function stubAgent(rawId: string): Agent { send() {}, steer() {}, inject() {}, - abort() {}, + cancel() {}, whenIdle() { return Promise.resolve() }, } } @@ -31,12 +31,12 @@ describe('AgentRegistry', () => { const agent = stubAgent('a1') const dispose = ctx.agents.register(agent) expect(created).toEqual(['a1']) - expect(ctx.agents.get('a1')).toBe(agent) + expect(ctx.agents.get(AgentId('a1'))).toBe(agent) expect(ctx.agents.list()).toEqual([agent]) dispose() expect(disposed).toEqual(['a1']) - expect(ctx.agents.get('a1')).toBeUndefined() + expect(ctx.agents.get(AgentId('a1'))).toBeUndefined() }) it('rejects duplicate ids and unregisters on fiber dispose (HMR safety)', async () => { @@ -65,14 +65,14 @@ describe('AgentRegistry', () => { // The throwing emit must roll the entry back, not leak it. expect(() => ctx.agents.register(stubAgent('main'))).toThrow('boom created listener') - expect(ctx.agents.get('main')).toBeUndefined() // rolled back, not leaked + expect(ctx.agents.get(AgentId('main'))).toBeUndefined() // rolled back, not leaked // A subsequent listener-free register of the SAME id succeeds and is // tracked exactly once (the duplicate-id check is not wedged). const dispose = ctx.agents.register(stubAgent('main')) expect(ctx.agents.list().map(a => a.id)).toEqual(['main']) dispose() - expect(ctx.agents.get('main')).toBeUndefined() + expect(ctx.agents.get(AgentId('main'))).toBeUndefined() }) }) @@ -81,8 +81,14 @@ describe('AgentRegistry factory seam', () => { function stubFactory() { const calls: { create: unknown[]; resume: unknown[] } = { create: [], resume: [] } const factory: import('@deepseek-ai/dsh-agent').AgentFactory = { - createAgent(options) { calls.create.push(options); return stubAgent(options.agentId) }, - resume(options) { calls.resume.push(options); return Promise.resolve(stubAgent(options.agentId)) }, + createAgent(options) { + calls.create.push(options) + return { agent: stubAgent(options.agentId), dispose: () => Promise.resolve() } + }, + resume(options) { + calls.resume.push(options) + return Promise.resolve({ agent: stubAgent(options.agentId), dispose: () => Promise.resolve() }) + }, } return { factory, calls } } @@ -90,8 +96,8 @@ describe('AgentRegistry factory seam', () => { it('create()/resume() throw when no factory is registered', async () => { const ctx = new Context() await ctx.plugin(AgentRegistry) - expect(() => ctx.agents.create({ agentId: 'a', sessionId: 's' })).toThrow(/no agent factory/) - await expect(ctx.agents.resume({ agentId: 'a', resumeSessionId: 's' })).rejects.toThrow(/no agent factory/) + expect(() => ctx.agents.create({ agentId: AgentId('a'), sessionId: SessionId('s') })).toThrow(/no agent factory/) + await expect(ctx.agents.resume({ agentId: AgentId('a'), resumeSessionId: SessionId('s') })).rejects.toThrow(/no agent factory/) }) it('setFactory registers a factory; create/resume delegate to it', async () => { @@ -100,13 +106,13 @@ describe('AgentRegistry factory seam', () => { const { factory, calls } = stubFactory() ctx.agents.setFactory(factory) - const created = ctx.agents.create({ agentId: 'c1', sessionId: 'sess-1', meta: { cwd: '/w' } }) - expect(created.id).toBe('c1') - expect(calls.create).toEqual([{ agentId: 'c1', sessionId: 'sess-1', meta: { cwd: '/w' } }]) + const created = ctx.agents.create({ agentId: AgentId('c1'), sessionId: SessionId('sess-1'), meta: { cwd: '/w' } }) + expect(created.agent.id).toBe('c1') + expect(calls.create).toEqual([{ agentId: AgentId('c1'), sessionId: SessionId('sess-1'), meta: { cwd: '/w' } }]) - const resumed = await ctx.agents.resume({ agentId: 'r1', resumeSessionId: 'old-sess' }) - expect(resumed.id).toBe('r1') - expect(calls.resume).toEqual([{ agentId: 'r1', resumeSessionId: 'old-sess' }]) + const resumed = await ctx.agents.resume({ agentId: AgentId('r1'), resumeSessionId: SessionId('old-sess') }) + expect(resumed.agent.id).toBe('r1') + expect(calls.resume).toEqual([{ agentId: AgentId('r1'), resumeSessionId: SessionId('old-sess') }]) }) it('setFactory rejects a second factory', async () => { @@ -123,10 +129,10 @@ describe('AgentRegistry factory seam', () => { const fiber = await ctx.plugin(Object.assign((inner: Context) => { dispose = inner.agents.setFactory(stubFactory().factory) }, { inject: ['agents'] })) - expect(() => ctx.agents.create({ agentId: 'a', sessionId: 's' })).not.toThrow() + expect(() => ctx.agents.create({ agentId: AgentId('a'), sessionId: SessionId('s') })).not.toThrow() void dispose await fiber.dispose() // factory slot cleared → create throws again - expect(() => ctx.agents.create({ agentId: 'a2', sessionId: 's2' })).toThrow(/no agent factory/) + expect(() => ctx.agents.create({ agentId: AgentId('a2'), sessionId: SessionId('s2') })).toThrow(/no agent factory/) }) }) diff --git a/packages/core/agent/tests/gen-cordis-catalog.spec.ts b/packages/core/agent/tests/gen-cordis-catalog.spec.ts new file mode 100644 index 0000000000..ee2ce47699 --- /dev/null +++ b/packages/core/agent/tests/gen-cordis-catalog.spec.ts @@ -0,0 +1,83 @@ +/** + * Negative-path tests for the cordis catalog generator (`scripts/gen-cordis-catalog.ts`). + * + * The generated catalog is frozen by a regenerate-and-diff freshness gate, so + * the freshness half is exercised by `pnpm run verify-cordis-catalog` in CI. + * What a freshness diff CANNOT prove is that the generator REJECTS malformed + * source the way it promises to — a missing `@mode` tag, or a tag that + * contradicts the signature shape. These tests drive `collectEvents()` against + * synthetic fixture packages to prove each guard fires (and that a well-formed + * event passes), mirroring the drift-guard negative tests for verify-type-equiv. + */ + +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { collectEvents } from '../../../../scripts/gen-cordis-catalog.ts' + +/** Write a fixture package exposing one `interface Events` block and return the + * scan root to hand `collectEvents`. */ +function fixtureRoot(eventsBlock: string): string { + const root = mkdtempSync(join(tmpdir(), 'cordis-catalog-')) + const dir = join(root, 'packages', 'group', 'fix', 'src') + mkdirSync(dir, { recursive: true }) + writeFileSync( + join(dir, 'index.ts'), + `declare module 'cordis' {\n interface Events {\n${eventsBlock}\n }\n}\n`, + ) + return root +} + +const roots: string[] = [] +const make = (block: string): string => { + const r = fixtureRoot(block) + roots.push(r) + return r +} + +afterEach(() => { + while (roots.length) rmSync(roots.pop()!, { recursive: true, force: true }) +}) + +describe('gen-cordis-catalog collectEvents', () => { + it('extracts a well-formed event with its @mode and JSDoc', () => { + const events = collectEvents(make( + ' /**\n * A thing happened.\n * @mode emit\n */\n \'fix/happened\'(id: string): void', + )) + expect(events).toHaveLength(1) + expect(events[0]).toMatchObject({ name: 'fix/happened', scope: 'fix', mode: 'emit', doc: 'A thing happened.' }) + }) + + it('classifies a trailing-next signature as a waterfall', () => { + const events = collectEvents(make( + ' /**\n * Intercept it.\n * @mode waterfall\n */\n \'fix/intercept\'(x: number, next: () => Promise): Promise', + )) + expect(events[0]?.mode).toBe('waterfall') + }) + + it('accepts a parallel (awaited, no next) event by trusting the tag', () => { + const events = collectEvents(make( + ' /**\n * Flush.\n * @mode parallel\n */\n \'fix/flush\'(): Promise | void', + )) + expect(events[0]?.mode).toBe('parallel') + }) + + it('hard-errors when an event is missing its @mode tag', () => { + expect(() => collectEvents(make( + ' /** No mode here. */\n \'fix/untagged\'(id: string): void', + ))).toThrow(/missing an @mode tag/) + }) + + it('hard-errors when @mode contradicts a trailing-next (waterfall) shape', () => { + expect(() => collectEvents(make( + ' /**\n * Mislabeled.\n * @mode emit\n */\n \'fix/wrong\'(x: number, next: () => Promise): Promise', + ))).toThrow(/trailing 'next' parameter .* tagged '@mode emit'/) + }) + + it('hard-errors when @mode waterfall has no trailing next to delegate to', () => { + expect(() => collectEvents(make( + ' /**\n * Not actually a waterfall.\n * @mode waterfall\n */\n \'fix/nonext\'(id: string): void', + ))).toThrow(/tagged '@mode waterfall' but has no trailing 'next'/) + }) +}) diff --git a/packages/core/agent/tsconfig.json b/packages/core/agent/tsconfig.json new file mode 100644 index 0000000000..4d23ac46d3 --- /dev/null +++ b/packages/core/agent/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + } + ] +} diff --git a/packages/session/README.md b/packages/core/session/README.md similarity index 50% rename from packages/session/README.md rename to packages/core/session/README.md index 7cf443fa71..682d5517dd 100644 --- a/packages/session/README.md +++ b/packages/core/session/README.md @@ -8,10 +8,20 @@ Creates and holds event-sourced `Session` instances. Persistence is intentionall ### Public API -- `ctx.sessions.create(id?: string, options?: { seed?: SessionEvent[]; meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number } }): Session` — Create a session. `options.seed` replays/forks an existing event log; `options.meta` attaches creation metadata (validated absolute `cwd`, `parentSession` lineage) as the immutable `SessionHeader`. The store fills `version`/`id` and defaults `createdAt` to now; a caller reconstructing a persisted session passes the original `createdAt` to preserve it. Disposed with the calling fiber. -- `ctx.sessions.get(id: string): Session | undefined` +- `ctx.sessions.create(id?: SessionId, options?: { seed?: SessionEvent[]; meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number } }): Session` — Create a session. `options.seed` replays/forks an existing event log; `options.meta` attaches creation metadata (validated absolute `cwd`, `parentSession` lineage) as the immutable `SessionHeader`. The store fills `version`/`id` and defaults `createdAt` to now; a caller reconstructing a persisted session passes the original `createdAt` to preserve it. Disposed with the calling fiber. +- `ctx.sessions.get(id: SessionId): Session | undefined` - `ctx.sessions.list(): Session[]` +#### Advanced: ordered-teardown lifecycle primitives + +`create()` covers the common case (the session is owned by the calling fiber). When a session must be torn down **in order with another resource** — so a final flush is captured before `onAppend` detaches — `create()`'s self-contained effect is wrong, because a fiber unload disposes sibling effects *concurrently*. For that, split the lifecycle and fold it into the owner's single effect: + +- `ctx.sessions.prepare(id?, options?): Session` — validate the id/cwd and construct the `Session`, WITHOUT entering it into the store. Same options as `create`. +- `ctx.sessions.enter(session): () => void` — wire `onAppend` → `session/event` and add the session to the store; returns the DETACH disposer. Does NOT emit `session/created` (the caller yields the disposer first, then calls `announce`, so a throwing listener rolls the attach back). The id was already validated by `prepare`, which runs in the same synchronous sequence, so `enter` does not re-check. +- `ctx.sessions.announce(session): void` — emit `session/created` for an entered session. + +`dsh-agent-loop`'s `AgentLoop.start` is the canonical consumer: it yields `enter`'s detach disposer, the registry unregister, and the loop-stop disposer into ONE composite effect, so teardown stops + awaits the loop (final flush captured) BEFORE detaching the session — whether the trigger is the `AgentHandle`'s `dispose()` or a fiber unload. + ### Events | Event | Mode | Purpose | @@ -27,17 +37,15 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. - `session.append(type, data): SessionEvent` — synchronous, never blocks on I/O. **Throws** if `data` is not losslessly JSON-serializable (BigInt, function, symbol, undefined, non-finite number, circular ref, or an exotic object like Map/Set/Date) — the event log is the durable source of truth, so this invariant is enforced at the source (exported as `isJsonValue` for backends to reuse on their replay/fork entry points). - `session.deriveMessages(): Message[]` — derive the LLM message history from the event log. Raw `assistant/chunk` events are skipped; `context/message` and `steering/message` render as tagged synthetic user messages. - `session.events`, `session.seq`, `session.id` -- `session.header: SessionHeader` — immutable creation metadata (`version`, `id`, `createdAt`, optional `cwd`/`parentSession`). Kept out of the event log (a storage concern, not replayable state); a minimal v1 header is synthesized for bare `Session` construction. +- `session.header: SessionHeader` — immutable creation metadata (`version`, `id`, `createdAt`, optional `cwd`/`parentSession`). Kept out of the event log (a storage concern, not replayable state); a minimal header (stamped with the current `SESSION_FORMAT_VERSION`) is synthesized for bare `Session` construction. ### Metadata types (`types.ts`) -- `SessionHeader` — immutable, written once: `{ version, id, createdAt, cwd?, parentSession? }`. -- `SessionSummary` — mutable, updateable without touching the log: `{ updatedAt, title?, firstPrompt? }`. -- `SessionMeta = SessionHeader & SessionSummary` — owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export these rather than own them (which would force a package cycle). +- `SessionHeader` — immutable session metadata, written once: `{ version, id, createdAt, cwd?, parentSession? }`. Owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export it rather than own it (which would force a package cycle). ### Session event vocabulary (`types.ts`) -The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`, `usage`, `error`. +The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`. Token usage rides on `assistant/message.usage`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. Merge-extensible via `SessionEventMap` — a compaction plugin adds `compaction/marker`, etc. @@ -45,7 +53,7 @@ Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types ### Extension points -- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`/`SessionSummary`/`SessionMeta`, `session.header`) is what such a backend stores beside the log. +- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log. - Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. ### What is NOT here (TODO) diff --git a/packages/session/package.json b/packages/core/session/package.json similarity index 71% rename from packages/session/package.json rename to packages/core/session/package.json index f4aa5839bc..8c6645e37e 100644 --- a/packages/session/package.json +++ b/packages/core/session/package.json @@ -5,25 +5,29 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "cordis": "^4.0.0-rc.6" } diff --git a/packages/session/src/index.ts b/packages/core/session/src/index.ts similarity index 64% rename from packages/session/src/index.ts rename to packages/core/session/src/index.ts index 4796c05f51..cef91c110c 100644 --- a/packages/session/src/index.ts +++ b/packages/core/session/src/index.ts @@ -9,7 +9,7 @@ import { Context, Service } from 'cordis' import { isAbsolute } from 'node:path' import type { ContentBlock, Message, MessageSource } from '@deepseek-ai/dsh-llm' -import { SessionId } from './types.ts' +import { SESSION_FORMAT_VERSION, SessionId } from './types.ts' import type { CreateSessionOptions, SessionEvent, SessionEventMap, SessionEventType, SessionHeader } from './types.ts' import { isJsonValue } from './json.ts' @@ -23,15 +23,24 @@ declare module 'cordis' { } interface Events { - /** A session was created in the store. */ + /** + * A session was created in the store. + * @mode emit + */ 'session/created'(session: Session): void - /** An event was appended to a session log (sync, fire-and-forget). */ + /** + * An event was appended to a session log (sync, fire-and-forget). This is + * the per-append feed a UI or invariant plugin tails. + * @mode emit + */ 'session/event'(session: Session, event: SessionEvent): void /** * Awaited durability checkpoint. The agent loop awaits * `ctx.parallel('session/flush', session)` at every turn end; persistence - * plugins (JSONL, SQLite) drain their write-behind - * buffers here and on fiber dispose. + * plugins (JSONL, SQLite) drain their write-behind buffers here and on + * fiber dispose. Awaited (parallel), not a waterfall: every listener runs + * and the loop waits for all of them, but none can veto. + * @mode parallel */ 'session/flush'(session: Session): Promise | void } @@ -70,9 +79,10 @@ export class Session { /** * Immutable creation metadata (format version, cwd, lineage). Supplied by * the store via `ctx.sessions.create()`. When a `Session` is constructed - * bare (tests, ad-hoc replay), a minimal v1 header is synthesized so - * `session.header` is always present. Kept out of the event log — it is a - * storage concern, not replayable conversation state. + * bare (tests, ad-hoc replay), a minimal header is synthesized (stamped with + * the current {@link SESSION_FORMAT_VERSION}) so `session.header` is always + * present. Kept out of the event log — it is a storage concern, not + * replayable conversation state. */ readonly header: SessionHeader @@ -103,7 +113,7 @@ export class Session { // structuredClone can never hit a non-cloneable value here. this.log = seed.map(event => structuredClone(event)) } - this.header = header ?? { version: 1, id, createdAt: Date.now() } + this.header = header ?? { version: SESSION_FORMAT_VERSION, id, createdAt: Date.now() } } get events(): readonly SessionEvent[] { @@ -151,7 +161,10 @@ export class Session { * * - `user/message` → user message * - `assistant/message` → assistant message (chunks are skipped — they are - * replay/UI data; the assembled message is authoritative for history) + * replay/UI data; the assembled message is authoritative for history). An + * EMPTY-content assistant/message is skipped: a max-tokens step cut off with + * no content still records an assistant/message to host its `usage`, but a + * content-less assistant turn must not enter the provider transcript. * - `tool/result` → user message carrying a tool-result block * - `context/message` / `steering/message` → tagged synthetic user messages * at their chronological position @@ -168,8 +181,7 @@ export class Session { const messages: Message[] = [] for (const event of this.log) { // Intentionally non-exhaustive: only message-producing events derive - // history; turn/step boundaries, chunks, usage, and errors are - // trace/replay data. + // history; turn/step boundaries and chunks are trace/replay data. // eslint-disable-next-line @typescript-eslint/switch-exhaustiveness-check switch (event.type) { case 'user/message': { @@ -177,6 +189,10 @@ export class Session { break } case 'assistant/message': { + // Skip an empty-content assistant/message: it exists only to host a + // max-tokens step's usage and must not inject a content-less assistant + // turn into the provider transcript. + if (event.data.content.length === 0) break messages.push({ role: 'assistant', content: structuredClone(event.data.content) }) break } @@ -211,7 +227,7 @@ export class Session { * subscribe to `session/event` and flush on `session/flush` / dispose. */ export class SessionStore extends Service { - private store = new Map() + private store = new Map() private counter = 0 constructor(ctx: Context) { @@ -219,17 +235,48 @@ export class SessionStore extends Service { } /** - * Create a session. `options.seed` populates the session with a copy of - * those events (replay/fork); `options.meta` attaches creation metadata - * (validated absolute `cwd`, `parentSession` lineage) as the immutable - * {@link SessionHeader} (the store fills `version`/`id`/`createdAt`). The - * session is a Cordis effect: disposing the calling fiber stops event - * notification and removes the session from the store. + * Create a session owned by the calling fiber: disposing that fiber stops + * event notification and removes the session from the store. `options.seed` + * populates the session with a copy of those events (replay/fork); + * `options.meta` attaches creation metadata (validated absolute `cwd`, + * `parentSession` lineage) as the immutable {@link SessionHeader} (the store + * fills `version`/`id`/`createdAt`). + * + * For an agent whose session must be torn down IN ORDER with its loop (so the + * loop's final flush is captured before `onAppend` detaches), do NOT use this + * — fold the session lifecycle into the agent's own effect via + * {@link prepare} + {@link enter} + {@link announce} (see `dsh-agent-loop`'s + * `startOwned`). * * @throws if a session with `id` already exists, or if `meta.cwd` is a * non-absolute path (storage backends key directories off it). */ - create(id?: string, options?: CreateSessionOptions): Session { + create(id?: SessionId, options?: CreateSessionOptions): Session { + const session = this.prepare(id, options) + // Single effect owned by the calling fiber. Yield the detach BEFORE + // announcing so a throwing `session/created` listener rolls the attach back + // (the generator effect disposes already-yielded disposers on a throw) + // instead of leaking the store entry + onAppend. + this.ctx.effect(function* (this: SessionStore) { + yield this.enter(session) + this.announce(session) + }.bind(this), 'sessions.create()') + return session + } + + /** + * Build a session WITHOUT entering it into the store — validate the id/cwd and + * construct the {@link Session} (with its immutable {@link SessionHeader}). + * Pairs with {@link enter} + {@link announce}: a caller that owns a composite + * `ctx.effect` (the agent factory) folds the session lifecycle into that ONE + * effect so a fiber unload tears the session + agent down as a single ORDERED + * chain rather than as racing sibling effects — which would detach `onAppend` + * before the loop's closing `session/flush`, dropping the closing events. + * + * @throws if a session with `id` already exists, or if `meta.cwd` is a + * non-absolute path. + */ + prepare(id?: SessionId, options?: CreateSessionOptions): Session { const sessionId = SessionId(id ?? `session-${++this.counter}`) if (this.store.has(sessionId)) throw new Error(`session "${sessionId}" already exists`) const cwd = options?.meta?.cwd @@ -237,32 +284,51 @@ export class SessionStore extends Service { throw new Error(`session cwd must be an absolute path, got "${cwd}"`) } const header: SessionHeader = { - version: 1, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: options?.meta?.createdAt ?? Date.now(), ...cwd !== undefined ? { cwd } : {}, ...options?.meta?.parentSession !== undefined ? { parentSession: options.meta.parentSession } : {}, } - const session = new Session(sessionId, options?.seed, header) - this.ctx.effect(function* (this: SessionStore) { - session.onAppend = (event) => { this.ctx.emit('session/event', session, event) } - this.store.set(sessionId, session) - // Yield the rollback BEFORE emitting `session/created`: a generator - // effect collects each yielded disposer before the next step runs, so a - // throwing `session/created` listener detaches onAppend and removes the - // store entry instead of leaking them (a leak would wedge the - // already-exists check until restart). The duplicate throw above fires - // before any mutation — it leaks nothing. - yield () => { - session.onAppend = undefined - this.store.delete(sessionId) - } - this.ctx.emit('session/created', session) - }.bind(this), 'sessions.create()') - return session + return new Session(sessionId, options?.seed, header) } - get(id: string): Session | undefined { + /** + * Enter a {@link prepare}d session into the store: wire `onAppend` → + * `session/event` and add it to the store. Returns the DETACH disposer + * (`onAppend = undefined` + store removal). Does NOT emit `session/created` — + * the caller yields this disposer inside its effect and THEN calls + * {@link announce}, so a throwing `session/created` listener rolls the attach + * back instead of leaking it. + * + * Re-checks the id for a duplicate: `prepare` and `enter` are public + * cross-package primitives and a caller may interleave arbitrary work (or + * another create) between them, so a stale prepared session must NOT overwrite + * a live store entry of the same id — its detach disposer would later delete + * the REAL session. The {@link create} convenience and the agent factory call + * the two back-to-back so they never trip this, but the public seam cannot + * assume that. + * + * @throws if a session with this id is already in the store. + */ + enter(session: Session): () => void { + if (this.store.has(session.id)) throw new Error(`session "${session.id}" already exists`) + session.onAppend = (event) => { this.ctx.emit('session/event', session, event) } + this.store.set(session.id, session) + return () => { + session.onAppend = undefined + this.store.delete(session.id) + } + } + + /** Emit `session/created` for an {@link enter}ed session. Separate from + * {@link enter} so the caller can yield the detach disposer first (rollback + * safety — see {@link enter}). */ + announce(session: Session): void { + this.ctx.emit('session/created', session) + } + + get(id: SessionId): Session | undefined { return this.store.get(id) } diff --git a/packages/session/src/json.ts b/packages/core/session/src/json.ts similarity index 100% rename from packages/session/src/json.ts rename to packages/core/session/src/json.ts diff --git a/packages/session/src/repair.ts b/packages/core/session/src/repair.ts similarity index 100% rename from packages/session/src/repair.ts rename to packages/core/session/src/repair.ts diff --git a/packages/session/src/types.ts b/packages/core/session/src/types.ts similarity index 78% rename from packages/session/src/types.ts rename to packages/core/session/src/types.ts index 53fca39795..2302b5b94d 100644 --- a/packages/session/src/types.ts +++ b/packages/core/session/src/types.ts @@ -1,4 +1,5 @@ -import type { Branded, CallId, ContentBlock, MessageSource, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm' +import type { Branded } from '@deepseek-ai/dsh-brand' +import type { CallId, ContentBlock, MessageSource, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm' /** Identifies one session in the store (and its persistence artifacts). */ export type SessionId = Branded<'SessionId'> @@ -8,6 +9,23 @@ export function SessionId(id: string): SessionId { return id as SessionId } +/** + * The on-disk session format version, stamped into every newly-written + * {@link SessionHeader} and enforced by every persistence backend on load. The + * single source of truth for the version — write sites and the load-time check + * all read it. + * + * It is **`0`** deliberately: while the harness is unreleased the on-disk format + * is **unstable / pre-release, with no compatibility implied**. Breaking changes + * to the persisted {@link SessionEventMap} shape (folding fields onto an event, + * removing a variant, …) happen freely and do NOT bump this — v0 absorbs all + * pre-release churn, and a backend simply REJECTS any log not at v0 (there is no + * migration; no persisted user data exists to preserve). A real, monotonically + * bumped version policy begins at the first tagged release, when a specific + * format boundary becomes worth distinguishing. + */ +export const SESSION_FORMAT_VERSION = 0 + /** * Immutable session metadata — written once at creation and never rewritten. * @@ -18,7 +36,11 @@ export function SessionId(id: string): SessionId { * metadata) writes such a header. */ export interface SessionHeader { - /** On-disk format version; a persistence backend rejects unknown versions. */ + /** + * On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the + * session is created. A persistence backend rejects any other version on load + * (no migration — see the constant). + */ version: number /** The session's id (mirrors the {@link Session}'s id). */ id: SessionId @@ -30,29 +52,6 @@ export interface SessionHeader { parentSession?: SessionId } -/** - * Mutable session metadata — updateable without touching the append-only log. - * A persistence backend stores this beside the log (a sidecar file, a header - * row) and rewrites only it on update. - */ -export interface SessionSummary { - /** Unix epoch milliseconds of the last mutation (event append or update). */ - updatedAt: number - /** Human-facing title (derived/edited), if any. */ - title?: string - /** The first user prompt, cached for listing previews. */ - firstPrompt?: string -} - -/** - * Full session metadata: the immutable {@link SessionHeader} merged with the - * mutable {@link SessionSummary}. Owned here in `dsh-session` (beside - * {@link SessionId}) because `Session.header` is typed by it; the persistence - * package imports/re-exports these rather than owning them, which would force - * a package cycle. - */ -export type SessionMeta = SessionHeader & SessionSummary - /** * Options for creating a {@link Session} via the store. `seed` replays/forks * an existing event log; `meta` carries the caller-supplied storage fields the @@ -110,7 +109,13 @@ export type TurnTrigger = TurnTriggerMap[keyof TurnTriggerMap] export interface TurnEndReasonMap { completed: { kind: 'completed' } aborted: { kind: 'aborted'; reason?: string } - error: { kind: 'error'; message: string; code?: string } + /** + * The turn failed: a step threw or the model reported a failure. `step` is the + * step number the failure occurred on (the operational error's location — the + * single durable record of an in-turn failure; live diagnostics also fire via + * `agent/error`). `code` is the error's code when one was attached. + */ + error: { kind: 'error'; step: number; message: string; code?: string } disposed: { kind: 'disposed' } 'max-tokens': { kind: 'max-tokens' } /** @@ -162,14 +167,17 @@ export interface SessionEventMap { 'context/message': { content: ContentBlock[]; source: MessageSource } /** Raw stream chunk — token-level replay fidelity. */ 'assistant/chunk': { turn: number; step: number; chunk: StreamChunk } - /** Assembled assistant message for one step (derived history uses this). */ - 'assistant/message': { turn: number; step: number; content: ContentBlock[] } + /** + * Assembled assistant message for one step (derived history uses this). + * Carries the step's `usage` when the adapter reported token accounting, so + * the model output and its accounting travel together (there is no separate + * usage record). `usage` is absent when the adapter reported none. + */ + 'assistant/message': { turn: number; step: number; content: ContentBlock[]; usage?: TokenUsage } 'tool/call': { turn: number; step: number; callId: CallId; name: string; arguments: string } 'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string } } /** Steering content injected between steps of a running turn. */ 'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource } - 'usage': { turn: number; step: number; usage: TokenUsage } - 'error': { turn: number; step: number; message: string; code?: string } } export type SessionEventType = keyof SessionEventMap diff --git a/packages/session/tests/properties.spec.ts b/packages/core/session/tests/properties.spec.ts similarity index 96% rename from packages/session/tests/properties.spec.ts rename to packages/core/session/tests/properties.spec.ts index 4d0b1b1e71..42149515f2 100644 --- a/packages/session/tests/properties.spec.ts +++ b/packages/core/session/tests/properties.spec.ts @@ -24,6 +24,7 @@ const textContentArb = fc.array( const messageEventArb: fc.Arbitrary = fc.oneof( textContentArb.map((content): Appendable => ({ type: 'user/message', data: { content, source: { kind: 'user' } } })), textContentArb.map((content): Appendable => ({ type: 'assistant/message', data: { turn: 1, step: 1, content } })), + textContentArb.map((content): Appendable => ({ type: 'assistant/message', data: { turn: 1, step: 1, content, usage: { inputTokens: 1, outputTokens: 1 } } })), fc.record({ id: fc.string({ minLength: 1 }), content: textContentArb, isError: fc.boolean() }) .map((r): Appendable => ({ type: 'tool/result', data: { turn: 1, step: 1, callId: CallId(r.id), content: r.content, isError: r.isError } })), ) @@ -35,8 +36,6 @@ const nonMessageEventArb: fc.Arbitrary = fc.oneof( fc.constant({ type: 'step/start', data: { turn: 1, step: 1 } }), fc.constant({ type: 'step/end', data: { turn: 1, step: 1 } }), fc.string().map((text): Appendable => ({ type: 'assistant/chunk', data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text } } })), - fc.constant({ type: 'usage', data: { turn: 1, step: 1, usage: { inputTokens: 1, outputTokens: 1 } } }), - fc.constant({ type: 'error', data: { turn: 1, step: 1, message: 'x' } }), ) const anyEventArb = fc.oneof(messageEventArb, nonMessageEventArb) diff --git a/packages/session/tests/repair.spec.ts b/packages/core/session/tests/repair.spec.ts similarity index 100% rename from packages/session/tests/repair.spec.ts rename to packages/core/session/tests/repair.spec.ts diff --git a/packages/session/tests/session.spec.ts b/packages/core/session/tests/session.spec.ts similarity index 81% rename from packages/session/tests/session.spec.ts rename to packages/core/session/tests/session.spec.ts index ac40a8ea3f..f095bdfb40 100644 --- a/packages/session/tests/session.spec.ts +++ b/packages/core/session/tests/session.spec.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import { CallId } from '@deepseek-ai/dsh-llm' -import SessionStore, { Session, SessionEvent, SessionId } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, Session, SessionEvent, SessionId } from '@deepseek-ai/dsh-session' describe('Session', () => { it('derives message history from the event log', () => { @@ -213,19 +213,53 @@ describe('SessionStore', () => { it('rejects duplicate ids and supports seeding', async () => { const ctx = new Context() await ctx.plugin(SessionStore) - const a = ctx.sessions.create('fixed') - expect(() => ctx.sessions.create('fixed')).toThrow('already exists') + const a = ctx.sessions.create(SessionId('fixed')) + expect(() => ctx.sessions.create(SessionId('fixed'))).toThrow('already exists') a.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }) - const forked = ctx.sessions.create('fork', { seed: [...a.events] }) + const forked = ctx.sessions.create(SessionId('fork'), { seed: [...a.events] }) expect(forked.deriveMessages()).toEqual(a.deriveMessages()) }) - it('synthesizes a minimal v1 header for a bare-created session', async () => { + it('enter() rejects a stale prepared session whose id is already live (no overwrite)', async () => { + // prepare()/enter() are public cross-package primitives that a caller may + // separate with arbitrary work. A stale prepared session must NOT overwrite + // a live store entry of the same id — its detach disposer would later delete + // the REAL session, breaking the store-uniqueness invariant. const ctx = new Context() await ctx.plugin(SessionStore) - const session = ctx.sessions.create('plain') - expect(session.header).toMatchObject({ version: 1, id: 'plain' }) + const stale = ctx.sessions.prepare(SessionId('racy')) + const live = ctx.sessions.create(SessionId('racy')) + expect(() => ctx.sessions.enter(stale)).toThrow(/already exists/) + // The live session is intact and still the store entry. + expect(ctx.sessions.get(SessionId('racy'))).toBe(live) + }) + + it('prepare() + enter() + announce() register a session and emit session/created', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const created: Session[] = [] + ctx.on('session/created', session => void created.push(session)) + + const session = ctx.sessions.prepare(SessionId('lifecycle')) + // prepare alone does NOT enter the store. + expect(ctx.sessions.get(SessionId('lifecycle'))).toBeUndefined() + const detach = ctx.sessions.enter(session) + expect(ctx.sessions.get(SessionId('lifecycle'))).toBe(session) + // enter does NOT announce. + expect(created).toEqual([]) + ctx.sessions.announce(session) + expect(created).toEqual([session]) + // The detach disposer removes the entry + stops notification. + detach() + expect(ctx.sessions.get(SessionId('lifecycle'))).toBeUndefined() + }) + + it('synthesizes a minimal current-version header for a bare-created session', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const session = ctx.sessions.create(SessionId('plain')) + expect(session.header).toMatchObject({ version: SESSION_FORMAT_VERSION, id: 'plain' }) expect(typeof session.header.createdAt).toBe('number') expect(session.header.cwd).toBeUndefined() expect(session.header.parentSession).toBeUndefined() @@ -234,11 +268,11 @@ describe('SessionStore', () => { it('attaches cwd and parentSession from meta to the header', async () => { const ctx = new Context() await ctx.plugin(SessionStore) - const session = ctx.sessions.create('child', { + const session = ctx.sessions.create(SessionId('child'), { meta: { cwd: '/work/project', parentSession: SessionId('parent') }, }) expect(session.header).toMatchObject({ - version: 1, + version: SESSION_FORMAT_VERSION, id: 'child', cwd: '/work/project', parentSession: 'parent', @@ -248,15 +282,15 @@ describe('SessionStore', () => { it('rejects a non-absolute meta.cwd', async () => { const ctx = new Context() await ctx.plugin(SessionStore) - expect(() => ctx.sessions.create('rel', { meta: { cwd: 'relative/path' } })) + expect(() => ctx.sessions.create(SessionId('rel'), { meta: { cwd: 'relative/path' } })) .toThrow(/cwd must be an absolute path/) // the rejected session was not registered - expect(ctx.sessions.get('rel')).toBeUndefined() + expect(ctx.sessions.get(SessionId('rel'))).toBeUndefined() }) - it('a bare Session() constructed without the store still exposes a v1 header', () => { + it('a bare Session() constructed without the store still exposes a current-version header', () => { const session = new Session(SessionId('bare')) - expect(session.header).toMatchObject({ version: 1, id: 'bare' }) + expect(session.header).toMatchObject({ version: SESSION_FORMAT_VERSION, id: 'bare' }) expect(typeof session.header.createdAt).toBe('number') }) @@ -266,15 +300,15 @@ describe('SessionStore', () => { let session!: Session const fiber = await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('scoped') + session = inner.sessions.create(SessionId('scoped')) }, { inject: ['sessions'] })) - expect(ctx.sessions.get('scoped')).toBe(session) + expect(ctx.sessions.get(SessionId('scoped'))).toBe(session) let observed = 0 ctx.on('session/event', () => void observed++) await fiber.dispose() - expect(ctx.sessions.get('scoped')).toBeUndefined() + expect(ctx.sessions.get(SessionId('scoped'))).toBeUndefined() session.append('user/message', { content: [{ type: 'text', text: 'late' }], source: { kind: 'user' } }) expect(observed).toBe(0) }) @@ -289,15 +323,15 @@ describe('SessionStore', () => { }) // The throwing emit must roll the store entry back, not leak it. - expect(() => ctx.sessions.create('fixed')).toThrow('boom created listener') - expect(ctx.sessions.get('fixed')).toBeUndefined() // rolled back, not leaked + expect(() => ctx.sessions.create(SessionId('fixed'))).toThrow('boom created listener') + expect(ctx.sessions.get(SessionId('fixed'))).toBeUndefined() // rolled back, not leaked // A subsequent create of the SAME id succeeds (the already-exists check is // not wedged) and its onAppend is correctly wired (events observable). const events: SessionEvent[] = [] ctx.on('session/event', (_session, event) => void events.push(event)) - const session = ctx.sessions.create('fixed') - expect(ctx.sessions.get('fixed')).toBe(session) + const session = ctx.sessions.create(SessionId('fixed')) + expect(ctx.sessions.get(SessionId('fixed'))).toBe(session) session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) expect(events).toHaveLength(1) }) diff --git a/packages/core/session/tsconfig.json b/packages/core/session/tsconfig.json new file mode 100644 index 0000000000..7ca1556695 --- /dev/null +++ b/packages/core/session/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../util/brand" + }, + { + "path": "../../llm/llm" + } + ] +} diff --git a/packages/system-prompt/README.md b/packages/core/system-prompt/README.md similarity index 100% rename from packages/system-prompt/README.md rename to packages/core/system-prompt/README.md diff --git a/packages/system-prompt/package.json b/packages/core/system-prompt/package.json similarity index 79% rename from packages/system-prompt/package.json rename to packages/core/system-prompt/package.json index eed76b8907..672f7a03ef 100644 --- a/packages/system-prompt/package.json +++ b/packages/core/system-prompt/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/system-prompt/src/index.ts b/packages/core/system-prompt/src/index.ts similarity index 93% rename from packages/system-prompt/src/index.ts rename to packages/core/system-prompt/src/index.ts index 25d87d2194..a5e6e4ed78 100644 --- a/packages/system-prompt/src/index.ts +++ b/packages/core/system-prompt/src/index.ts @@ -15,9 +15,18 @@ declare module 'cordis' { } interface Events { - /** Waterfall around prompt assembly — mutate/extend the assembly. */ + /** + * Waterfall around prompt assembly — mutate or extend the + * {@link PromptAssembly} (sections + tool schemas) before it is rendered. + * Bound to the {@link SystemPrompt} service; call `next()` to delegate. + * @mode waterfall + */ 'system-prompt/assemble'(this: SystemPrompt, assembly: PromptAssembly, next: () => Promise): Promise - /** A section or tool provider was registered or unregistered. */ + /** + * A section or tool provider was registered or unregistered (the assembly + * inputs changed). + * @mode emit + */ 'system-prompt/change'(): void } } diff --git a/packages/system-prompt/tests/system-prompt.spec.ts b/packages/core/system-prompt/tests/system-prompt.spec.ts similarity index 100% rename from packages/system-prompt/tests/system-prompt.spec.ts rename to packages/core/system-prompt/tests/system-prompt.spec.ts diff --git a/packages/core/system-prompt/tsconfig.json b/packages/core/system-prompt/tsconfig.json new file mode 100644 index 0000000000..9f687793d7 --- /dev/null +++ b/packages/core/system-prompt/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../llm/llm" + } + ] +} diff --git a/packages/tools/README.md b/packages/core/tools/README.md similarity index 100% rename from packages/tools/README.md rename to packages/core/tools/README.md diff --git a/packages/tools/package.json b/packages/core/tools/package.json similarity index 83% rename from packages/tools/package.json rename to packages/core/tools/package.json index a92015e55c..a6d3bbe0ca 100644 --- a/packages/tools/package.json +++ b/packages/core/tools/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/tools/src/index.ts b/packages/core/tools/src/index.ts similarity index 98% rename from packages/tools/src/index.ts rename to packages/core/tools/src/index.ts index eee77445eb..5a17aa2b0c 100644 --- a/packages/tools/src/index.ts +++ b/packages/core/tools/src/index.ts @@ -36,11 +36,15 @@ declare module 'cordis' { * Waterfall around every tool execution — the single seam where sandbox, * permission, hook, and plan-mode plugins wrap or veto a call. Listeners * receive `(exec, next)`: call `next()` to proceed (possibly around your - * own logic), or return a ToolExecutionResult without calling `next()` - * to short-circuit (veto). + * own logic), or return a {@link ToolExecutionResult} without calling + * `next()` to short-circuit (veto). + * @mode waterfall */ 'tools/execute'(this: ToolRegistry, exec: ToolExecution, next: () => Promise): Promise - /** A tool was registered or unregistered. */ + /** + * A tool was registered or unregistered (the available tool set changed). + * @mode emit + */ 'tools/change'(): void } } diff --git a/packages/tools/src/schema.ts b/packages/core/tools/src/schema.ts similarity index 97% rename from packages/tools/src/schema.ts rename to packages/core/tools/src/schema.ts index 5e8887f11b..b717eabf9a 100644 --- a/packages/tools/src/schema.ts +++ b/packages/core/tools/src/schema.ts @@ -39,7 +39,14 @@ export interface SchemaProp { description?: string /** Enum of allowed values (strings only). */ enum?: string[] - /** Default value. */ + /** + * Default value, emitted into the JSON Schema only (validation never applies + * it — see the validator note below). + * + * XXX(unused-default): no tool definition in the repo sets `default`; it rides + * into the wire schema for a model that no tool surfaces it to. Drop the field + * and its converter line unless a real tool needs a model-visible default. + */ default?: unknown /** Nested properties for type: 'object'. */ properties?: SchemaSpec diff --git a/packages/tools/tests/properties.spec.ts b/packages/core/tools/tests/properties.spec.ts similarity index 100% rename from packages/tools/tests/properties.spec.ts rename to packages/core/tools/tests/properties.spec.ts diff --git a/packages/tools/tests/tools.spec.ts b/packages/core/tools/tests/tools.spec.ts similarity index 100% rename from packages/tools/tests/tools.spec.ts rename to packages/core/tools/tests/tools.spec.ts diff --git a/packages/core/tools/tsconfig.json b/packages/core/tools/tsconfig.json new file mode 100644 index 0000000000..dedc111d87 --- /dev/null +++ b/packages/core/tools/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/system-prompt" + }, + { + "path": "../../core/agent" + } + ] +} diff --git a/packages/invariants/tsconfig.json b/packages/invariants/tsconfig.json deleted file mode 100644 index 54fbb4adac..0000000000 --- a/packages/invariants/tsconfig.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../llm" }, - { "path": "../session" }, - { "path": "../agent" } - ] -} diff --git a/packages/llm-deepseek/tsconfig.json b/packages/llm-deepseek/tsconfig.json deleted file mode 100644 index eea89a4aac..0000000000 --- a/packages/llm-deepseek/tsconfig.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../llm" } - ] -} diff --git a/packages/llm-pi-ai/tsconfig.json b/packages/llm-pi-ai/tsconfig.json deleted file mode 100644 index eea89a4aac..0000000000 --- a/packages/llm-pi-ai/tsconfig.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../llm" } - ] -} diff --git a/packages/llm/README.md b/packages/llm/README.md index 3c16c3153c..3fc5c9cf9e 100644 --- a/packages/llm/README.md +++ b/packages/llm/README.md @@ -1,46 +1,11 @@ -# dsh-llm +# llm/ — LLM capability family -Provider-neutral LLM vocabulary and abstract service. This package defines the canonical language spoken by the agent loop, session logs, and every plugin. +The LLM seam and its provider adapters. The interface package (`llm`) owns the abstract service, the content-block vocabulary, and the stream-chunk assembler; the adapters are concrete implementations that register on `ctx.llm`. All **product** packages. -## Service: `LlmService` (ctx key: `llm`) - -An adapter registry plus streaming / non-streaming call surfaces. Both call surfaces are interceptable via waterfall events. - -### Public API - -- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` Register an adapter for the given model names. Disposed with the calling fiber. -- `ctx.llm.models(): string[]` — model names with a registered adapter. -- `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). -- `ctx.llm.streamBlocks(options: GenerateOptions): AsyncIterable` Stream as completed content blocks (convenience view). -- `ctx.llm.generate(options: GenerateOptions): Promise` One model call, fully assembled. - -### Events - -| Event | Mode | Purpose | +| Package | Role | ctx key | |---|---|---| -| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) | -| `llm/generate` | waterfall | Intercept/wrap every non-streaming model call | -| `llm/adapter-change` | emit | An adapter was registered or unregistered | +| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` | +| `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | +| `llm-pi-ai/` | DeepSeek adapter via `@earendil-works/pi-ai` (design twin) | (registers on `ctx.llm`) | -### Extension points - -- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` to add a new model provider. -- Wrap `llm/stream` or `llm/generate` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc. - -### Content-block vocabulary (`types.ts`) - -Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`, `image`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. - -Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. - -### Classes - -- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`. -- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. Used by the agent loop (raw chunks for replay - + assembled for history) and by `streamBlocks()`/`generate()`. -- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams. -- `LlmError` — extends `HarnessError`; `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) plus an optional numeric `status` when the failure came from a non-2xx provider response. - -### Real adapters - -Two adapters implement `LlmAdapter` against this vocabulary, deliberately built on different internals to keep the contract honest (see [the twin LLM adapters](../../docs/rfc/implemented/2026-06-13-twin-llm-adapters.md)): [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) (hand-rolled fetch/SSE) and [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) (via `@earendil-works/pi-ai`). The pair pinned down the `StreamChunk` conventions now documented in `types.ts` (usage before finish, raw-string tool arguments, the two sanctioned error paths). +The interface lives at `llm/llm/`; adapters are flat siblings under the group. A new provider adapter joins here and registers on `ctx.llm` without touching the interface. See [twin LLM adapters](../../docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md) for why two adapters exist. diff --git a/packages/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md similarity index 100% rename from packages/llm-deepseek/README.md rename to packages/llm/llm-deepseek/README.md diff --git a/packages/llm-deepseek/package.json b/packages/llm/llm-deepseek/package.json similarity index 81% rename from packages/llm-deepseek/package.json rename to packages/llm/llm-deepseek/package.json index 077a2db169..8ebf71c5b5 100644 --- a/packages/llm-deepseek/package.json +++ b/packages/llm/llm-deepseek/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts similarity index 100% rename from packages/llm-deepseek/src/adapter.ts rename to packages/llm/llm-deepseek/src/adapter.ts diff --git a/packages/llm-deepseek/src/index.ts b/packages/llm/llm-deepseek/src/index.ts similarity index 100% rename from packages/llm-deepseek/src/index.ts rename to packages/llm/llm-deepseek/src/index.ts diff --git a/packages/llm-deepseek/src/serialize.ts b/packages/llm/llm-deepseek/src/serialize.ts similarity index 100% rename from packages/llm-deepseek/src/serialize.ts rename to packages/llm/llm-deepseek/src/serialize.ts diff --git a/packages/llm-deepseek/src/sse.ts b/packages/llm/llm-deepseek/src/sse.ts similarity index 100% rename from packages/llm-deepseek/src/sse.ts rename to packages/llm/llm-deepseek/src/sse.ts diff --git a/packages/llm-deepseek/src/translate.ts b/packages/llm/llm-deepseek/src/translate.ts similarity index 100% rename from packages/llm-deepseek/src/translate.ts rename to packages/llm/llm-deepseek/src/translate.ts diff --git a/packages/llm-deepseek/src/types.ts b/packages/llm/llm-deepseek/src/types.ts similarity index 100% rename from packages/llm-deepseek/src/types.ts rename to packages/llm/llm-deepseek/src/types.ts diff --git a/packages/llm-deepseek/tests/adapter.e2e.ts b/packages/llm/llm-deepseek/tests/adapter.e2e.ts similarity index 93% rename from packages/llm-deepseek/tests/adapter.e2e.ts rename to packages/llm/llm-deepseek/tests/adapter.e2e.ts index ebec6e62ce..b01b498dff 100644 --- a/packages/llm-deepseek/tests/adapter.e2e.ts +++ b/packages/llm/llm-deepseek/tests/adapter.e2e.ts @@ -1,9 +1,10 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from 'cordis' import LlmService, { CallId } from '@deepseek-ai/dsh-llm' -import type { GenerateResult, Message, ToolSchema } from '@deepseek-ai/dsh-llm' +import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import type { Config } from '@deepseek-ai/dsh-llm-deepseek' +import { assemble, type AssembledResult } from './assemble.ts' /** * Real-API e2e for the hand-rolled adapter: V4 Flash + V4 Pro across @@ -31,7 +32,7 @@ function ask(text: string): Message[] { return [{ role: 'user', content: [{ type: 'text', text }] }] } -function textOf(result: GenerateResult): string { +function textOf(result: AssembledResult): string { return result.message.content .filter(block => block.type === 'text') .map(block => block.text) @@ -51,7 +52,7 @@ const weatherTool: ToolSchema = { describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () => { it('flash + thinking disabled: plain text generation', async () => { const ctx = await harness(FLASH, { thinking: 'disabled' }) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model: FLASH, messages: ask('Reply with exactly the word: pong'), maxTokens: 50, @@ -65,7 +66,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () it('flash + thinking enabled (effort high): reasoning blocks + reasoning tokens', async () => { const ctx = await harness(FLASH, { thinking: 'enabled', reasoningEffort: 'high' }) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model: FLASH, messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'), maxTokens: 2000, @@ -82,7 +83,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () const ctx = await harness(PRO, { thinking: 'enabled', reasoningEffort: effort }) // Turn 1: the model must call the tool (and think before it). - const first = await ctx.llm.generate({ + const first = await assemble(ctx,{ model: PRO, messages: ask('What is the weather in Paris right now? Use the get_weather tool.'), tools: [weatherTool], @@ -96,7 +97,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () // Turn 2: send the tool result back WITH the assistant's reasoning // block in history (the official thinking+tools passback rule). - const second = await ctx.llm.generate({ + const second = await assemble(ctx,{ model: PRO, messages: [ ...ask('What is the weather in Paris right now? Use the get_weather tool.'), @@ -120,7 +121,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () it('pro + thinking disabled: plain generation without reasoning blocks', async () => { const ctx = await harness(PRO, { thinking: 'disabled' }) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model: PRO, messages: ask('Reply with exactly the word: pong'), maxTokens: 50, diff --git a/packages/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts similarity index 93% rename from packages/llm-deepseek/tests/adapter.spec.ts rename to packages/llm/llm-deepseek/tests/adapter.spec.ts index 553affecb7..1abbebc060 100644 --- a/packages/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -5,6 +5,7 @@ import { Context } from 'cordis' import LlmService, { LlmError } from '@deepseek-ai/dsh-llm' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { DeepSeekAdapter, httpErrorCode } from '@deepseek-ai/dsh-llm-deepseek' +import { assemble } from './assemble.ts' /** One scripted behavior for the next request the mock server receives. */ type Behavior = @@ -90,11 +91,11 @@ async function harness(baseURL: string, config: object = {}) { } describe('DeepSeekAdapter against a mock server', () => { - it('streams a text generation end to end through ctx.llm.generate', async () => { + it('streams a text generation end to end through the assembler', async () => { const server = await mockServer([{ kind: 'sse', events: textEvents }]) const ctx = await harness(server.url) - const result = await ctx.llm.generate({ + const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], }) @@ -130,7 +131,7 @@ describe('DeepSeekAdapter against a mock server', () => { const server = await mockServer([{ kind: 'sse', events: textEvents }]) const ctx = await harness(server.url, { thinking: 'disabled', reasoningEffort: 'high' }) - await ctx.llm.generate({ + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], }) @@ -155,15 +156,15 @@ describe('DeepSeekAdapter against a mock server', () => { } const server = await mockServer([behavior, behavior, behavior]) const ctx = await harness(server.url) - await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })) + await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })) .rejects.toThrow(`failed with ${status}`) await expect( - ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) .catch((error: unknown) => (error as LlmError).code), ).resolves.toBe(code) // The numeric HTTP status is carried on the error for explicit handling. await expect( - ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) .catch((error: unknown) => (error as LlmError).status), ).resolves.toBe(status) }) @@ -171,14 +172,14 @@ describe('DeepSeekAdapter against a mock server', () => { it('keeps the status-line message for JSON error bodies without a message', async () => { const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }]) const ctx = await harness(server.url) - await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })) + await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })) .rejects.toThrow(/HTTP 500/) }) it('keeps the status-line message for non-JSON error bodies', async () => { const server = await mockServer([{ kind: 'http-error', status: 502, body: 'Bad Gateway', contentType: 'text/plain' }]) const ctx = await harness(server.url) - await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })) + await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })) .rejects.toThrow(/HTTP 502/) }) @@ -207,7 +208,7 @@ describe('DeepSeekAdapter against a mock server', () => { events: ['{"choices":[{"delta":{"content":"par"}}]}'], }]) const ctx = await harness(server.url) - await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })) + await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })) .rejects.toThrow(/terminated|socket|without \[DONE\]/) }) @@ -278,7 +279,7 @@ describe('plugin registration and config', () => { vi.stubEnv('DEEPSEEK_BASE_URL', 'http://env-host:1') const server = await mockServer([{ kind: 'sse', events: textEvents }]) const ctx = await harness(server.url) // harness passes explicit config - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(server.requests).toHaveLength(1) // hit the explicit URL, not env }) @@ -288,7 +289,7 @@ describe('plugin registration and config', () => { const ctx = new Context() await ctx.plugin(LlmService) await ctx.plugin(LlmDeepSeek, { apiKey: 'k', models: ['deepseek-v4-flash'] }) - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(server.requests).toHaveLength(1) }) diff --git a/packages/llm/llm-deepseek/tests/assemble.ts b/packages/llm/llm-deepseek/tests/assemble.ts new file mode 100644 index 0000000000..b0182615e0 --- /dev/null +++ b/packages/llm/llm-deepseek/tests/assemble.ts @@ -0,0 +1,26 @@ +/** + * Test helper: drive `ctx.llm.stream()` through a `BlockAssembler` and return + * the assembled message + usage + finish reason. This exercises the same + * streaming path production uses (the loop), rather than a service-level + * one-shot convenience method. + */ + +import { BlockAssembler } from '@deepseek-ai/dsh-llm' +import type { Context } from 'cordis' +import type { FinishReason, GenerateOptions, Message, TokenUsage } from '@deepseek-ai/dsh-llm' + +export interface AssembledResult { + message: Message + usage?: TokenUsage + finish: FinishReason +} + +export async function assemble(ctx: Context, options: GenerateOptions): Promise { + const assembler = new BlockAssembler() + for await (const chunk of ctx.llm.stream(options)) assembler.push(chunk) + return { + message: assembler.message(), + ...assembler.usage !== undefined ? { usage: assembler.usage } : {}, + finish: assembler.finish, + } +} diff --git a/packages/llm-deepseek/tests/serialize.spec.ts b/packages/llm/llm-deepseek/tests/serialize.spec.ts similarity index 100% rename from packages/llm-deepseek/tests/serialize.spec.ts rename to packages/llm/llm-deepseek/tests/serialize.spec.ts diff --git a/packages/llm-deepseek/tests/sse.spec.ts b/packages/llm/llm-deepseek/tests/sse.spec.ts similarity index 100% rename from packages/llm-deepseek/tests/sse.spec.ts rename to packages/llm/llm-deepseek/tests/sse.spec.ts diff --git a/packages/llm-deepseek/tests/translate.spec.ts b/packages/llm/llm-deepseek/tests/translate.spec.ts similarity index 99% rename from packages/llm-deepseek/tests/translate.spec.ts rename to packages/llm/llm-deepseek/tests/translate.spec.ts index 40b5ee5944..d6968faed5 100644 --- a/packages/llm-deepseek/tests/translate.spec.ts +++ b/packages/llm/llm-deepseek/tests/translate.spec.ts @@ -47,7 +47,7 @@ describe('translate: text', () => { ))) { assembler.push(chunk) } - const result = assembler.result() + const result = { message: assembler.message(), finish: assembler.finish } expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }]) expect(result.finish).toEqual({ kind: 'stop' }) }) diff --git a/packages/llm/llm-deepseek/tsconfig.json b/packages/llm/llm-deepseek/tsconfig.json new file mode 100644 index 0000000000..e9de391ba1 --- /dev/null +++ b/packages/llm/llm-deepseek/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../llm/llm" + } + ] +} diff --git a/packages/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md similarity index 100% rename from packages/llm-pi-ai/README.md rename to packages/llm/llm-pi-ai/README.md diff --git a/packages/llm-pi-ai/package.json b/packages/llm/llm-pi-ai/package.json similarity index 83% rename from packages/llm-pi-ai/package.json rename to packages/llm/llm-pi-ai/package.json index 6eb08a4b06..30911915ff 100644 --- a/packages/llm-pi-ai/package.json +++ b/packages/llm/llm-pi-ai/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts similarity index 97% rename from packages/llm-pi-ai/src/adapter.ts rename to packages/llm/llm-pi-ai/src/adapter.ts index 38b05dc007..e15cce8252 100644 --- a/packages/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -14,6 +14,7 @@ import { stream as piStream } from '@earendil-works/pi-ai' import type { Model } from '@earendil-works/pi-ai' import { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import { CallId } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, StreamChunk, ToolSchema } from '@deepseek-ai/dsh-llm' import { toPiContext, toStreamChunks } from './convert.ts' @@ -69,8 +70,8 @@ type Payload = { stop?: unknown } -function rawToolArguments(options: GenerateOptions): Map { - const raw = new Map() +function rawToolArguments(options: GenerateOptions): Map { + const raw = new Map() for (const message of options.messages) { if (message.role !== 'assistant') continue for (const block of message.content) { @@ -116,7 +117,7 @@ function patchPayload(payload: unknown, options: GenerateOptions, reasoning: PiA for (const call of message.tool_calls ?? []) { /* v8 ignore next -- malformed pi-ai payload guard: real tool calls always carry a string id */ if (typeof call.id !== 'string') continue - const raw = rawById.get(call.id) + const raw = rawById.get(CallId(call.id)) /* v8 ignore next -- pi-ai always emits a function object for assistant tool_calls; guard malformed payloads defensively */ if (raw !== undefined && call.function !== undefined) call.function.arguments = raw } diff --git a/packages/llm-pi-ai/src/convert.ts b/packages/llm/llm-pi-ai/src/convert.ts similarity index 99% rename from packages/llm-pi-ai/src/convert.ts rename to packages/llm/llm-pi-ai/src/convert.ts index 4610ff01c9..0ddc41386a 100644 --- a/packages/llm-pi-ai/src/convert.ts +++ b/packages/llm/llm-pi-ai/src/convert.ts @@ -57,7 +57,7 @@ function parseArguments(raw: string): Record { * same id. */ export function toPiContext(options: GenerateOptions): PiContext { - const toolNames = new Map() + const toolNames = new Map() const messages: PiMessage[] = [] for (const message of options.messages) { diff --git a/packages/llm-pi-ai/src/index.ts b/packages/llm/llm-pi-ai/src/index.ts similarity index 100% rename from packages/llm-pi-ai/src/index.ts rename to packages/llm/llm-pi-ai/src/index.ts diff --git a/packages/llm-pi-ai/tests/adapter.e2e.ts b/packages/llm/llm-pi-ai/tests/adapter.e2e.ts similarity index 89% rename from packages/llm-pi-ai/tests/adapter.e2e.ts rename to packages/llm/llm-pi-ai/tests/adapter.e2e.ts index 133539f6be..fa30226ddf 100644 --- a/packages/llm-pi-ai/tests/adapter.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.e2e.ts @@ -1,10 +1,11 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from 'cordis' import LlmService, { CallId } from '@deepseek-ai/dsh-llm' -import type { GenerateResult, Message, ToolSchema } from '@deepseek-ai/dsh-llm' +import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' import type { Config } from '@deepseek-ai/dsh-llm-pi-ai' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' +import { assemble, type AssembledResult } from './assemble.ts' /** * Real-API e2e for the pi-ai-backed adapter: V4 Flash + V4 Pro across all @@ -33,14 +34,14 @@ function ask(text: string): Message[] { return [{ role: 'user', content: [{ type: 'text', text }] }] } -function textOf(result: GenerateResult): string { +function textOf(result: AssembledResult): string { return result.message.content .filter(block => block.type === 'text') .map(block => block.text) .join('') } -function blockKinds(result: GenerateResult): string[] { +function blockKinds(result: AssembledResult): string[] { return result.message.content.map(block => block.type) } @@ -57,7 +58,7 @@ const weatherTool: ToolSchema = { describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => { it.each([FLASH, PRO])('%s + reasoning off: plain text generation', async (model) => { const ctx = await harness(model, { reasoning: 'off' }) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model, messages: ask('Reply with exactly the word: pong'), maxTokens: 50, @@ -69,7 +70,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => it.each([FLASH, PRO])('%s + reasoning high: reasoning blocks present', async (model) => { const ctx = await harness(model, { reasoning: 'high' }) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model, messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'), maxTokens: 2000, @@ -82,7 +83,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => it('pro + reasoning xhigh (wire max): tool-call round trip', async () => { const ctx = await harness(PRO, { reasoning: 'xhigh' }) - const first = await ctx.llm.generate({ + const first = await assemble(ctx,{ model: PRO, messages: ask('What is the weather in Paris right now? Use the get_weather tool.'), tools: [weatherTool], @@ -94,7 +95,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => expect(call!.name).toBe('get_weather') expect(JSON.parse(call!.arguments)).toMatchObject({ city: expect.stringMatching(/paris/i) as string }) - const second = await ctx.llm.generate({ + const second = await assemble(ctx,{ model: PRO, messages: [ ...ask('What is the weather in Paris right now? Use the get_weather tool.'), @@ -128,8 +129,8 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => const prompt = ask('Reply with exactly the word: pong') const [fromDeepSeek, fromPiAi] = await Promise.all([ - deepseekCtx.llm.generate({ model: FLASH, messages: prompt, maxTokens: 50 }), - piCtx.llm.generate({ model: FLASH, messages: prompt, maxTokens: 50 }), + assemble(deepseekCtx, { model: FLASH, messages: prompt, maxTokens: 50 }), + assemble(piCtx, { model: FLASH, messages: prompt, maxTokens: 50 }), ]) expect(blockKinds(fromPiAi)).toEqual(blockKinds(fromDeepSeek)) expect(fromPiAi.finish.kind).toBe(fromDeepSeek.finish.kind) diff --git a/packages/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts similarity index 94% rename from packages/llm-pi-ai/tests/adapter.spec.ts rename to packages/llm/llm-pi-ai/tests/adapter.spec.ts index 97b20f4617..63f9f90456 100644 --- a/packages/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -5,6 +5,7 @@ import { Context } from 'cordis' import LlmService, { CallId, LlmError } from '@deepseek-ai/dsh-llm' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' import { buildModel, PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai' +import { assemble } from './assemble.ts' /** Scripted SSE responses, one per request (OpenAI chat-completions shape). */ interface MockServer { @@ -79,11 +80,11 @@ async function harness(baseURL: string, config: object = {}) { } describe('PiAiAdapter against a mock server', () => { - it('streams a text generation through ctx.llm.generate', async () => { + it('streams a text generation through the assembler', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) - const result = await ctx.llm.generate({ + const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], }) @@ -96,7 +97,7 @@ describe('PiAiAdapter against a mock server', () => { const server = await mockServer([{ events: toolEvents }]) const ctx = await harness(server.url) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [{ role: 'user', content: [{ type: 'text', text: 'weather?' }] }], tools: [{ @@ -114,7 +115,7 @@ describe('PiAiAdapter against a mock server', () => { const server = await mockServer([{ events: thinkingEvents }]) const ctx = await harness(server.url, { reasoning: 'high' }) - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [{ role: 'user', content: [{ type: 'text', text: 'think' }] }], }) @@ -127,7 +128,7 @@ describe('PiAiAdapter against a mock server', () => { it('sends DeepSeek thinking fields when reasoning is configured', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url, { reasoning: 'xhigh' }) - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(server.requests[0]).toMatchObject({ thinking: { type: 'enabled' }, reasoning_effort: 'max', // xhigh maps to max via thinkingLevelMap @@ -137,21 +138,21 @@ describe('PiAiAdapter against a mock server', () => { it('disables thinking for reasoning: off', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url, { reasoning: 'off' }) - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(server.requests[0]).toMatchObject({ thinking: { type: 'disabled' } }) }) it('injects stop sequences through onPayload', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [], stop: ['END'] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], stop: ['END'] }) expect(server.requests[0]).toMatchObject({ stop: ['END'] }) }) it('preserves per-tool strict exactly through onPayload', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) - await ctx.llm.generate({ + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], tools: [ @@ -173,7 +174,7 @@ describe('PiAiAdapter against a mock server', () => { it('preserves raw replayed tool-call arguments in the provider payload', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) - await ctx.llm.generate({ + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [{ role: 'assistant', @@ -192,7 +193,7 @@ describe('PiAiAdapter against a mock server', () => { body: JSON.stringify({ error: { message: 'bad key' } }), }]) const ctx = await harness(server.url) - const result = await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(result.finish).toMatchObject({ kind: 'error', code: 'AUTH' }) expect((result.finish as { message: string }).message).toMatch(/bad key|401/) }) @@ -204,13 +205,13 @@ describe('PiAiAdapter against a mock server', () => { ] as const)('maps HTTP %s to stable error code %s', async (status, code) => { const server = await mockServer([{ status, body: JSON.stringify({ error: { message: `provider ${status}` } }) }]) const ctx = await harness(server.url) - const result = await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(result.finish).toMatchObject({ kind: 'error', code }) }) it('rejects prefill with UNSUPPORTED', async () => { const ctx = await harness('http://127.0.0.1:1') - await expect(ctx.llm.generate({ + await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], prefill: [{ type: 'text', text: 'Sure' }], @@ -244,7 +245,7 @@ describe('option spreads and env fallbacks', () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) const controller = new AbortController() - await ctx.llm.generate({ + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], temperature: 0.5, @@ -262,7 +263,7 @@ describe('option spreads and env fallbacks', () => { const ctx = new Context() await ctx.plugin(LlmService) await ctx.plugin(LlmPiAi, { models: ['deepseek-v4-flash'] }) - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) expect(server.requests).toHaveLength(1) } finally { vi.unstubAllEnvs() @@ -311,7 +312,7 @@ describe('review fixes', () => { it('defaults omitted reasoning config to thinking ENABLED (provider default)', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) // no reasoning key at all - await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }) + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) const request = server.requests[0] as Record expect(request.thinking).toEqual({ type: 'enabled' }) expect('reasoning_effort' in request).toBe(false) @@ -320,7 +321,7 @@ describe('review fixes', () => { it('replays reasoning_content on assistant tool-call turns (passback rule)', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) - await ctx.llm.generate({ + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [ { role: 'user', content: [{ type: 'text', text: 'weather?' }] }, @@ -376,7 +377,7 @@ describe('review fixes: abort wiring', () => { const controller = new AbortController() controller.abort('already cancelled') // pi-ai surfaces the abort as an in-stream error event → aborted finish. - const result = await ctx.llm.generate({ + const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], signal: controller.signal, @@ -388,7 +389,7 @@ describe('review fixes: abort wiring', () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) const controller = new AbortController() - const pending = ctx.llm.generate({ + const pending = assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], signal: controller.signal, diff --git a/packages/llm/llm-pi-ai/tests/assemble.ts b/packages/llm/llm-pi-ai/tests/assemble.ts new file mode 100644 index 0000000000..b0182615e0 --- /dev/null +++ b/packages/llm/llm-pi-ai/tests/assemble.ts @@ -0,0 +1,26 @@ +/** + * Test helper: drive `ctx.llm.stream()` through a `BlockAssembler` and return + * the assembled message + usage + finish reason. This exercises the same + * streaming path production uses (the loop), rather than a service-level + * one-shot convenience method. + */ + +import { BlockAssembler } from '@deepseek-ai/dsh-llm' +import type { Context } from 'cordis' +import type { FinishReason, GenerateOptions, Message, TokenUsage } from '@deepseek-ai/dsh-llm' + +export interface AssembledResult { + message: Message + usage?: TokenUsage + finish: FinishReason +} + +export async function assemble(ctx: Context, options: GenerateOptions): Promise { + const assembler = new BlockAssembler() + for await (const chunk of ctx.llm.stream(options)) assembler.push(chunk) + return { + message: assembler.message(), + ...assembler.usage !== undefined ? { usage: assembler.usage } : {}, + finish: assembler.finish, + } +} diff --git a/packages/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts similarity index 100% rename from packages/llm-pi-ai/tests/convert.spec.ts rename to packages/llm/llm-pi-ai/tests/convert.spec.ts diff --git a/packages/llm/llm-pi-ai/tsconfig.json b/packages/llm/llm-pi-ai/tsconfig.json new file mode 100644 index 0000000000..e9de391ba1 --- /dev/null +++ b/packages/llm/llm-pi-ai/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../llm/llm" + } + ] +} diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md new file mode 100644 index 0000000000..4227f5fdef --- /dev/null +++ b/packages/llm/llm/README.md @@ -0,0 +1,41 @@ +# dsh-llm + +Provider-neutral LLM vocabulary and abstract service. This package defines the canonical language spoken by the agent loop, session logs, and every plugin. + +## Service: `LlmService` (ctx key: `llm`) + +An adapter registry plus a single streaming call surface, interceptable via a waterfall event. + +### Public API + +- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` Register an adapter for the given model names. Disposed with the calling fiber. +- `ctx.llm.models(): string[]` — model names with a registered adapter. +- `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`. + +### Events + +| Event | Mode | Purpose | +|---|---|---| +| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) | + +### Extension points + +- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` to add a new model provider. +- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc. + +### Content-block vocabulary (`types.ts`) + +Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`, `image`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. + +Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. + +### Classes + +- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`. +- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history. +- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams. +- `LlmError` — extends `HarnessError`; `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) plus an optional numeric `status` when the failure came from a non-2xx provider response. + +### Real adapters + +Two adapters implement `LlmAdapter` against this vocabulary, deliberately built on different internals to keep the contract honest (see [the twin LLM adapters](../../../docs/rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md)): [`@deepseek-ai/dsh-llm-deepseek`](../llm-deepseek) (hand-rolled fetch/SSE) and [`@deepseek-ai/dsh-llm-pi-ai`](../llm-pi-ai) (via `@earendil-works/pi-ai`). The pair pinned down the `StreamChunk` conventions now documented in `types.ts` (usage before finish, raw-string tool arguments, the two sanctioned error paths). diff --git a/packages/llm/package.json b/packages/llm/llm/package.json similarity index 68% rename from packages/llm/package.json rename to packages/llm/llm/package.json index 317edc7ac2..1dc84e13d7 100644 --- a/packages/llm/package.json +++ b/packages/llm/llm/package.json @@ -5,24 +5,28 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", "peerDependencies": { + "@deepseek-ai/dsh-brand": "^0.0.1", "cordis": "^4.0.0-rc.6" }, "devDependencies": { + "@deepseek-ai/dsh-brand": "workspace:^", "cordis": "^4.0.0-rc.6" } } diff --git a/packages/llm/src/assembler.ts b/packages/llm/llm/src/assembler.ts similarity index 68% rename from packages/llm/src/assembler.ts rename to packages/llm/llm/src/assembler.ts index a61d6cf044..328ef01c54 100644 --- a/packages/llm/src/assembler.ts +++ b/packages/llm/llm/src/assembler.ts @@ -1,13 +1,14 @@ /** * Incremental chunk-to-message assembler. This is the single canonical assembly - * algorithm used by both the agent loop and the LLM service convenience views. + * algorithm used by the agent loop to build an assistant message from a chunk + * stream while logging the raw chunks for replay fidelity. * * @module @deepseek-ai/dsh-llm/assembler */ import { CallId } from './brand.ts' import { assertNever } from './never.ts' -import type { ContentBlock, FinishReason, GenerateResult, Message, StreamChunk, TokenUsage } from './types.ts' +import type { ContentBlock, FinishReason, Message, StreamChunk, TokenUsage } from './types.ts' interface PartialBlock { blockType: string @@ -23,9 +24,8 @@ interface PartialBlock { * Incrementally assembles raw {@link StreamChunk}s into complete * {@link ContentBlock}s and a final assistant {@link Message}. * - * This is the single shared assembly implementation: the agent loop feeds it - * while logging raw chunks for replay fidelity, and `LlmService.generate()` / - * `streamBlocks()` use it to offer assembled views of the same stream. + * The agent loop feeds it while logging raw chunks for replay fidelity, then + * reads `blocks()` / `message()` / `usage` / `finish` once the stream ends. * * Tolerant of delta-only protocols (no block-start/end); deltas arriving for * an index already closed by `block-end` are ignored (malformed stream) so a @@ -34,7 +34,6 @@ interface PartialBlock { export class BlockAssembler { private partials = new Map() private order: number[] = [] - private flushed = 0 private _usage: TokenUsage | undefined private _finish: FinishReason | undefined @@ -129,44 +128,6 @@ export class BlockAssembler { return this.order.map(index => this.assemble(this.mustGet(index), index)) } - /** - * Streaming flush: returns (once) every block that is complete AND has no - * incomplete block before it in stream order. Call after each `push()`; - * blocks come out strictly in stream order, so a streaming consumer sees - * exactly the sequence `blocks()` would produce. - */ - flushReady(): ContentBlock[] { - const ready: ContentBlock[] = [] - while (this.flushed < this.order.length) { - const index = this.order[this.flushed] - /* v8 ignore next 3 -- noUncheckedIndexedAccess guard: loop condition guarantees index exists in a non-empty array */ - if (index === undefined) break - const partial = this.mustGet(index) - if (!partial.block) break - ready.push(partial.block) - this.flushed += 1 - } - return ready - } - - /** - * End-of-stream flush: returns (once) all not-yet-flushed blocks, in stream - * order, assembling still-open ones from their deltas (delta-only - * protocols). After this, `flushReady()` + `flushRemaining()` together have - * yielded exactly `blocks()`. - */ - flushRemaining(): ContentBlock[] { - const remaining: ContentBlock[] = [] - while (this.flushed < this.order.length) { - const index = this.order[this.flushed] - /* v8 ignore next 3 -- noUncheckedIndexedAccess guard: loop condition guarantees index exists */ - if (index === undefined) break - remaining.push(this.assemble(this.mustGet(index), index)) - this.flushed += 1 - } - return remaining - } - get usage(): TokenUsage | undefined { return this._usage } @@ -179,13 +140,4 @@ export class BlockAssembler { message(): Message { return { role: 'assistant', content: this.blocks() } } - - /** The assembled non-streaming result. */ - result(): GenerateResult { - return { - message: this.message(), - ...this._usage !== undefined ? { usage: this._usage } : {}, - finish: this.finish, - } - } } diff --git a/packages/llm/llm/src/brand.ts b/packages/llm/llm/src/brand.ts new file mode 100644 index 0000000000..3082cc0141 --- /dev/null +++ b/packages/llm/llm/src/brand.ts @@ -0,0 +1,23 @@ +/** + * dsh-llm's owned branded id: `CallId` (tool-call correlation). + * + * The `Branded` primitive itself lives in `@deepseek-ai/dsh-brand` (a + * zero-dependency type-only package) so every owner of a cross-boundary id can + * brand it without depending on dsh-llm; see that package's README for the + * nominal-typing policy. + * + * @module @deepseek-ai/dsh-llm/brand + */ + +import type { Branded } from '@deepseek-ai/dsh-brand' + +/** + * Correlates a model-issued tool call with its result. Provider-issued for + * real adapters; synthesized by mocks/assembler fallbacks. + */ +export type CallId = Branded<'CallId'> + +/** Brand a string as a {@link CallId}. */ +export function CallId(id: string): CallId { + return id as CallId +} diff --git a/packages/llm/src/error.ts b/packages/llm/llm/src/error.ts similarity index 100% rename from packages/llm/src/error.ts rename to packages/llm/llm/src/error.ts diff --git a/packages/llm/src/index.ts b/packages/llm/llm/src/index.ts similarity index 56% rename from packages/llm/src/index.ts rename to packages/llm/llm/src/index.ts index 460316ea50..320838a8a6 100644 --- a/packages/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -1,14 +1,13 @@ /** - * LLM service: adapter registry with waterfall-interceptable streaming and - * non-streaming call surfaces. Exports the `LlmService` default, the abstract - * `LlmAdapter` for provider backends, and `BlockAssembler` for chunk assembly. + * LLM service: adapter registry with a waterfall-interceptable streaming call + * surface. Exports the `LlmService` default, the abstract `LlmAdapter` for + * provider backends, and `BlockAssembler` for chunk assembly. * * @module @deepseek-ai/dsh-llm */ import { Context, Service } from 'cordis' -import type { ContentBlock, GenerateOptions, GenerateResult, StreamChunk } from './types.ts' -import { BlockAssembler } from './assembler.ts' +import type { GenerateOptions, StreamChunk } from './types.ts' import { HarnessError } from './error.ts' export * from './brand.ts' @@ -23,12 +22,13 @@ declare module 'cordis' { } interface Events { - /** Waterfall around every streaming model call (retry, caching, routing). */ + /** + * Waterfall around every streaming model call (retry, caching, routing). + * Bound to the {@link LlmService}; call `next()` to reach the resolved + * adapter's stream, or yield your own chunks to short-circuit. + * @mode waterfall + */ 'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable - /** Waterfall around every non-streaming model call. */ - 'llm/generate'(this: LlmService, options: GenerateOptions, next: () => Promise): Promise - /** An adapter was registered or unregistered. */ - 'llm/adapter-change'(): void } } @@ -63,8 +63,8 @@ export abstract class LlmAdapter { } /** - * The abstract `llm` service: an adapter registry plus streaming / - * non-streaming call surfaces, both interceptable via waterfall events. + * The abstract `llm` service: an adapter registry plus a streaming model-call + * surface, interceptable via the `llm/stream` waterfall. */ export class LlmService extends Service { private adapters = new Map() @@ -76,8 +76,7 @@ export class LlmService extends Service { /** * Register an adapter for the given model names. Throws `LlmError` with code * `DUPLICATE_ADAPTER` if any model already has an adapter (all-or-nothing). - * Emits `llm/adapter-change` on registration and disposal. Disposed with the - * fiber. + * Disposed with the fiber. */ registerAdapter(models: string[], adapter: LlmAdapter): () => void { const dispose = this.ctx.effect(function* (this: LlmService) { @@ -87,17 +86,9 @@ export class LlmService extends Service { } } for (const model of models) this.adapters.set(model, adapter) - // Yield the rollback BEFORE emitting the change event: a generator effect - // collects each yielded disposer before running the next step, so a - // throwing `llm/adapter-change` listener rolls the mutation back instead - // of leaking the entry (which would wedge the duplicate check until - // restart). The duplicate throws above fire before any mutation, so they - // correctly leak nothing. yield () => { for (const model of models) this.adapters.delete(model) - this.ctx.emit('llm/adapter-change') } - this.ctx.emit('llm/adapter-change') }.bind(this), 'llm.registerAdapter()') // ctx.effect's disposer returns Promise; our disposer API is // synchronous fire-and-forget — discard the (always-resolved) promise. @@ -125,36 +116,6 @@ export class LlmService extends Service { return this.adapter(options.model).stream(options) }) } - - /** - * Stream one model call as completed content blocks — a convenience view - * for consumers that don't care about token-level deltas. Blocks are - * yielded strictly in stream order as soon as they (and everything before - * them) complete; blocks left open at end of stream (delta-only protocols) - * are assembled and flushed last, so the sequence always equals - * `generate()`'s `message.content`. - */ - async * streamBlocks(options: GenerateOptions): AsyncIterable { - const assembler = new BlockAssembler() - for await (const chunk of this.stream(options)) { - assembler.push(chunk) - yield * assembler.flushReady() - } - yield * assembler.flushRemaining() - } - - /** - * One model call, fully assembled (drains the chunk stream). Dispatches - * through the `llm/generate` waterfall (and the inner stream through - * `llm/stream`). Same completion guarantees as `streamBlocks()`. - */ - generate(options: GenerateOptions): Promise { - return this.ctx.waterfall(this, 'llm/generate', options, async () => { - const assembler = new BlockAssembler() - for await (const chunk of this.stream(options)) assembler.push(chunk) - return assembler.result() - }) - } } export default LlmService diff --git a/packages/llm/src/never.ts b/packages/llm/llm/src/never.ts similarity index 100% rename from packages/llm/src/never.ts rename to packages/llm/llm/src/never.ts diff --git a/packages/llm/src/types.ts b/packages/llm/llm/src/types.ts similarity index 97% rename from packages/llm/src/types.ts rename to packages/llm/llm/src/types.ts index 863de94b16..63fc0f5b0c 100644 --- a/packages/llm/src/types.ts +++ b/packages/llm/llm/src/types.ts @@ -193,10 +193,3 @@ export interface GenerateOptions { stop?: string[] signal?: AbortSignal } - -/** Non-streaming result, assembled from the chunk stream. */ -export interface GenerateResult { - message: Message - usage?: TokenUsage - finish: FinishReason -} diff --git a/packages/llm/tests/assembler.spec.ts b/packages/llm/llm/tests/assembler.spec.ts similarity index 83% rename from packages/llm/tests/assembler.spec.ts rename to packages/llm/llm/tests/assembler.spec.ts index 6ed281add2..e8ad04e3b5 100644 --- a/packages/llm/tests/assembler.spec.ts +++ b/packages/llm/llm/tests/assembler.spec.ts @@ -81,36 +81,6 @@ describe('BlockAssembler', () => { expect(() => assembler.blocks()).toThrow('BlockAssembler invariant violated') }) - it('assembles open blocks at end of stream via flushRemaining', () => { - const assembler = new BlockAssembler() - assembler.push({ type: 'text-delta', index: 0, text: 'open' }) - assembler.push({ type: 'reasoning-delta', index: 1, text: 'thinking' }) - - // flushReady returns nothing because index 0 is incomplete and blocking - const ready = assembler.flushReady() - expect(ready).toEqual([]) - - // flushRemaining assembles everything still open - const remaining = assembler.flushRemaining() - expect(remaining).toEqual([ - { type: 'text', text: 'open' }, - { type: 'reasoning', text: 'thinking' }, - ]) - - // blocks() now matches the flushed view - expect(assembler.blocks()).toEqual(remaining) - }) - - it('result() omits usage key when no usage was received', () => { - const assembler = new BlockAssembler() - assembler.push({ type: 'text-delta', index: 0, text: 'msg' }) - const result = assembler.result() - expect(result.message).toBeDefined() - expect(result.finish).toEqual({ kind: 'stop' }) - // usage should NOT be present on the object at all - expect('usage' in result).toBe(false) - }) - it('ignores duplicate block-start for the same index', () => { const assembler = new BlockAssembler() assembler.push({ type: 'block-start', index: 0, blockType: 'text' }) @@ -142,13 +112,11 @@ describe('BlockAssembler', () => { ]) }) - it('includes usage in result() when usage was received', () => { + it('exposes usage via the getter when a usage chunk was received', () => { const assembler = new BlockAssembler() assembler.push({ type: 'text-delta', index: 0, text: 'msg' }) assembler.push({ type: 'usage', usage: { inputTokens: 5, outputTokens: 3 } }) - const result = assembler.result() - expect(result.usage).toEqual({ inputTokens: 5, outputTokens: 3 }) - expect('usage' in result).toBe(true) + expect(assembler.usage).toEqual({ inputTokens: 5, outputTokens: 3 }) }) }) @@ -172,25 +140,25 @@ describe('BlockAssembler regressions (property-test findings)', () => { // Found by fast-check (the property-testing RFC): two block-ends at the same index made the // streamed prefix (first block) disagree with final blocks() (second // block). The first close must win — same straggler rule as post-close - // deltas — so streaming and one-shot assembly stay identical. + // deltas — so the prefix returned incrementally by push() and the final + // blocks() stay identical. const chunks: StreamChunk[] = [ { type: 'block-end', index: 0, block: { type: 'reasoning', text: 'first' } }, { type: 'block-end', index: 0, block: { type: 'text', text: 'second' } }, ] const streaming = new BlockAssembler() - const flushed = [] + const closed = [] for (const chunk of chunks) { - streaming.push(chunk) - flushed.push(...streaming.flushReady()) + const block = streaming.push(chunk) + if (block) closed.push(block) } - flushed.push(...streaming.flushRemaining()) const oneShot = new BlockAssembler() for (const chunk of chunks) oneShot.push(chunk) - expect(flushed).toEqual([{ type: 'reasoning', text: 'first' }]) + expect(closed).toEqual([{ type: 'reasoning', text: 'first' }]) expect(oneShot.blocks()).toEqual([{ type: 'reasoning', text: 'first' }]) - expect(flushed).toEqual(oneShot.blocks()) + expect(closed).toEqual(oneShot.blocks()) }) it('push returns undefined for a duplicate block-end (it closed nothing)', () => { diff --git a/packages/llm/tests/properties.spec.ts b/packages/llm/llm/tests/properties.spec.ts similarity index 69% rename from packages/llm/tests/properties.spec.ts rename to packages/llm/llm/tests/properties.spec.ts index 13c7bbd8c2..c63d56abbb 100644 --- a/packages/llm/tests/properties.spec.ts +++ b/packages/llm/llm/tests/properties.spec.ts @@ -4,13 +4,13 @@ * The assembler is protocol-shaped: arbitrary interleavings of block-start, * deltas, block-end, usage, and finish — valid and malformed (duplicate * indices, stragglers after block-end, missing block-start, delta-only). The - * invariants below are the contract the agent loop and LlmService rely on. + * invariants below are the contract the agent loop relies on. */ import { describe, expect, it } from 'vitest' import fc from 'fast-check' import { BlockAssembler } from '@deepseek-ai/dsh-llm' -import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm' +import type { StreamChunk } from '@deepseek-ai/dsh-llm' import { CallId } from '@deepseek-ai/dsh-llm' // A small pool of indices so collisions (duplicate-index bugs) are common. @@ -55,38 +55,6 @@ function feed(chunks: StreamChunk[]): BlockAssembler { } describe('BlockAssembler properties', () => { - it('flushReady() ++ flushRemaining() === blocks(), in order', () => { - fc.assert(fc.property(streamArb, (chunks) => { - const streaming = new BlockAssembler() - const flushed: ContentBlock[] = [] - for (const chunk of chunks) { - streaming.push(chunk) - flushed.push(...streaming.flushReady()) - } - flushed.push(...streaming.flushRemaining()) - - const oneShot = feed(chunks).blocks() - expect(flushed).toEqual(oneShot) - })) - }) - - it('streamBlocks-style flush never yields a block before an earlier open one', () => { - // flushReady is strict-order: once it stops at an open index, no later - // index may be emitted until that one closes. We assert the flushed prefix - // is always a prefix of the final blocks() order. - fc.assert(fc.property(streamArb, (chunks) => { - const streaming = new BlockAssembler() - const flushed: ContentBlock[] = [] - for (const chunk of chunks) { - streaming.push(chunk) - flushed.push(...streaming.flushReady()) - } - const finalSoFar = streaming.blocks() - // Everything flushed mid-stream is a prefix of the full ordered blocks. - expect(finalSoFar.slice(0, flushed.length)).toEqual(flushed) - })) - }) - it('partials map size never exceeds the number of distinct indices seen', () => { fc.assert(fc.property(streamArb, (chunks) => { const distinct = new Set() @@ -131,20 +99,4 @@ describe('BlockAssembler properties', () => { } })) }) - - it('streaming and one-shot assembly agree on usage and finish', () => { - fc.assert(fc.property(streamArb, (chunks) => { - // Streaming consumer: push + flush as it goes. - const streaming = new BlockAssembler() - for (const chunk of chunks) { - streaming.push(chunk) - streaming.flushReady() - } - streaming.flushRemaining() - // One-shot consumer: push all, then read. - const oneShot = feed(chunks) - expect(streaming.usage).toEqual(oneShot.usage) - expect(streaming.finish).toEqual(oneShot.finish) - })) - }) }) diff --git a/packages/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts similarity index 68% rename from packages/llm/tests/service.spec.ts rename to packages/llm/llm/tests/service.spec.ts index 35333c0ffd..f669069c44 100644 --- a/packages/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -19,24 +19,22 @@ const SCRIPT: StreamChunk[] = [ ] describe('LlmService', () => { - it('routes stream() to the registered adapter and generate() assembles it', async () => { + it('routes stream() to the registered adapter', async () => { const ctx = new Context() await ctx.plugin(LlmService) ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT)) const chunks: StreamChunk[] = [] for await (const chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) chunks.push(chunk) - expect(chunks).toHaveLength(3) - - const result = await ctx.llm.generate({ model: 'test-model', messages: [] }) - expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }]) - expect(result.finish).toEqual({ kind: 'stop' }) + expect(chunks).toEqual(SCRIPT) }) it('throws NO_ADAPTER for unregistered models', async () => { const ctx = new Context() await ctx.plugin(LlmService) - await expect(ctx.llm.generate({ model: 'nope', messages: [] })).rejects.toThrow('no adapter registered') + await expect((async () => { + for await (const _ of ctx.llm.stream({ model: 'nope', messages: [] })) { /* drain */ } + })()).rejects.toThrow('no adapter registered') }) it('unregisters adapters when the owning fiber is disposed (HMR safety)', async () => { @@ -71,21 +69,6 @@ describe('LlmService', () => { expect(chunks[0]).toMatchObject({ index: 99 }) }) - it('lets llm/generate waterfall listeners intercept and transform the result', async () => { - const ctx = new Context() - await ctx.plugin(LlmService) - ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT)) - - ctx.on('llm/generate', async function (_options, next) { - const result = await next() - return { ...result, finish: { kind: 'max-tokens' } as const } - }) - - const result = await ctx.llm.generate({ model: 'test-model', messages: [] }) - expect(result.finish).toEqual({ kind: 'max-tokens' }) - expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }]) - }) - it('creates LlmError with a code for programmatic handling', () => { const err = new LlmError('something went wrong', 'CUSTOM_CODE') expect(err).toBeInstanceOf(Error) @@ -116,20 +99,13 @@ describe('LlmService', () => { expect(isHarnessError('nope')).toBe(false) }) - it('disposes adapter registration on adapter-change event emission', async () => { + it('removes the adapter when the returned disposer is called', async () => { const ctx = new Context() await ctx.plugin(LlmService) - const changes: string[][] = [] - ctx.on('llm/adapter-change', () => { - changes.push([...ctx.llm.models()]) - }) - const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT)) - expect(changes).toEqual([['m1']]) - + expect(ctx.llm.models()).toEqual(['m1']) dispose() - expect(changes).toEqual([['m1'], []]) expect(ctx.llm.models()).toEqual([]) }) @@ -147,25 +123,19 @@ describe('LlmService', () => { } }) - it('rolls back the adapter entry when an adapter-change listener throws (P1-1)', async () => { + it('re-registers a model after its prior registration is disposed', async () => { const ctx = new Context() await ctx.plugin(LlmService) - // A change listener that throws on the FIRST emit only. - let threw = false - ctx.on('llm/adapter-change', () => { - if (!threw) { threw = true; throw new Error('boom change listener') } - }) - - // The throwing emit must roll the mutation back, not leak it. - expect(() => ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))).toThrow('boom change listener') - expect(ctx.llm.models()).toEqual([]) // entry rolled back, not leaked - - // A subsequent listener-free register of the SAME model succeeds and - // contributes exactly once (the duplicate check is not wedged). const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT)) expect(ctx.llm.models()).toEqual(['m1']) dispose() expect(ctx.llm.models()).toEqual([]) + + // The duplicate check is not wedged: the same model registers cleanly again. + const disposeAgain = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT)) + expect(ctx.llm.models()).toEqual(['m1']) + disposeAgain() + expect(ctx.llm.models()).toEqual([]) }) }) diff --git a/packages/llm/llm/tsconfig.json b/packages/llm/llm/tsconfig.json new file mode 100644 index 0000000000..342f636170 --- /dev/null +++ b/packages/llm/llm/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../util/brand" + } + ] +} diff --git a/packages/llm/src/brand.ts b/packages/llm/src/brand.ts deleted file mode 100644 index 38d69fe37b..0000000000 --- a/packages/llm/src/brand.ts +++ /dev/null @@ -1,32 +0,0 @@ -/** - * Branded (nominal) ID types. - * - * A brand makes structurally-identical strings non-interchangeable at the - * type level: an `AgentId` cannot be passed where a `CallId` is expected, - * even though both are strings at runtime. Construction goes through the - * per-type factory (a plain cast inside — zero runtime cost); comparison, - * logging, and serialization all behave as ordinary strings. - * - * Policy: core packages brand the IDs they own — `CallId` here (tool-call - * correlation), `SessionId` in dsh-session, `AgentId` in dsh-agent. Branding - * is for IDs that cross package boundaries and could plausibly be confused; - * not every string needs a brand. - * - * @module @deepseek-ai/dsh-llm/brand - */ - -declare const BRAND: unique symbol - -/** A string carrying a compile-time-only brand `B`. */ -export type Branded = string & { readonly [BRAND]: B } - -/** - * Correlates a model-issued tool call with its result. Provider-issued for - * real adapters; synthesized by mocks/assembler fallbacks. - */ -export type CallId = Branded<'CallId'> - -/** Brand a string as a {@link CallId}. */ -export function CallId(id: string): CallId { - return id as CallId -} diff --git a/packages/llm/tsconfig.json b/packages/llm/tsconfig.json deleted file mode 100644 index 2617271c44..0000000000 --- a/packages/llm/tsconfig.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" } - ] -} diff --git a/packages/session-persistence-jsonl/src/index.ts b/packages/session-persistence-jsonl/src/index.ts deleted file mode 100644 index 7a0c97637c..0000000000 --- a/packages/session-persistence-jsonl/src/index.ts +++ /dev/null @@ -1,882 +0,0 @@ -/** - * JSONL durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-jsonl`). - * - * Two concerns in one plugin: - * - * 1. **The backend** — a concrete {@link SessionPersistence}: one append-only - * `.jsonl` event log per session (a header line then one `SessionEvent` per - * line, verbatim including `assistant/chunk` so `seq` stays contiguous) plus - * a small atomic `.summary.json` sidecar for the mutable `SessionSummary`. - * Lazy materialization (no file until the first `append`), atomic first - * write, and load-time repair of a never-committed crash tail. - * - * 2. **The write path** — the `session/event` → buffer → `session/flush` drain - * that generalizes the example `session-jsonl.ts`: snapshot each event when - * it is buffered (the live `session.events` object is mutable), persist - * forks once on `session/created`, maintain a per-session write cursor so a - * resumed session never re-appends already-stored events, and seed existing - * live sessions on plugin apply (HMR does not replay `session/created`). - * - * @module @deepseek-ai/dsh-session-persistence-jsonl - */ - -import { Context } from 'cordis' -import z from 'schemastery' -import { open, mkdir, readFile, readdir, rename, link, rm, truncate } from 'node:fs/promises' -import { dirname, resolve } from 'node:path' -import { randomBytes } from 'node:crypto' -import { - SessionPersistence, assertSerializable, seedCoversPrefix, -} from '@deepseek-ai/dsh-session-persistence' -import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' -import { - encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, sidecarPath, toHeaderLine, -} from './format.ts' - -export interface Config { - /** - * Root directory for all session files. Required (no default): a default of - * `process.cwd()` would scatter session files as the process's cwd changes - * (bash calls, subprocesses). Sessions group under per-cwd subdirectories. - */ - root: string -} - -/** Per-session write state held by the backend's in-memory bookkeeping. */ -interface SessionState { - meta: SessionMeta - /** The next seq the backend expects to append (the stored log length). */ - cursor: number - /** Whether the `.jsonl` file has been physically materialized. */ - materialized: boolean - /** - * The live Session this state was bound to via `onCreated`, if any. Used to - * detect a DIFFERENT live session reusing a tracked id (a collision): state - * created through the public `create()`/`load()` API has no owner, but state - * bound to a live session lets `onCreated` reject a second, unrelated session - * object on the same id instead of silently no-opping (which would leave the - * new session's events to be dropped against the old cursor). - */ - owner?: Session -} - -/** - * Whether `error` is a "no such file/directory" (`ENOENT`) failure — the ONLY - * filesystem error that legitimately means "this session/root is absent" for a - * durable backend. Any OTHER error (`EACCES`, `ENOTDIR`, transient I/O) must - * surface rather than be silently reported as absence: masking it would let - * `list()` report no sessions, `load()` report "not found", and collision - * checks proceed under a false absence assumption — all unsafe for durable - * persistence. (A NodeJS filesystem rejection carries a string `code`.) - */ -function isENOENT(error: unknown): boolean { - return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT' -} - -async function settledErrors(promises: Iterable>): Promise { - const settled = await Promise.allSettled([...promises]) - const errors: unknown[] = [] - for (const result of settled) { - if (result.status === 'rejected') errors.push(result.reason) - } - return errors -} - -/** - * The JSONL persistence backend. Load as a plugin; it registers as - * `ctx.sessionPersistence` and installs the write-path listeners. - */ -export class SessionPersistenceJsonl extends SessionPersistence { - static inject = ['sessions'] - - static Config: z = z.object({ - root: z.string().required(), - }) - - private root: string - /** Backend bookkeeping keyed by session id (NOT the live Session object). */ - private states = new Map() - /** Write-behind buffers keyed by the live Session (write path). */ - private buffers = new Map() - /** - * Per-session serialization: every backend operation chains onto the prior - * one for the same id, so concurrent flushes / a flush racing onCreated never - * interleave file writes or read a half-built state. Keyed by session id. - */ - private chains = new Map>() - /** - * Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not - * its id: a disposed fiber's session can be replaced by a different live - * Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache - * would hand the new object the old object's init promise — skipping - * onCreated for the new session, so its events start at seq 0 while flush - * filters against the stale cursor and silently drops them. Keying by object - * gives each live Session its own init. flush awaits it before appending. - */ - private inits = new Map>() - - constructor(ctx: Context, public config: Config) { - super(ctx) - // Resolve the configured root to an ABSOLUTE path ONCE, here. A relative - // root (the examples use `./.sessions`) would otherwise re-resolve against - // `process.cwd()` at every later readdir/open — so if any plugin or test - // changed cwd between create, append, and load, one session's files could - // split across directories. Pinning it at construction makes all paths - // stable regardless of later cwd changes. - this.root = resolve(config.root) - this.installWritePath() - } - - // --- SessionPersistence backend surface (all serialized per session id) --- - - create(meta: SessionMeta): Promise { - // Snapshot the metadata at call time: the op runs later (behind the - // per-session chain) and the snapshot is also stored as the lazy state, so - // keeping the caller's object by reference would let a later mutation of - // `id`/`cwd` register under one key but materialize under a different - // path/header. A shallow copy is enough — SessionMeta is a flat record. - const snapshot: SessionMeta = { ...meta } - return this.serialize(snapshot.id, () => this.createCore(snapshot)) - } - - private async createCore(meta: SessionMeta): Promise { - // Do NOT clobber an existing session. If we already track it, or a log - // exists on disk under this id, refuse — the SessionId IS the identity, and - // silently resetting state (cursor 0, materialized false) over committed - // data would let the next append rename over the existing log. - if (this.states.has(meta.id)) { - throw new Error(`session "${meta.id}" already exists in this backend`) - } - // Scan ALL cwd buckets (pass undefined), not just meta.cwd's: load/has/adopt - // identify a session by id alone and search every bucket, so an id already - // persisted under a DIFFERENT cwd must still block creation here. Probing - // only meta.cwd's bucket would let two logs share one id and make resume - // (which picks the first matching bucket) nondeterministic. - if (await this.findLog(meta.id, undefined) !== undefined) { - throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`) - } - // Pure lazy: record intent only. No file until the first append, so an - // abandoned (never-appended) session leaves nothing on disk and stays - // absent from has()/list(). - this.states.set(meta.id, { meta, cursor: 0, materialized: false }) - } - - /** - * Run `op` after any in-flight operation for the same session id, so writes - * for one session never interleave (two flushes, a flush racing a load, an - * update racing an append). Errors do not poison the chain — the next op - * still runs. NOTE: serialized public methods must NOT call each other (that - * would deadlock on the same chain); they call the unserialized `*Core` - * helpers instead. - */ - private serialize(id: SessionId, op: () => Promise): Promise { - const prior = this.chains.get(id) ?? Promise.resolve() - const next = prior.then(op, op) - // Keep the chain alive but swallow this op's rejection for the NEXT waiter - // (the caller still sees the real rejection via `next`). - this.chains.set(id, next.then(() => undefined, () => undefined)) - return next - } - - // `async` so the synchronous validate/clone below reject (not throw) per the - // Promise contract — callers use `await expect(...).rejects`. - async append(id: SessionId, events: readonly SessionEvent[]): Promise { - // Validate serializability BEFORE cloning, so a bad event surfaces the typed - // "non-JSON-serializable" error rather than an opaque DataCloneError from - // structuredClone below. (In an async method this throw becomes a rejection, - // honoring the Promise contract rather than throwing synchronously.) - assertSerializable(events) - // Deep-snapshot the batch here, BEFORE the op waits behind the per-session - // chain: the op may await before serializing, so a caller that passes a live - // array (e.g. session.events) and mutates it — OR mutates an event object - // inside it — before the op runs would otherwise have those changes - // persisted, or advance the cursor past what was actually written. - // structuredClone covers both the array and the event objects (safe now that - // serializability is checked above). The clone happens synchronously (before - // the first await), so it is taken at call time. - const batch = events.map(e => structuredClone(e)) - return this.serialize(id, () => this.appendCore(id, batch)) - } - - private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { - if (events.length === 0) return - assertSerializable(events) - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) // calls loadCore, not load - - // Contiguity contract: each event's seq must continue the stored log. - for (const [i, event] of events.entries()) { - if (event.seq !== state.cursor + i) { - throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`) - } - } - - if (!state.materialized) { - await this.materialize(state, events) - } else { - await this.appendLines(state, events) - } - // The durable event log is the transaction: advance the cursor as soon as - // the log write commits. The sidecar (mutable summary) is best-effort here - // — a failed sidecar write must NOT reject an append whose log already - // landed (that would desync the cursor and let a retry duplicate seqs). - state.cursor += events.length - await this.touchSummary(state).catch(() => { /* sidecar is recoverable metadata; log is durable */ }) - } - - load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { - return this.serialize(id, () => this.loadCore(id)) - } - - private async loadCore(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { - const cwd = this.states.get(id)?.meta.cwd - const file = await this.findLog(id, cwd) - if (file === undefined) throw new Error(`session "${id}" not found`) - const buffer = await readFile(file.path) - const { meta, events, committedBytes } = scanLog(buffer) - this.assertVersion(meta) - - const summary = await this.readSidecar(id, meta.cwd) - const fullMeta: SessionMeta = { ...meta, ...summary } - - // Crash-recovery: if the log ended mid-turn (an open turn with real, - // preserved events but no closing turn/end), close it durably DURING load so - // disk, the returned log, and the cursor all agree — both append routes then - // continue with no special-casing. Synthesize the boundary events (a - // step/end if a step was open, then a turn/end {kind:'interrupted'}); the - // interrupted turn's real events are preserved, never truncated (a turn can - // be huge — the session-persistence RFC). - const closers = interruptedTurnClosers(events) - const balanced = [...events, ...closers] - - // Set state BEFORE the repair writes so they can resolve the log path. - const needsTorn = committedBytes < buffer.byteLength - const state: SessionState = { - meta: { ...fullMeta }, - cursor: events.length, - materialized: true, - } - this.states.set(id, state) - - if (needsTorn) { - // Discard the torn trailing fragment (a final line never fully flushed) - // before writing the closers, so the closers land at a clean EOF. - await this.repair(state, committedBytes) - } - if (closers.length > 0) { - // Durably append the synthetic closers, then advance the cursor to the - // balanced length. After this, disk == balanced and the next append (live - // or direct) continues cleanly. No sidecar touch here: load is not a - // summary-changing op (the closers carry no new title/firstPrompt), and - // the next real append bumps `updatedAt` — keeping the summary write off - // the recovery path avoids a second best-effort failure mode. - await this.appendLines(state, closers) - state.cursor = balanced.length - } - - return { meta: fullMeta, events: balanced } - } - - private async adoptLiveDiskPrefix( - session: Session, - seed: readonly SessionEvent[], - file: { path: string; cwd: string | undefined }, - ): Promise { - const buffer = await readFile(file.path) - const { meta, events, committedBytes } = scanLog(buffer) - this.assertVersion(meta) - if (!seedCoversPrefix(seed, events)) { - throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`) - } - - const summary = await this.readSidecar(session.header.id, meta.cwd) - const state: SessionState = { - meta: { ...meta, ...summary }, - cursor: events.length, - materialized: true, - owner: session, - } - this.states.set(session.header.id, state) - - if (committedBytes < buffer.byteLength) { - await this.repair(state, committedBytes) - } - const suffix = seed.slice(events.length) - if (suffix.length > 0) await this.appendCore(session.header.id, suffix) - } - - async list(): Promise { - const metas: SessionMeta[] = [] - for (const dir of await this.listCwdDirs()) { - for (const name of await this.listJsonl(dir)) { - // Read ONLY the header line, not the whole log: a session picker must - // scale with the number of sessions, not the total size of every - // conversation (the log persists every assistant/chunk verbatim, so a - // full scanLog here would be O(total history)). - const first = await this.readFirstLine(`${dir}/${name}`) - if (first === undefined) continue // empty/half-written file - const meta = parseHeaderMeta(first) - if (meta === undefined) continue // not a session header - const summary = await this.readSidecarForList(meta.id, meta.cwd) - metas.push({ ...meta, ...summary }) - } - } - return metas - } - - /** - * Read the first newline-terminated line of a file without loading the whole - * file. Returns undefined if the file is empty or has no complete first line - * (a half-written log). Reads in bounded chunks so a huge log costs only the - * header read. - */ - private async readFirstLine(path: string): Promise { - const handle = await open(path, 'r') - try { - const chunks: Buffer[] = [] - const buf = Buffer.alloc(8192) - for (;;) { - const { bytesRead } = await handle.read(buf, 0, buf.length, null) - if (bytesRead === 0) return undefined // EOF with no newline → no complete line - const slice = buf.subarray(0, bytesRead) - const nl = slice.indexOf(0x0a) - if (nl !== -1) { - chunks.push(slice.subarray(0, nl)) - return Buffer.concat(chunks).toString('utf8') - } - chunks.push(Buffer.from(slice)) - } - } finally { - await handle.close() - } - } - - async has(id: SessionId): Promise { - const state = this.states.get(id) - if (state?.materialized) return true - const cwd = state?.meta.cwd - return (await this.findLog(id, cwd)) !== undefined - } - - delete(id: SessionId): Promise { - return this.serialize(id, () => this.deleteCore(id)) - } - - private async deleteCore(id: SessionId): Promise { - const cwd = this.states.get(id)?.meta.cwd - const file = await this.findLog(id, cwd) - if (file) await rm(file.path, { force: true }) - // Remove the sidecar too. A lazy session (update() before the first - // append()) has a `.summary.json` sidecar but NO `.jsonl` log, and after a - // restart the in-memory cwd is gone — so keying sidecar removal off the log - // or the in-memory cwd would leak its possibly-sensitive title/firstPrompt. - // Scan every cwd bucket for the sidecar by its (sanitized) filename. - await this.removeSidecars(id) - this.states.delete(id) - } - - /** Remove a session's summary sidecar from EVERY cwd bucket (id is unique). */ - private async removeSidecars(id: SessionId): Promise { - const target = `${encodeSegment(id)}.summary.json` - for (const dir of await this.listCwdDirs()) { - await rm(`${dir}/${target}`, { force: true }) - } - } - - update(id: SessionId, summary: Partial): Promise { - return this.serialize(id, () => this.updateCore(id, summary)) - } - - private async updateCore(id: SessionId, summary: Partial): Promise { - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) - // Build the NEXT meta separately and commit it to in-memory state only AFTER - // the sidecar write succeeds. update's only durable effect is the sidecar, - // so a failure DOES reject (unlike append, whose log is the transaction and - // sidecar is best-effort) — but if we mutated state.meta first, a later - // touchSummary() on a successful append would persist the rejected - // title/firstPrompt, making a failed update durable after the fact. - const nextMeta: SessionMeta = { ...state.meta, ...summary, updatedAt: summary.updatedAt ?? Date.now() } - if (state.materialized) await this.writeSidecar(nextMeta) - state.meta = nextMeta - } - - // --- materialization / append / repair --- - - /** Atomically write the header line + first batch (temp-write, fsync, rename). */ - private async materialize(state: SessionState, events: readonly SessionEvent[]): Promise { - const dir = sessionDir(this.root, state.meta.cwd) - await mkdir(this.root, { recursive: true, mode: 0o700 }) - await this.syncDir(dirname(this.root)) - await mkdir(dir, { recursive: true, mode: 0o700 }) - await this.syncDir(this.root) - const finalPath = logPath(this.root, state.meta.cwd, state.meta.id) - // Never rename over an existing committed log: materialize is the FIRST - // write of a session the backend believes is new. A file here means a - // different session shares this id on disk — reject loudly rather than - // clobber committed data. (createCore already guards the create path before - // this point, so this is unreachable-in-practice defense-in-depth against a - // TOCTOU/fork race; ignored for coverage.) - /* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */ - if (await this.exists(finalPath)) { - throw new Error(`refusing to materialize "${state.meta.id}": a log already exists on disk (load/resume it instead)`) - } - const header = JSON.stringify(toHeaderLine(state.meta)) - const body = events.map(eventLine).join('\n') - const content = header + '\n' + body + '\n' - - const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp` - const handle = await open(tmp, 'wx', 0o600) - try { - await handle.writeFile(content) - await handle.sync() - } finally { - await handle.close() - } - // Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the - // final path already exists, so two processes materializing the same id - // concurrently cannot clobber each other (both could pass the exists() check - // above, but only one link() wins). rename() would silently overwrite the - // log the other process just committed. - let linked = false - try { - await link(tmp, finalPath) - linked = true - } finally { - // If link FAILED (EEXIST on a race, or any I/O error), the temp is the - // only reference and must be removed before the original error propagates. - // If link SUCCEEDED, the temp cleanup is deferred to AFTER the publish is - // durable (below) so a temp-rm failure can never reject a session whose - // log already published — that would leave state.materialized false and - // wedge every retry on the exists() backstop above. - /* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */ - if (!linked) await rm(tmp, { force: true }) - } - // link() succeeded — the log is published. fsync the directory so the new - // entry survives a power loss: on POSIX filesystems the new link is not - // crash-durable until the parent directory's metadata is synced. The seam - // contract is "append returns once durable", and materialize is the first - // append's write — so the directory entry must be durable before we return. - await this.syncDir(dir) - state.materialized = true - // Best-effort temp cleanup: the log is already published and durable, so a - // failure to remove the (now-redundant) temp hard link must NOT reject the - // append. A leftover `*.tmp` is harmless — it is never read, and the next - // materialize of this id is guarded by exists()/link(). Swallow only the - // rm failure; nothing else of consequence runs in the try. - try { - await rm(tmp, { force: true }) - } catch { - /* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */ - } - } - - /** fsync a directory so a just-created/renamed entry inside it is crash-durable. */ - private async syncDir(dir: string): Promise { - const handle = await open(dir, 'r') - try { - await handle.sync() - } finally { - await handle.close() - } - } - - /** - * Append event lines at EOF and fsync. On a write/sync failure AFTER the - * kernel accepted some bytes (ENOSPC, an fsync error), truncate the file back - * to its pre-append size before rethrowing: `cursor` is unchanged, so the - * batch will be retried, and without this rollback the retry would append - * AFTER the partial bytes — producing duplicate seqs that make `scanLog` see a - * gap and render the session unloadable. - */ - private async appendLines(state: SessionState, events: readonly SessionEvent[]): Promise { - const path = logPath(this.root, state.meta.cwd, state.meta.id) - const handle = await open(path, 'a') - try { - const { size: before } = await handle.stat() - try { - await handle.writeFile(events.map(eventLine).join('\n') + '\n') - await handle.sync() - } catch (error) { - // Roll back whatever bytes landed so a retry starts from a clean EOF. - await handle.truncate(before) - await handle.sync() - throw error - } - } finally { - await handle.close() - } - } - - /** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */ - private async repair(state: SessionState, offset: number): Promise { - const path = logPath(this.root, state.meta.cwd, state.meta.id) - await truncate(path, offset) - const handle = await open(path, 'r+') - try { - await handle.sync() - } finally { - await handle.close() - } - } - - // --- sidecar (mutable summary) --- - - private async touchSummary(state: SessionState): Promise { - state.meta = { ...state.meta, updatedAt: Date.now() } - await this.writeSidecar(state.meta) - } - - /** - * Atomic sidecar write (temp-write + rename), summary fields only. - * - * Deliberately NOT directory-fsynced (unlike {@link materialize}): the - * sidecar holds mutable, recoverable summary metadata (updatedAt, title, - * firstPrompt), not source-of-truth log data. The rename is atomic so a - * reader never sees a torn file, but a power loss may lose the most recent - * summary — acceptable because it is re-derivable and the durable log (the - * transaction) is independently synced. Strict crash-durability is reserved - * for the event log. - */ - private async writeSidecar(meta: SessionMeta): Promise { - const dir = sessionDir(this.root, meta.cwd) - await mkdir(dir, { recursive: true, mode: 0o700 }) - const path = sidecarPath(this.root, meta.cwd, meta.id) - const summary: SessionSummary = { - updatedAt: meta.updatedAt, - ...meta.title !== undefined ? { title: meta.title } : {}, - ...meta.firstPrompt !== undefined ? { firstPrompt: meta.firstPrompt } : {}, - } - const tmp = `${path}.${randomBytes(6).toString('hex')}.tmp` - // Exclusive owner-only create ('wx', 0o600), matching the log-materialization - // temp write: the sidecar can carry user data (title/firstPrompt), so a - // predictable/pre-existing temp path must never be silently truncated and - // followed (symlink race / disclosure). The random suffix already makes a - // collision unlikely; 'wx' makes reuse an error rather than a clobber. - const handle = await open(tmp, 'wx', 0o600) - try { - await handle.writeFile(JSON.stringify(summary)) - } finally { - await handle.close() - } - await rename(tmp, path) - } - - /** - * Read the mutable-summary sidecar, or `undefined` if it is absent (a session - * that has never been `update()`d). Non-ENOENT failures surface on strict - * load/adopt paths so corrupt metadata does not masquerade as a clean default. - */ - private async readSidecar(id: SessionId, cwd: string | undefined): Promise { - try { - const raw = await readFile(sidecarPath(this.root, cwd, id), 'utf8') - return JSON.parse(raw) as SessionSummary - } catch (error) { - if (isENOENT(error)) return undefined - throw error - } - } - - /** - * Best-effort summary read for list(): a corrupt sidecar should degrade one - * row to header metadata, not hide every session from a picker. - */ - private async readSidecarForList(id: SessionId, cwd: string | undefined): Promise { - try { - return await this.readSidecar(id, cwd) - } catch (error: unknown) { - this.ctx.logger.warn(`session-persistence-jsonl: ignoring unreadable summary for session "${id}" while listing: ${String(error)}`) - return undefined - } - } - - // --- discovery helpers --- - - /** Find a session's log file across cwd buckets (when cwd is unknown). */ - private async findLog(id: SessionId, cwd: string | undefined): Promise<{ path: string; cwd: string | undefined } | undefined> { - if (cwd !== undefined) { - const path = logPath(this.root, cwd, id) - return (await this.exists(path)) ? { path, cwd } : undefined - } - // Unknown cwd: scan buckets for a matching file name. - const target = encodeSegment(id) + '.jsonl' - for (const dir of await this.listCwdDirs()) { - const path = `${dir}/${target}` - if (await this.exists(path)) { - // Recover cwd from the header for accurate sidecar pathing. - const { meta } = scanLog(await readFile(path)) - return { path, cwd: meta.cwd } - } - } - return undefined - } - - /** The cwd-bucket directories under the root (absolute paths). */ - private async listCwdDirs(): Promise { - try { - const entries = await readdir(this.root, { withFileTypes: true }) - return entries.filter(e => e.isDirectory()).map(e => `${this.root}/${e.name}`) - } catch (error) { - // ENOENT = the root has not been created yet → genuinely no sessions. - // Any other error (EACCES, ENOTDIR, transient I/O) must NOT be reported - // as "no sessions" — a durable backend cannot silently pretend persisted - // state is absent on a storage fault. - if (isENOENT(error)) return [] - throw error - } - } - - private async listJsonl(dir: string): Promise { - const entries = await readdir(dir) - return entries.filter(n => n.endsWith('.jsonl')) - } - - private async exists(path: string): Promise { - try { - const handle = await open(path, 'r') - await handle.close() - return true - } catch (error) { - // Only ENOENT means absent. A permission/I/O error must surface, not be - // collapsed to `false` — otherwise load() reports "not found" and - // collision checks proceed under a false absence assumption. - if (isENOENT(error)) return false - throw error - } - } - - /** Build a state for a session discovered on disk but not yet in memory. */ - private async adopt(id: SessionId): Promise { - // loadCore (NOT load) — adopt runs inside an already-serialized op, so - // re-entering the chain via the public load() would deadlock. - await this.loadCore(id) - const state = this.states.get(id) - /* v8 ignore next -- loadCore always sets the state for the id */ - if (!state) throw new Error(`failed to adopt session "${id}"`) - return state - } - - private assertVersion(meta: SessionMeta): void { - if (meta.version !== 1) { - throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) - } - } - - // --- write path (session/event → flush drain) --- - - private installWritePath(): void { - const ctx = this.ctx - - // Capture the header on creation; persist a fork's seed once. Record the - // init promise so flush/dispose can await it (onCreated is async). - ctx.on('session/created', (session) => { void this.initFor(session) }) - - // Snapshot + buffer every event (the live object is mutable; clone so a - // later in-place mutation of session.events cannot rewrite a buffered - // event). Serializability is guaranteed at the source — `Session.append` - // rejects non-JSON-serializable data before the event ever enters the log - // or this emit — so structuredClone here can never hit a non-cloneable - // value, and the durable log can never diverge from session.events. - ctx.on('session/event', (session, event) => { - let buffer = this.buffers.get(session) - if (!buffer) this.buffers.set(session, buffer = []) - buffer.push(structuredClone(event)) - }) - - // Drain to the backend at the durability checkpoint. - ctx.on('session/flush', session => this.flush(session)) - - // Dispose must reach quiescence: await every session's init + final drain - // BEFORE returning, so no write lands after teardown (orphan rename/ENOENT). - ctx.effect(() => async () => { - const errors = [ - ...await settledErrors(this.inits.values()), - ...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))), - ...await settledErrors(this.chains.values()), - ] - if (errors.length > 0) { - throw new AggregateError(errors, 'session-persistence-jsonl dispose failed') - } - }, 'session-persistence-jsonl write path') - - // HMR: a hot reload does not replay session/created, so seed existing live - // sessions (mirrors dsh-invariants). - for (const session of ctx.sessions.list()) void this.initFor(session) - } - - /** Start (once) the async init for a session and remember its promise. */ - private initFor(session: Session): Promise { - const existing = this.inits.get(session) - if (existing) return existing - // Snapshot the seed SYNCHRONOUSLY here — initFor runs inside the - // `session/created` emit, before any later `append` adds non-seed events. - // A clone freezes it against later mutation of the live event objects. - const seed = session.events.map(e => structuredClone(e)) - const p = this.onCreated(session, seed) - // Attach a no-op rejection handler so a failing init (e.g. an id collision) - // does not surface as an unhandled rejection if no flush observes `p` before - // it rejects. The REAL error is still delivered: flush/dispose await the - // same `p` from the map and see the rejection there. - p.catch(() => { /* observed by flush/dispose via the stored promise */ }) - this.inits.set(session, p) - return p - } - - /** - * Whether a live `session`'s `seed` reproduces the first `cursor` persisted - * events. Reads the on-disk committed prefix and compares. A `cursor` of 0 - * (nothing persisted yet) trivially matches. Used when a live session claims - * ownerless state left by a prior `load()`/`create()` — to reject a fresh, - * unrelated session that reuses the id and would otherwise have its seq - * 0..cursor-1 events filtered as already-written. - */ - private async seedMatchesPersisted(session: Session, seed: readonly SessionEvent[], cursor: number): Promise { - if (cursor === 0) return true - const onDisk = await this.findLog(session.header.id, session.header.cwd) - /* v8 ignore next -- a cursor > 0 means the log was materialized, so it exists */ - if (onDisk === undefined) return false - const { events: diskEvents } = scanLog(await readFile(onDisk.path)) - return seedCoversPrefix(seed, diskEvents.slice(0, cursor)) - } - - /** - * On session/created: sync the backend's in-memory state to a live Session. - * - * Cases, by whether this backend tracks the id and whether a log is on disk: - * 1. Already in `states` (created here, or a prior load/resume) → no-op. - * 2. Not tracked, a log EXISTS on disk, and it is a seq-aligned PREFIX of the - * live session's current events → ADOPT it (HMR/reload): a fresh backend - * instance (empty `states`) meets a live session whose log a previous - * instance materialized; the live object already carries that history (it - * is the source of truth this run), so we continue from the stored length - * instead of re-creating. This keeps persistence alive across hot reload. - * 3. Not tracked, a log EXISTS on disk, but it is NOT a prefix of the live - * session's events → REJECT: a different session collides on the id. The - * SessionId is the identity, so two unrelated sessions sharing one is a - * bug, not a resume — fail loudly rather than clobber committed data. - * 4. Not tracked and NO log on disk → a genuinely new session: register its - * meta (lazy) and persist its `seed` once. - * - * The public `create(meta)` API is stricter still (rejects ANY on-disk id): - * there the caller asserts "brand new", so even a prefix match is a bug. - * - * The seed events were copied into the Session by its constructor WITHOUT - * emitting session/event, so the write-behind buffer never sees them — the - * one explicit `append(seed)` below is the only persistence of the seed. - * Events appended AFTER creation flow through the session/event buffer and - * are persisted by flush (filtered by the write cursor), never here. - */ - private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise { - const id = session.header.id - const tracked = this.states.get(id) - if (tracked !== undefined) { - // case 1: already tracked. - // (owner === session is a defensive same-object guard: initFor dedupes by - // session object, so onCreated never actually runs twice for one session.) - /* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */ - if (tracked.owner === session) return - if (tracked.owner === undefined) { - // Ownerless state was created via the public create()/load() API. The - // FIRST live session to arrive claims it — but ONLY if its seed is the - // already-persisted prefix. A load() for preview leaves cursor at the - // persisted length; a fresh, unrelated session reusing that id has a - // seed shorter than (or not matching) that prefix, so flush would filter - // its seq 0..cursor-1 events as already-written and silently graft the - // new conversation onto the old log. Verify the seed covers the cursor. - if (!await this.seedMatchesPersisted(session, seed, tracked.cursor)) { - throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`) - } - tracked.owner = session - // Persist the live seed SUFFIX beyond the persisted prefix. Constructor - // seed events (from sessions.create(id, { seed })) never emit - // session/event, so the write-behind buffer never sees them — without - // this they would be lost and a later flush would seq-mismatch. (cursor - // is 0 for a public create(), so this covers the whole seed there.) - const suffix = seed.slice(tracked.cursor) - if (suffix.length > 0) await this.append(id, suffix) - return - } - // The state is owned by a DIFFERENT live session. We may reclaim the id - // ONLY if that owner left nothing behind: never materialized a log (cursor - // 0, not materialized) AND has no write-behind buffer still pending. A - // session that appended events but was disposed before its first flush is - // NOT materialized yet but DOES have buffered events — reclaiming then - // would let that stale buffer drain against the new session's state - // (persisting old events under the new id, or dropping the new session's - // seq-0 events). Such an owner, and any materialized owner, is a real - // collision and rejects; only a truly-abandoned (artifact-free) id is - // freed, honoring lazy materialization's "leaves nothing behind" promise. - const ownerBuffer = this.buffers.get(tracked.owner) - if (!tracked.materialized && !ownerBuffer?.length) { - this.states.delete(id) - } else { - throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`) - } - } - - const onDisk = await this.findLog(id, session.header.cwd) - if (onDisk !== undefined) { - // case 2: adopt a LIVE prefix. Do NOT route through loadCore(): loadCore - // crash-repairs open turns as interrupted, which is right for a true load - // after a crash but wrong for HMR while the live Session is still the - // authority and may append the real step/turn end later. - await this.serialize(id, () => this.adoptLiveDiskPrefix(session, seed, onDisk)) - return - } - - // case 4: a genuinely new session. Register its meta (lazy), then persist - // its seed (events present at creation time) once. - const meta: SessionMeta = { ...session.header, updatedAt: Date.now() } - await this.create(meta) - // Bind this state to the live session so a later DIFFERENT session reusing - // the id is detected as a collision (case 1) rather than silently no-opped. - const created = this.states.get(id) - /* v8 ignore next -- create() always sets the state for the id */ - if (created !== undefined) created.owner = session - if (seed.length > 0) { - await this.append(id, seed) - } - } - - private async flush(session: Session): Promise { - // Wait for the session's init (onCreated) to finish so the state/cursor and - // any fork-seed persistence are in place before we drain. Awaiting the same - // promise initFor stored also surfaces an init failure (e.g. an id - // collision) here, where the caller of session/flush observes it. - await this.inits.get(session) - // Serialize the WHOLE drain (read cursor → append → splice) on the - // per-session chain. Two concurrent flushes (e.g. an idle inject()'s - // fire-and-forget flush racing an explicit checkpoint) would otherwise both - // read the same cursor, both compute the same `fresh` slice, and the second - // append would seq-mismatch against the cursor the first already advanced. - await this.serialize(session.header.id, () => this.drain(session)) - } - - /** Drain a session's write buffer to disk. Caller serializes this per id. */ - private async drain(session: Session): Promise { - const buffer = this.buffers.get(session) - if (!buffer?.length) return - // Copy WITHOUT removing: the buffer is the only durable-pending copy of - // these events (session/event does not re-emit). Splicing before the append - // means a failed append (disk error, or a seq mismatch after a dropped bad - // event) permanently loses a completed turn. Drain the buffer only AFTER - // the append commits; events pushed during the await sit past batch.length - // and survive the prefix splice, so a retry/dispose re-drains the rest. - const batch = buffer.slice() - const state = this.states.get(session.header.id) - // Only append events at or beyond the write cursor (a resumed session's - // seed is already on disk; the cursor was set to the loaded length). flush - // awaits the init above, which always sets state, so the `?? 0` fallback is - // a defensive guard that never fires in practice. - /* v8 ignore next -- state is always set by the awaited init before flush */ - const cursor = state?.cursor ?? 0 - const fresh = batch.filter(e => e.seq >= cursor) - // appendCore (NOT the serialized append) — drain already runs inside the - // per-session chain, so re-entering it via append() would deadlock. - if (fresh.length > 0) await this.appendCore(session.header.id, fresh) - buffer.splice(0, batch.length) - } -} - -export default SessionPersistenceJsonl diff --git a/packages/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session-persistence-jsonl/tests/jsonl.spec.ts deleted file mode 100644 index 8257a9853c..0000000000 --- a/packages/session-persistence-jsonl/tests/jsonl.spec.ts +++ /dev/null @@ -1,1290 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { Context } from 'cordis' -import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session' -import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' -import { encodeSegment, logPath, scanLog, sessionDir, sidecarPath } from '../src/format.ts' -import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' - -let root: string -const dirs: string[] = [] - -async function freshRoot(): Promise { - const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-')) - dirs.push(dir) - return dir -} - -afterEach(async () => { - for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) -}) - -// Run the shared backend contract against the real JSONL backend. -runPersistenceContract('jsonl', async () => { - const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-')) - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir }) - return { - persistence: ctx.sessionPersistence, - dispose: async () => { - await fiber.dispose() - await rm(dir, { recursive: true, force: true }) - }, - } -}) - -describe('SessionPersistenceJsonl: format helpers', () => { - it('encodeSegment neutralizes traversal, separators, and absolute paths', () => { - expect(encodeSegment('..')).toBe('~002E~002E') - expect(encodeSegment('.')).toBe('~002E') - expect(encodeSegment('a/b')).toBe('a~002Fb') - expect(encodeSegment('/etc/passwd')).toBe('~002Fetc~002Fpasswd') - expect(encodeSegment('a\u0000b')).toBe('a~0000b') - expect(encodeSegment('plain-ID_1.2')).toBe('plain-ID_1.2') // safe chars pass through - expect(encodeSegment('a~b')).toBe('a~007Eb') // ~ itself is escaped - }) - - it('encodeSegment is injective over UTF-16, incl. lone surrogates', () => { - // Distinct lone surrogates must NOT collide (Buffer.from would normalize - // both to U+FFFD; code-unit escaping keeps them distinct). - const hi = encodeSegment(String.fromCharCode(0xD800)) - const lo = encodeSegment(String.fromCharCode(0xDC00)) - expect(hi).toBe('~D800') - expect(lo).toBe('~DC00') - expect(hi).not.toBe(lo) - // A literal "~002F" input cannot collide with the encoding of "/". - expect(encodeSegment('~002F')).not.toBe(encodeSegment('/')) - }) - - it('encodeSegment rejects an empty id', () => { - expect(() => encodeSegment('')).toThrow(/empty/) - }) -}) - -describe('SessionPersistenceJsonl: durability and crash semantics', () => { - let ctx: Context - beforeEach(async () => { - root = await freshRoot() - ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - }) - afterEach(async () => { await ctx.fiber.dispose() }) - - it('lazy materialization: create() writes no file until the first append', async () => { - const m = meta('lazy', '/work') - await ctx.sessionPersistence.create(m) - // nothing on disk yet - const dir = sessionDir(root, '/work') - await expect(stat(logPath(root, '/work', m.id))).rejects.toThrow() - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) - - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // now materialized - expect((await stat(logPath(root, '/work', m.id))).isFile()).toBe(true) - expect(await ctx.sessionPersistence.has(m.id)).toBe(true) - void dir - }) - - it('round-trip is byte-identical (incl. assistant/chunk verbatim)', async () => { - const m = meta('chunks') - const log: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, - { type: 'assistant/chunk', seq: 2, time: 3, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'he' } } }, - { type: 'assistant/chunk', seq: 3, time: 4, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'llo' } } }, - { type: 'assistant/message', seq: 4, time: 5, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] } }, - { type: 'step/end', seq: 5, time: 6, data: { turn: 1, step: 1 } }, - { type: 'turn/end', seq: 6, time: 7, data: { turn: 1, reason: { kind: 'completed' } } }, - ] - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, log) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events).toEqual(log) // chunks preserved, contiguous seqs - }) - - it('crash recovery: load preserves the interrupted turn and closes it with a synthetic turn/end {interrupted}', async () => { - const m = meta('crash', '/proj') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5, turn/end at 5 - - // Simulate a crash mid-second-turn: append raw lines that are NOT closed by - // a turn/end (turn/start + step/start are fully written), plus a final - // partial line with no newline (a torn fragment never fully flushed). - const path = logPath(root, '/proj', m.id) - await writeFile(path, [ - JSON.stringify({ type: 'turn/start', seq: 6, time: 8, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }), - JSON.stringify({ type: 'step/start', seq: 7, time: 9, data: { turn: 2, step: 1 } }), - '{"type":"assistant/chunk","seq":8,"ti', // truncated partial line (no newline) - ].join('\n'), { flag: 'a' }) - - // load PRESERVES the interrupted turn's real events (turn/start 6, step/start - // 7) — a turn can be huge, so they must not be truncated — and durably closes - // the orphaned turn with synthetic step/end (8) + turn/end {interrupted} (9). - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) - const last = loaded.events.at(-1)! - expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) - const stepEnd = loaded.events[8]! - expect(stepEnd.type).toBe('step/end') - // the torn seq-8 chunk fragment did not survive - expect(loaded.events.some(e => e.type === 'assistant/chunk' && e.seq === 8)).toBe(false) - - // The next append continues at seq 10 (the balanced length). - const turn3 = [ - { type: 'turn/start', seq: 10, time: 11, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 11, time: 12, data: { turn: 3, reason: { kind: 'completed' } } }, - ] as SessionEvent[] - await ctx.sessionPersistence.append(m.id, turn3) - const reloaded = await ctx.sessionPersistence.load(m.id) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) - }) - - it('committed events are never rewritten: only the crash tail is repaired', async () => { - const m = meta('append-only') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const before = await readFile(logPath(root, undefined, m.id), 'utf8') - const committedPrefix = before // the whole committed log - - // A crash tail then a repair-append. - await writeFile(logPath(root, undefined, m.id), '\n{"partial', { flag: 'a' }) - await ctx.sessionPersistence.load(m.id) - await ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[]) - const after = await readFile(logPath(root, undefined, m.id), 'utf8') - // the committed prefix is byte-for-byte intact at the head of the file - expect(after.startsWith(committedPrefix)).toBe(true) - }) - - it('a failed appendLines truncates partial bytes so a retry has no seq gap', async () => { - const m = meta('truncate-retry') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) // materialized, seqs 0..5 - const sizeBefore = (await stat(logPath(root, undefined, m.id))).size - - // Force the NEXT fsync (inside appendLines) to fail once, AFTER writeFile - // has already put bytes on disk — simulating an ENOSPC/fsync error - // mid-append. The recovery truncate() also fsyncs, so allow that one. - const handle = await (await import('node:fs/promises')).open(logPath(root, undefined, m.id), 'r') - const proto = Object.getPrototypeOf(handle) as { sync: () => Promise } - await handle.close() - const realSync = proto.sync - let failed = false - const spy = vi.spyOn(proto, 'sync').mockImplementation(async function (this: unknown) { - if (!failed) { failed = true; throw new Error('simulated fsync ENOSPC') } - return realSync.call(this) - }) - - const turn2 = [ - { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[] - // The append rejects, but the partial bytes are truncated back: the file is - // its pre-append size and the cursor is unchanged. - await expect(ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/ENOSPC/) - expect((await stat(logPath(root, undefined, m.id))).size).toBe(sizeBefore) - spy.mockRestore() - - // The retry now succeeds with NO seq gap — the log is contiguous 0..7. - await ctx.sessionPersistence.append(m.id, turn2) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - }) - - it('append snapshots its batch: mutating the caller array after the call is ignored', async () => { - const m = meta('snapshot') - await ctx.sessionPersistence.create(m) - const events = oneTurnLog() // seqs 0..5 - const p = ctx.sessionPersistence.append(m.id, events) - // Mutate the caller's array immediately after calling append (before the - // queued op runs). The backend must persist the snapshot taken at call time, - // not the mutated array. - events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }) - await p - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6 - }) - - it('append deep-snapshots event objects: mutating an event after the call is ignored', async () => { - const m = meta('deep-snapshot') - await ctx.sessionPersistence.create(m) - const events = oneTurnLog() - const userMsg = events[1] // the user/message event - const p = ctx.sessionPersistence.append(m.id, events) - // Mutate an event OBJECT (not just the array) after calling append. The deep - // snapshot taken at call time must shield the persisted data. - if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }] - await p - const loaded = await ctx.sessionPersistence.load(m.id) - const persisted = JSON.stringify(loaded.events) - expect(persisted).toContain('hi') // original content - expect(persisted).not.toContain('MUTATED') - }) - - it('load returns a meta copy: mutating it does not corrupt backend pathing', async () => { - const m = meta('meta-copy', '/proj') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - // A consumer mutates the returned meta's cwd. The backend's stored pathing - // metadata must be unaffected, so a later append still finds the right log. - loaded.meta.cwd = '/evil' - await ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[]) - // The append landed in the ORIGINAL /proj log, not beside an /evil path. - const reloaded = await ctx.sessionPersistence.load(m.id) - expect(reloaded.meta.cwd).toBe('/proj') - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - }) - - it('rejects an unknown format version on load', async () => { - const m = meta('v2') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // Corrupt the header version on disk. - const path = logPath(root, undefined, m.id) - const lines = (await readFile(path, 'utf8')).split('\n') - const header = JSON.parse(lines[0]!) as { version: number } - header.version = 2 - lines[0] = JSON.stringify(header) - await writeFile(path, lines.join('\n')) - // Fresh backend (no in-memory state) → must reject on load. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await expect(ctx2.sessionPersistence.load(m.id)).rejects.toThrow(/version/) - await ctx2.fiber.dispose() - }) - - it('rejects a re-append of an already-stored seq', async () => { - const m = meta('reappend') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toThrow(/seq mismatch/) - }) - - it('path-traversal session ids are neutralized (no escape from root)', async () => { - const evil = SessionId('../../etc/pwn') - const m = { version: 1, id: evil, createdAt: 1, updatedAt: 1 } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(evil, oneTurnLog()) - // The file lives UNDER root, not at ../../etc. - const all: string[] = [] - async function walk(dir: string): Promise { - for (const e of await readdir(dir, { withFileTypes: true })) { - const p = join(dir, e.name) - if (e.isDirectory()) await walk(p) - else all.push(p) - } - } - await walk(root) - expect(all.length).toBeGreaterThan(0) - expect(all.every(p => p.startsWith(root))).toBe(true) - }) -}) - -describe('SessionPersistenceJsonl: write path (session/event → flush)', () => { - it('persists a live session driven through the store, surviving reload', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const session = ctx.sessions.create('live', { meta: { cwd: '/w' } }) - for (const e of oneTurnLog()) session.append(e.type, e.data) - await ctx.parallel('session/flush', session) - - const loaded = await ctx.sessionPersistence.load(SessionId('live')) - expect(loaded.events).toHaveLength(6) - expect(loaded.meta.cwd).toBe('/w') - await ctx.fiber.dispose() - }) - - it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const session = ctx.sessions.create('mutate') - const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } }) - // Mutate the live event object AFTER it was buffered. - ;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED' - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - const loaded = await ctx.sessionPersistence.load(SessionId('mutate')) - const first = loaded.events[0] - expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original') - await ctx.fiber.dispose() - }) - - it('fork: a seeded new session persists its seed once', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const seed = oneTurnLog() - // A fork: a brand-new id whose seed came from elsewhere. - const forked = ctx.sessions.create('forked', { seed }) - // onCreated persisted the seed asynchronously; wait a tick. - await new Promise(r => setTimeout(r, 10)) - const loaded = await ctx.sessionPersistence.load(SessionId('forked')) - expect(loaded.events).toEqual(seed) - // A flush with no NEW events must not double-write. - await ctx.parallel('session/flush', forked) - const reloaded = await ctx.sessionPersistence.load(SessionId('forked')) - expect(reloaded.events).toEqual(seed) - await ctx.fiber.dispose() - }) - - it('concurrent sessions do not cross buffers', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - - const a = ctx.sessions.create('sa') - const b = ctx.sessions.create('sb') - a.append('user/message', { content: [{ type: 'text', text: 'A' }], source: { kind: 'user' } }) - b.append('user/message', { content: [{ type: 'text', text: 'B' }], source: { kind: 'user' } }) - a.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - b.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', a) - await ctx.parallel('session/flush', b) - - const la = await ctx.sessionPersistence.load(SessionId('sa')) - const lb = await ctx.sessionPersistence.load(SessionId('sb')) - expect(JSON.stringify(la.events)).toContain('"A"') - expect(JSON.stringify(la.events)).not.toContain('"B"') - expect(JSON.stringify(lb.events)).toContain('"B"') - expect(JSON.stringify(lb.events)).not.toContain('"A"') - await ctx.fiber.dispose() - }) - - it('HMR: applying the plugin seeds existing live sessions', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - // A session exists BEFORE the persistence plugin is applied. - const session = ctx.sessions.create('pre-existing') - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - - await ctx.plugin(SessionPersistenceJsonl, { root }) - // The plugin seeded it on apply; a subsequent flush persists its events. - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing')) - expect(loaded.events.length).toBeGreaterThanOrEqual(2) - await ctx.fiber.dispose() - }) - - it('HMR: dispose drains remaining buffers', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - const fiber = await ctx.plugin(SessionPersistenceJsonl, { root }) - const sessFiber = await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('drain') - }, { inject: ['sessions'] })) - session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - // No explicit flush — dispose must drain. - await fiber.dispose() - await sessFiber.dispose() - - // A fresh backend reads what the disposed one drained. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const loaded = await ctx2.sessionPersistence.load(SessionId('drain')) - expect(loaded.events.length).toBeGreaterThanOrEqual(2) - await ctx2.fiber.dispose() - }) - - it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - // The session lives in its OWN fiber so it survives the backend reload. - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-adopt') - }, { inject: ['sessions'] })) - - // Backend instance 1 materializes the session on disk. - const backend1 = await ctx.plugin(SessionPersistenceJsonl, { root }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Hot-reload the backend: dispose instance 1, plug in instance 2 over the - // SAME root while the session stays live. Instance 2 has an empty states - // map but the log is on disk — it must ADOPT (not reject) so flush keeps - // working. A second turn appended after reload then persists. - await backend1.dispose() - await ctx.plugin(SessionPersistenceJsonl, { root }) - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow() - - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt')) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) - - it('HMR: adoption persists the live SUFFIX that was ahead of the on-disk prefix', async () => { - root = await freshRoot() - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-suffix') - }, { inject: ['sessions'] })) - - // Instance 1 flushes turn 1 to disk. - const backend1 = await ctx.plugin(SessionPersistenceJsonl, { root }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT - // flushing turn 2. Turn 2 is now ONLY in the live session's events; the new - // backend never buffered it via session/event. - await backend1.dispose() - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - - // Instance 2 adopts the on-disk prefix (turn 1) and MUST also persist the - // live suffix (turn 2) carried in the session's events — otherwise turn 2 is - // lost and a later flush would mismatch. - await ctx.plugin(SessionPersistenceJsonl, { root }) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3]) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) -}) - - -describe('SessionPersistenceJsonl: scanLog unit', () => { - it('rejects a header-less / empty log', () => { - expect(() => scanLog(Buffer.from(''))).toThrow() - }) - - it('rejects a corrupt header line', () => { - expect(() => scanLog(Buffer.from('not json\n'))).toThrow(/header/) - }) - - it('rejects a non-session first line', () => { - expect(() => scanLog(Buffer.from('{"type":"event"}\n'))).toThrow(/session header/) - }) - - it('a seq gap after the last turn/end bounds the preserved tail (torn fragment tolerated)', () => { - const log = [ - JSON.stringify({ type: 'session', version: 1, id: 'g', createdAt: 1 }), - JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), - JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1 - ].join('\n') + '\n' - // No committed turn/end, so the gap is a tolerated crash boundary: scanLog - // PRESERVES the contiguous prefix (turn/start seq 0) — real interrupted-turn - // work, not discarded — and stops at the gap. The orphaned open turn is - // closed by loadCore's synthetic turn/end, not here. - expect(scanLog(Buffer.from(log)).events.map(e => e.seq)).toEqual([0]) - }) - - it('rejects a seq gap BEFORE a later committed turn/end (committed data damaged)', () => { - const log = [ - JSON.stringify({ type: 'session', version: 1, id: 'g2', createdAt: 1 }), - JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), - JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1 - JSON.stringify({ type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }), - ].join('\n') + '\n' - // A turn/end exists, so the prefix up to it is committed — but it has a hole. - // Truncating it would silently drop committed data → unloadable. - expect(() => scanLog(Buffer.from(log))).toThrow(/seq gap in committed region/) - }) - - it('rejects a corrupt line BEFORE a later committed turn/end (committed data damaged)', () => { - const log = [ - JSON.stringify({ type: 'session', version: 1, id: 'c', createdAt: 1 }), - '{not json', // corrupt, sits in the committed region (a turn/end follows) - JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }), - ].join('\n') + '\n' - expect(() => scanLog(Buffer.from(log))).toThrow(/unparsable committed event/) - }) - - it('a header-only log (no event lines at all) preserves nothing — committedBytes is the header', () => { - const log = JSON.stringify({ type: 'session', version: 1, id: 'h0', createdAt: 1 }) + '\n' - const scanned = scanLog(Buffer.from(log)) - expect(scanned.events).toEqual([]) - // committedBytes falls back to the header line's end (no preserved events). - expect(scanned.committedBytes).toBe(Buffer.byteLength(log, 'utf8')) - }) - - it('a corrupt line after the last turn/end bounds the preserved tail', () => { - const log = [ - JSON.stringify({ type: 'session', version: 1, id: 'c2', createdAt: 1 }), - JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), - '{not json', // corrupt crash fragment, no turn/end committed - ].join('\n') + '\n' - // The contiguous prefix (turn/start seq 0) is preserved; the corrupt - // fragment after it is the tolerated crash boundary. - expect(scanLog(Buffer.from(log)).events.map(e => e.seq)).toEqual([0]) - }) - - it('tolerates a seq gap AFTER a turn/end (uncommitted tail)', () => { - const log = [ - JSON.stringify({ type: 'session', version: 1, id: 't', createdAt: 1 }), - JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), - JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }), - JSON.stringify({ type: 'step/start', seq: 9, time: 3, data: { turn: 2, step: 1 } }), // gap in uncommitted tail - ].join('\n') + '\n' - const { events } = scanLog(Buffer.from(log)) - expect(events.map(e => e.seq)).toEqual([0, 1]) // tail dropped - }) -}) - -describe('SessionPersistenceJsonl: edge cases', () => { - let ctx: Context - beforeEach(async () => { - root = await freshRoot() - ctx = new Context() - await ctx.plugin(SessionStore) - await ctx.plugin(SessionPersistenceJsonl, { root }) - }) - afterEach(async () => { await ctx.fiber.dispose() }) - - it('load rejects a missing session', async () => { - await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/) - }) - - it('append of an empty batch is a no-op', async () => { - const m = meta('empty-batch') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, []) - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) - }) - - it('append resolves even when the best-effort sidecar write fails (log is the transaction)', async () => { - const m = meta('sidecar-fail') - await ctx.sessionPersistence.create(m) - // Force the sidecar write to reject AFTER the durable log append commits. - // The append must still resolve and advance the cursor — a failed sidecar - // is recoverable metadata and must never desync the log (which would let a - // retry duplicate seqs). This exercises the `.catch()` on touchSummary. - const backend = ctx.sessionPersistence as unknown as { writeSidecar: (state: unknown) => Promise } - const original = backend.writeSidecar.bind(backend) - backend.writeSidecar = () => Promise.reject(new Error('disk full')) - await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).resolves.toBeUndefined() - backend.writeSidecar = original - // The durable log landed in full despite the sidecar failure. - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) - }) - - it('append rejects non-JSON-serializable undefined-producing data', async () => { - const m = meta('undef') - await ctx.sessionPersistence.create(m) - // A value whose JSON.stringify yields undefined (a bare function as data). - const bad = [{ type: 'user/message', seq: 0, time: 1, data: (() => 0) as unknown }] as unknown as SessionEvent[] - await expect(ctx.sessionPersistence.append(m.id, bad)).rejects.toThrow(/non-JSON-serializable/) - }) - - it('delete of a non-existent session is a no-op', async () => { - await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined() - }) - - it('update adopts a session that exists only on disk', async () => { - const m = meta('disk-only') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // A fresh backend has no in-memory state → update must adopt from disk. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await ctx2.sessionPersistence.update(m.id, { title: 'adopted' }) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('adopted') - await ctx2.fiber.dispose() - }) - - it('a failed update does not become durable via a later append', async () => { - const m = meta('update-fail') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // Force the sidecar write to fail for the update. - const backend = ctx.sessionPersistence as unknown as { writeSidecar: (meta: unknown) => Promise } - const original = backend.writeSidecar.bind(backend) - backend.writeSidecar = () => Promise.reject(new Error('disk full')) - await expect(ctx.sessionPersistence.update(m.id, { title: 'rejected-title' })).rejects.toThrow(/disk full/) - backend.writeSidecar = original - // A later successful append's touchSummary must NOT persist the rejected - // title (it was never committed to in-memory state). - await ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[]) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBeUndefined() - }) - - it('update before the first append keeps summary in memory and writes no orphan sidecar', async () => { - const m = meta('lazy-update', '/a') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.update(m.id, { title: 'secret', firstPrompt: 'sensitive' }) - const sidecar = sidecarPath(root, '/a', m.id) - await expect(stat(sidecar)).rejects.toThrow() - await expect(stat(logPath(root, '/a', m.id))).rejects.toThrow() - - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('secret') - expect(loaded.meta.firstPrompt).toBe('sensitive') - expect((await stat(sidecar)).isFile()).toBe(true) - }) - - it('a lazy update leaves no sidecar that can leak into a future same-id session after restart', async () => { - await ctx.sessionPersistence.create(meta('restart-lazy', '/a')) - await ctx.sessionPersistence.update(SessionId('restart-lazy'), { title: 'secret' }) - await expect(stat(sidecarPath(root, '/a', SessionId('restart-lazy')))).rejects.toThrow() - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const m2 = meta('restart-lazy', '/a') - await ctx2.sessionPersistence.create(m2) - await ctx2.sessionPersistence.append(m2.id, oneTurnLog()) - const loaded = await ctx2.sessionPersistence.load(m2.id) - expect(loaded.meta.title).toBeUndefined() - await ctx2.fiber.dispose() - }) - - it('delete removes a materialized cwd-bucket sidecar after a restart', async () => { - await ctx.sessionPersistence.create(meta('restart-del', '/a')) - await ctx.sessionPersistence.append(SessionId('restart-del'), oneTurnLog()) - await ctx.sessionPersistence.update(SessionId('restart-del'), { title: 'secret' }) - const sidecar = sidecarPath(root, '/a', SessionId('restart-del')) - expect((await stat(sidecar)).isFile()).toBe(true) - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await ctx2.sessionPersistence.delete(SessionId('restart-del')) - await expect(stat(sidecar)).rejects.toThrow() - await expect(stat(logPath(root, '/a', SessionId('restart-del')))).rejects.toThrow() - await ctx2.fiber.dispose() - }) - - it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { - // A live session is created then disposed BEFORE its first append: cursor 0, - // never materialized, nothing on disk. A new live session reusing the id - // must reclaim it (lazy materialization promises no lingering artifact), - // not wedge on an "already bound" collision until restart. - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let firstSession!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - firstSession = inner.sessions.create('abandoned', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(firstSession) // let the lazy create register the state - await firstFiber.dispose() // disposed before any append → never materialized - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('abandoned', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - // The new session claims the id without error and can persist a turn. - await expect(backend.inits.get(reuse)).resolves.toBeUndefined() - reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', reuse) - const loaded = await ctx.sessionPersistence.load(SessionId('abandoned')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - }) - - it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => { - // A session that appended events but was disposed BEFORE its first flush is - // not materialized yet but still holds a write-behind buffer. Reusing the id - // must be rejected (not reclaimed), or the stale buffer would drain against - // the new session — persisting old events under the new id or dropping the - // new session's seq-0 events. - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let first!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - first = inner.sessions.create('buffered', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(first) - // Append a turn but do NOT flush — events sit in the write-behind buffer. - first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - first.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await firstFiber.dispose() // disposed before flush; not materialized, buffer pending - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('buffered', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(reuse)).rejects.toThrow(/already bound to a different live session/) - }) - - it('create snapshots its meta: mutating the caller object after the call is ignored', async () => { - const m = meta('create-snap', '/orig') - const p = ctx.sessionPersistence.create(m) - // Mutate the caller's meta object immediately after calling create. - m.cwd = '/mutated' - await p - await ctx.sessionPersistence.append(SessionId('create-snap'), oneTurnLog()) - // The log materialized under the ORIGINAL cwd, not the mutated one. - expect((await stat(logPath(root, '/orig', SessionId('create-snap')))).isFile()).toBe(true) - await expect(stat(logPath(root, '/mutated', SessionId('create-snap')))).rejects.toThrow() - }) - - it('list discovers sessions across multiple cwd buckets', async () => { - await ctx.sessionPersistence.create(meta('p1', '/projA')) - await ctx.sessionPersistence.append(SessionId('p1'), oneTurnLog()) - await ctx.sessionPersistence.create(meta('p2', '/projB')) - await ctx.sessionPersistence.append(SessionId('p2'), oneTurnLog()) - await ctx.sessionPersistence.create(meta('p3')) // no cwd → _no-cwd bucket - await ctx.sessionPersistence.append(SessionId('p3'), oneTurnLog()) - - const ids = (await ctx.sessionPersistence.list()).map(x => x.id).sort() - expect(ids).toEqual(['p1', 'p2', 'p3']) - }) - - it('list tolerates one corrupt sidecar and still returns other sessions', async () => { - const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) - const bad = meta('bad-list-summary', '/proj') - await ctx.sessionPersistence.create(bad) - await ctx.sessionPersistence.append(bad.id, oneTurnLog()) - await ctx.sessionPersistence.update(bad.id, { title: 'hidden by corrupt sidecar' }) - await writeFile(sidecarPath(root, '/proj', bad.id), '{not json') - const good = meta('good-list-summary', '/proj') - await ctx.sessionPersistence.create(good) - await ctx.sessionPersistence.append(good.id, oneTurnLog()) - await ctx.sessionPersistence.update(good.id, { title: 'visible' }) - - const listed = await ctx.sessionPersistence.list() - - const badListed = listed.find(m => m.id === bad.id) - expect(badListed).toMatchObject({ id: bad.id }) - expect(badListed).not.toHaveProperty('title') - expect(listed.find(m => m.id === good.id)).toMatchObject({ id: good.id, title: 'visible' }) - expect(warn).toHaveBeenCalledWith(expect.stringContaining('bad-list-summary')) - }) - - it('list on an empty root returns nothing', async () => { - expect(await ctx.sessionPersistence.list()).toEqual([]) - }) - - it('list skips empty and non-header .jsonl files (metadata-only read)', async () => { - // A real session… - await ctx.sessionPersistence.create(meta('real', '/p')) - await ctx.sessionPersistence.append(SessionId('real'), oneTurnLog()) - // …alongside two junk files in the _no-cwd bucket: an EMPTY file (readFirstLine - // returns undefined) and a file whose first line is not a session header - // (parseHeaderMeta returns undefined). Both are skipped, not listed. - const bucket = join(root, '_no-cwd') - await mkdir(bucket, { recursive: true }) - await writeFile(join(bucket, 'empty.jsonl'), '') - await writeFile(join(bucket, 'notheader.jsonl'), '{"type":"turn/start"}\n') - await writeFile(join(bucket, 'badjson.jsonl'), 'not json at all\n') - - const ids = (await ctx.sessionPersistence.list()).map(x => x.id).sort() - expect(ids).toEqual(['real']) - }) - - it('list reads a header line longer than the 8KB read chunk', async () => { - // readFirstLine accumulates across reads when the first line exceeds its - // buffer. Plant a valid header whose line is > 8192 bytes (a long extra - // field is tolerated by the header type guard) and confirm list() reads it. - const bucket = join(root, '_no-cwd') - await mkdir(bucket, { recursive: true }) - const bigHeader = JSON.stringify({ type: 'session', version: 1, id: 'big', createdAt: 1, pad: 'x'.repeat(9000) }) - await writeFile(join(bucket, 'big.jsonl'), bigHeader + '\n') - const ids = (await ctx.sessionPersistence.list()).map(x => x.id) - expect(ids).toContain('big') - }) - - it('has() finds a session on disk under an unknown cwd (cross-bucket scan)', async () => { - const m = meta('scan-me', '/somewhere') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // A fresh backend with no in-memory state → has() must scan disk buckets. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - expect(await ctx2.sessionPersistence.has(m.id)).toBe(true) - expect(await ctx2.sessionPersistence.has(SessionId('absent'))).toBe(false) - await ctx2.fiber.dispose() - }) - - it('resume/adopt: a live session whose id is already on disk continues from the stored length', async () => { - // First lifecycle: persist a session through the store. - const s1 = ctx.sessions.create('resumed', { meta: { cwd: '/r' } }) - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx.parallel('session/flush', s1) - - // Second lifecycle: a NEW backend + a session re-created with the same id - // and SEEDED with the loaded events (the resume path). onCreated must adopt - // the on-disk log (not re-persist the seed), and a new turn appends at seq 6. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const loaded = await ctx2.sessionPersistence.load(SessionId('resumed')) - const s2 = ctx2.sessions.create('resumed', { seed: loaded.events, meta: { cwd: '/r' } }) - await new Promise(r => setTimeout(r, 10)) // let onCreated adopt - // Append a fresh turn through the live session. - s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await ctx2.parallel('session/flush', s2) - - const reloaded = await ctx2.sessionPersistence.load(SessionId('resumed')) - // 6 original + 2 new, contiguous, no duplicated seed. - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await ctx2.fiber.dispose() - }) - - it('HMR adoption does not crash-repair an active open turn as interrupted', async () => { - const dir = await freshRoot() - const hmr = new Context() - await hmr.plugin(SessionStore) - const first = await hmr.plugin(SessionPersistenceJsonl, { root: dir }) - const session = hmr.sessions.create('hmr-open', { meta: { cwd: '/hmr' } }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('step/start', { turn: 1, step: 1 }) - await hmr.parallel('session/flush', session) - - await first.dispose() - await appendFile(logPath(dir, '/hmr', SessionId('hmr-open')), '{"torn":') - const second = await hmr.plugin(SessionPersistenceJsonl, { root: dir }) - session.append('step/end', { turn: 1, step: 1 }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await hmr.parallel('session/flush', session) - - const loaded = await hmr.sessionPersistence.load(SessionId('hmr-open')) - expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end']) - expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) - await second.dispose() - await hmr.fiber.dispose() - }) - - it('a NEW live session whose id collides with an on-disk log is rejected, not silently adopted', async () => { - // Persist a session on disk. - const s1 = ctx.sessions.create('collide', { meta: { cwd: '/a' } }) - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx.parallel('session/flush', s1) - const before = await readFile(logPath(root, '/a', SessionId('collide')), 'utf8') - - // A FRESH backend + a NEW live session with the same id but NO explicit - // load/resume. onCreated must NOT adopt-from-disk (resume is explicit); it - // treats this as a new session and create() rejects because a log already - // exists on disk. The rejection surfaces via the init promise (flush awaits - // it); the on-disk committed log is left byte-for-byte intact. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const backend = ctx2.sessionPersistence as unknown as { inits: Map> } - const s2 = ctx2.sessions.create('collide', { meta: { cwd: '/a' } }) - // The init for the new live session rejects (observed via the per-session - // init map and, in production, via flush which awaits the same promise). - await expect(backend.inits.get(s2)).rejects.toThrow(/already has a persisted log on disk/) - // The committed log is untouched (no clobber). - expect(await readFile(logPath(root, '/a', SessionId('collide')), 'utf8')).toBe(before) - await ctx2.fiber.dispose() - }) - - it('a DIFFERENT live session object reusing a disposed id gets its own init (no stale cache)', async () => { - // Session A materializes a log under id "reuse". - const sessFiberA = await ctx.plugin(Object.assign((inner: Context) => { - const a = inner.sessions.create('reuse', { meta: { cwd: '/a' } }) - for (const e of oneTurnLog()) a.append(e.type, e.data) - }, { inject: ['sessions'] })) - // Drain A, then dispose ITS fiber (the live session A is gone) while the - // backend stays loaded. - for (const s of ctx.sessions.list()) await ctx.parallel('session/flush', s) - await sessFiberA.dispose() - - // A NEW live Session object reuses id "reuse". The init cache is keyed by - // the Session OBJECT, so this gets its OWN onCreated (not A's stale promise) - // — which detects the on-disk collision and rejects, rather than silently - // appending the new session's events onto A's log under a stale cursor. - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let b!: Session - await ctx.plugin(Object.assign((inner: Context) => { - b = inner.sessions.create('reuse', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(b)).rejects.toThrow(/already bound to a different live session|already has a persisted log on disk/) - }) - - it('a live session claims cursor-0 ownerless state created via the public API', async () => { - // create() registers ownerless state with cursor 0 (lazy, nothing persisted - // yet). A live session with that id then arrives and claims it without a - // prefix check (cursor 0 matches trivially), persisting its seed. - await ctx.sessionPersistence.create(meta('lazy-claim', '/a')) - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let live!: Session - await ctx.plugin(Object.assign((inner: Context) => { - live = inner.sessions.create('lazy-claim', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(live)).resolves.toBeUndefined() - live.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - live.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', live) - const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - }) - - it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => { - // Materialize a log, then load() it into the backend's state WITHOUT a live - // session — leaving state.owner undefined and cursor at the persisted length - // (the public preview path). - await ctx.sessionPersistence.create(meta('preview', '/a')) - await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('preview')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A FRESH (empty-seed) live session reusing that id must be rejected: its - // seq 0..cursor-1 events would otherwise be filtered as already-persisted - // and its conversation grafted onto the old log. - let fresh!: Session - await ctx.plugin(Object.assign((inner: Context) => { - fresh = inner.sessions.create('preview', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(fresh)).rejects.toThrow(/do not match this live session|already has a persisted log/) - }) - - it('a session whose seed matches the loaded prefix claims ownerless state', async () => { - // Materialize a log and load it (ownerless state, cursor = 6). - await ctx.sessionPersistence.create(meta('match', '/a')) - await ctx.sessionPersistence.append(SessionId('match'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('match')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A live session SEEDED with the persisted log legitimately continues it — - // its seed reproduces the loaded prefix, so it claims the ownerless state. - let cont!: Session - await ctx.plugin(Object.assign((inner: Context) => { - cont = inner.sessions.create('match', { seed: oneTurnLog(), meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(cont)).resolves.toBeUndefined() - }) - - it('claiming ownerless state persists the seed suffix beyond the prefix', async () => { - // Materialize a one-turn log and load it (ownerless state, cursor = 6). - await ctx.sessionPersistence.create(meta('suffix-claim', '/a')) - await ctx.sessionPersistence.append(SessionId('suffix-claim'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('suffix-claim')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A live session seeded with the prefix PLUS a second turn (seqs 6,7). The - // suffix (constructor seed, never emits session/event) must be persisted on - // claim, not lost. - const seed = [ - ...oneTurnLog(), - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[] - let cont!: Session - await ctx.plugin(Object.assign((inner: Context) => { - cont = inner.sessions.create('suffix-claim', { seed, meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(cont) - const loaded = await ctx.sessionPersistence.load(SessionId('suffix-claim')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - }) - - it('claiming cursor-0 ownerless state persists the whole constructor seed', async () => { - // create() registers ownerless state with cursor 0 (lazy, nothing on disk). - await ctx.sessionPersistence.create(meta('lazy-seed', '/a')) - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A live session seeded with a full turn claims it; the whole seed (cursor - // is 0) must be persisted. - let cont!: Session - await ctx.plugin(Object.assign((inner: Context) => { - cont = inner.sessions.create('lazy-seed', { seed: oneTurnLog(), meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await backend.inits.get(cont) - const loaded = await ctx.sessionPersistence.load(SessionId('lazy-seed')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) - }) - - it('a seed with matching seq/type/time but DIFFERENT data is rejected (deep prefix compare)', async () => { - // Materialize and load (ownerless, cursor = 6). - await ctx.sessionPersistence.create(meta('divergent', '/a')) - await ctx.sessionPersistence.append(SessionId('divergent'), oneTurnLog()) - await ctx.sessionPersistence.load(SessionId('divergent')) - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - // A seed that keeps every seq/type/time but mutates a payload must NOT be - // accepted as "the same session" — otherwise drain filters those seqs as - // already persisted and the divergent payload is silently lost. - const tampered = oneTurnLog() - const userMsg = tampered[1] - if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'DIFFERENT' }] - let bad!: Session - await ctx.plugin(Object.assign((inner: Context) => { - bad = inner.sessions.create('divergent', { seed: tampered, meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(bad)).rejects.toThrow(/do not match this live session|already has a persisted log/) - }) - - it('a second live session reusing a bound id is rejected', async () => { - // A live session materializes and owns the id. - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - const a = inner.sessions.create('bound', { meta: { cwd: '/a' } }) - a.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - a.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - }, { inject: ['sessions'] })) - for (const s of ctx.sessions.list()) await ctx.parallel('session/flush', s) - await firstFiber.dispose() - - const backend = ctx.sessionPersistence as unknown as { inits: Map> } - let second!: Session - await ctx.plugin(Object.assign((inner: Context) => { - second = inner.sessions.create('bound', { meta: { cwd: '/a' } }) - }, { inject: ['sessions'] })) - await expect(backend.inits.get(second)) - .rejects.toThrow(/already bound to a different live session|already has a persisted log|do not match/) - }) - - it('round-trips a header with parentSession (fork lineage)', async () => { - const m: SessionMeta = { version: 1, id: SessionId('forked-child'), createdAt: 1, updatedAt: 1, parentSession: SessionId('the-parent') } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.parentSession).toBe('the-parent') - }) - - it('loads a log that has no sidecar (default summary)', async () => { - // Hand-write a valid log WITHOUT a sidecar, then load it. - const dir = sessionDir(root, undefined) - await (await import('node:fs/promises')).mkdir(dir, { recursive: true }) - const header = JSON.stringify({ type: 'session', version: 1, id: 'no-sidecar', createdAt: 5 }) - const body = oneTurnLog().map(e => JSON.stringify(e)).join('\n') - await writeFile(logPath(root, undefined, SessionId('no-sidecar')), header + '\n' + body + '\n') - const loaded = await ctx.sessionPersistence.load(SessionId('no-sidecar')) - expect(loaded.events).toHaveLength(6) - expect(loaded.meta.title).toBeUndefined() // no sidecar → no title - // With no sidecar, updatedAt falls back to the header createdAt (5), NOT 0 - // — reporting an active session as updated at the Unix epoch would be wrong. - expect(loaded.meta.updatedAt).toBe(5) - }) - - it('load rejects a corrupt sidecar instead of treating it as absent', async () => { - const m = meta('bad-sidecar') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - await writeFile(sidecarPath(root, undefined, m.id), '{not json') - await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow() - }) - - it('list returns nothing when the root directory does not exist', async () => { - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root: join(root, 'does-not-exist-yet') }) - expect(await ctx2.sessionPersistence.list()).toEqual([]) - await ctx2.fiber.dispose() - }) - - it('list surfaces a non-ENOENT root error (ENOTDIR) instead of reporting no sessions', async () => { - // A durable backend must NOT collapse a storage fault to "no sessions". Point - // the root at a regular FILE: readdir then fails with ENOTDIR, which must - // propagate rather than be swallowed as an empty listing. - const filePath = join(root, 'not-a-dir') - await writeFile(filePath, 'x') - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root: filePath }) - await expect(ctx2.sessionPersistence.list()).rejects.toThrow(/ENOTDIR/) - await ctx2.fiber.dispose() - }) - - it('exists() surfaces a non-ENOENT lookup error (ENOTDIR) instead of reporting absent', async () => { - // Same contract on the existence path: a non-ENOENT error from the per-id - // open() must surface, not be collapsed to "not found" (which would let a - // collision check proceed under a false absence assumption). A LAZY session - // (created, never appended) keeps its cwd in state, so has() reaches - // findLog(id, cwd) → exists(logPath). Make that cwd's bucket DIRECTORY a - // regular file: open()ing `bucket/.jsonl` under it then fails ENOTDIR. - const cwd = '/x' - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await ctx2.sessionPersistence.create(meta('exists-fault', cwd)) // lazy: no bucket yet - await writeFile(sessionDir(root, cwd), 'x') // bucket path is now a FILE - await expect(ctx2.sessionPersistence.has(SessionId('exists-fault'))).rejects.toThrow(/ENOTDIR/) - await ctx2.fiber.dispose() - }) - - it('append() to a disk-only session adopts it and repairs a crash tail', async () => { - // Persist a session, then corrupt its tail, all through ONE backend. - const m = meta('disk-append', '/d') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - await writeFile(logPath(root, '/d', m.id), '\n{"partial crash', { flag: 'a' }) - - // A FRESH backend with no in-memory state: append directly (no prior load) - // → append must adopt from disk, and the adopt's load schedules a repair - // that the same append then performs before writing. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await ctx2.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[]) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await ctx2.fiber.dispose() - }) - - it('a header-only log (open turn, no turn/end) preserves the open turn on load and closes it', async () => { - // A session whose only durable content is an unclosed first turn. scanLog - // preserves the turn/start; loadCore closes it with a synthetic - // turn/end {interrupted} so the returned log is balanced. - const m = meta('open-turn', '/h') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - ] as SessionEvent[]) - const { events } = await ctx.sessionPersistence.load(m.id) - expect(events.map(e => e.type)).toEqual(['turn/start', 'turn/end']) - const end = events[1]! - expect(end.type === 'turn/end' && end.data.reason).toEqual({ kind: 'interrupted' }) - }) - - it('initFor is idempotent: a re-seeded existing session is not re-initialized', async () => { - const session = ctx.sessions.create('idem', { meta: { cwd: '/i' } }) - session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - // Re-emit session/created for the SAME live session (idempotent initFor). - ctx.emit('session/created', session) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('idem')) - expect(loaded.events).toHaveLength(2) // not doubled - }) - - - it('flush before init resolves with no state uses cursor 0', async () => { - // Drive a fork (seed) flush where the buffer holds the seed; the fresh - // events filter against cursor. Exercises the state-undefined cursor path. - const session = ctx.sessions.create('flush-nostate') - // Append directly to the live session and flush IMMEDIATELY, before the - // async onCreated init has necessarily set state. - session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate')) - expect(loaded.events).toHaveLength(2) - }) - - it('createCore rejects creating an id this backend already tracks', async () => { - await ctx.sessionPersistence.create(meta('dup')) - await expect(ctx.sessionPersistence.create(meta('dup'))).rejects.toThrow(/already exists in this backend/) - }) - - it('createCore rejects creating an id whose log already exists on disk', async () => { - const m = meta('on-disk', '/od') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // A fresh backend (no in-memory state) must refuse to create over the log. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await expect(ctx2.sessionPersistence.create(meta('on-disk', '/od'))).rejects.toThrow(/already has a persisted log on disk/) - await ctx2.fiber.dispose() - }) - - it('createCore rejects an id already on disk under a DIFFERENT cwd bucket', async () => { - // Persist the id under cwd A. - const a = meta('dup-id', '/projA') - await ctx.sessionPersistence.create(a) - await ctx.sessionPersistence.append(a.id, oneTurnLog()) - // A fresh backend creating the SAME id under cwd B must still refuse: load/ - // has identify by id across all buckets, so a second log would make resume - // nondeterministic. create scans every bucket, not just meta.cwd's. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await expect(ctx2.sessionPersistence.create(meta('dup-id', '/projB'))) - .rejects.toThrow(/already has a persisted log on disk/) - await ctx2.fiber.dispose() - }) - - it('flush keeps buffered events when the append fails (no silent loss)', async () => { - root = await freshRoot() - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const session = ctx2.sessions.create('flush-fail') - // A full turn lands in the write-behind buffer. - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - // Make the durable materialize fail on the next flush. - const backend = ctx2.sessionPersistence as unknown as { materialize: (...args: unknown[]) => Promise } - const origMat = backend.materialize.bind(backend) - backend.materialize = () => Promise.reject(new Error('disk full')) - await expect(ctx2.parallel('session/flush', session)).rejects.toThrow(/disk full/) - // The events are STILL buffered (not silently dropped): a retry persists them. - backend.materialize = origMat - await ctx2.parallel('session/flush', session) - const loaded = await ctx2.sessionPersistence.load(SessionId('flush-fail')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - await ctx2.fiber.dispose() - }) - - it('rejects non-JSON event data: BigInt, function, circular, Map, undefined property', async () => { - const m = meta('serial') - await ctx.sessionPersistence.create(m) - const bad = (extra: unknown) => [{ type: 'user/message', seq: 0, time: 1, data: { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra } }] as unknown as SessionEvent[] - await expect(ctx.sessionPersistence.append(m.id, bad(1n))).rejects.toThrow(/non-JSON-serializable/) - await expect(ctx.sessionPersistence.append(m.id, bad(() => 0))).rejects.toThrow(/non-JSON-serializable/) - await expect(ctx.sessionPersistence.append(m.id, bad(Symbol('s')))).rejects.toThrow(/non-JSON-serializable/) - await expect(ctx.sessionPersistence.append(m.id, bad(new Map()))).rejects.toThrow(/non-JSON-serializable/) - await expect(ctx.sessionPersistence.append(m.id, bad(undefined))).rejects.toThrow(/non-JSON-serializable/) - await expect(ctx.sessionPersistence.append(m.id, bad(Infinity))).rejects.toThrow(/non-JSON-serializable/) - // a circular structure - const circ: Record = {} - circ.self = circ - await expect(ctx.sessionPersistence.append(m.id, bad(circ))).rejects.toThrow(/non-JSON-serializable/) - // The session was never materialized by any of the rejected appends. - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) - }) - - it('accepts well-formed JSON values (null, booleans, nested arrays/objects)', async () => { - const m = meta('json-ok') - await ctx.sessionPersistence.create(m) - const ev = [{ type: 'user/message', seq: 0, time: 1, data: { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra: { a: null, b: true, c: [1, 2, { d: 'nested' }] } } }] as unknown as SessionEvent[] - await ctx.sessionPersistence.append(m.id, ev) - expect(await ctx.sessionPersistence.has(m.id)).toBe(true) - }) - - it('Session.append rejects a non-serializable event at the source (never enters the log)', () => { - const session = ctx.sessions.create('reject-bad') - // Serializability is enforced at the source: Session.append throws on a - // BigInt-bearing event BEFORE it enters session.events, so the durable log - // can never diverge from the live log. The throw surfaces at the caller's - // append site, not asynchronously in a backend flush. - expect(() => { - session.append('user/message', { content: [{ type: 'text', text: 'bad' }], source: { kind: 'user' }, bad: 1n } as never) - }).toThrow(/non-JSON-serializable/) - // The bad event was rejected, so the log stayed empty. - expect(session.events.length).toBe(0) - }) - -}) diff --git a/packages/session-persistence-jsonl/tsconfig.json b/packages/session-persistence-jsonl/tsconfig.json deleted file mode 100644 index 3595f989bd..0000000000 --- a/packages/session-persistence-jsonl/tsconfig.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../session" }, - { "path": "../session-persistence" } - ] -} diff --git a/packages/session-persistence-sqlite/src/index.ts b/packages/session-persistence-sqlite/src/index.ts deleted file mode 100644 index d28fc8d6f7..0000000000 --- a/packages/session-persistence-sqlite/src/index.ts +++ /dev/null @@ -1,563 +0,0 @@ -/** - * SQLite durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-sqlite`). - * - * A SECOND {@link SessionPersistence} implementation, built to validate that - * the abstract seam + the shared `runPersistenceContract` suite are genuinely - * backend-agnostic: the same append-only / contiguous-seq / lazy-materialization - * / interrupted-turn-close-on-load semantics the JSONL backend expresses over - * file bytes, expressed here over `node:sqlite` rows. Each `SessionEvent` maps - * 1:1 onto a row `(session_id, seq, type, time, data)`; `append` is an INSERT - * inside a transaction that asserts the contiguous-seq contract; the mutable - * `SessionSummary` lives in the `sessions` metadata row. - * - * Like the JSONL backend it is also the write-path plugin: it installs the - * `session/event` → buffer → `session/flush` drain, persists a fork's seed once - * on `session/created`, keeps a per-session write cursor so a resumed session - * never re-appends stored events, and seeds existing live sessions on apply - * (HMR does not replay `session/created`). - * - * @module @deepseek-ai/dsh-session-persistence-sqlite - */ - -import { Context } from 'cordis' -import z from 'schemastery' -import { DatabaseSync } from 'node:sqlite' -import { mkdir } from 'node:fs/promises' -import { dirname, resolve } from 'node:path' -import { - SessionPersistence, assertSerializable, seedCoversPrefix, -} from '@deepseek-ai/dsh-session-persistence' -import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' -import { - openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow, -} from './schema.ts' - -export { SCHEMA_VERSION } from './schema.ts' - -/** Plugin configuration. */ -export interface Config { - /** - * Filesystem path to the SQLite database file. The special value `:memory:` - * opens an in-process database (tests); a file path is created (with parent - * dirs) on construction. - */ - path: string -} - -/** Backend bookkeeping for a session id (NOT the live Session object). */ -interface SessionState { - meta: SessionMeta - /** Next seq to write — equals the number of committed events. */ - cursor: number - /** Whether the session has at least one persisted event (materialized). */ - materialized: boolean - /** The live Session that owns this state (collision detection); see onCreated. */ - owner?: Session -} - -async function settledErrors(promises: Iterable>): Promise { - const settled = await Promise.allSettled([...promises]) - const errors: unknown[] = [] - for (const result of settled) { - if (result.status === 'rejected') errors.push(result.reason) - } - return errors -} - -/** - * The SQLite persistence backend. Load as a plugin; it registers as - * `ctx.sessionPersistence` and installs the write-path listeners. - */ -export class SessionPersistenceSqlite extends SessionPersistence { - static inject = ['sessions'] - - static Config: z = z.object({ - path: z.string().required(), - }) - - private db!: DatabaseSync - private ready: Promise - /** Backend bookkeeping keyed by session id (NOT the live Session object). */ - private states = new Map() - /** Write-behind buffers keyed by the live Session (write path). */ - private buffers = new Map() - /** Per-session serialization chain (keyed by session id). */ - private chains = new Map>() - /** Per-session init promise (onCreated), keyed by the LIVE Session object. */ - private inits = new Map>() - - constructor(ctx: Context, public config: Config) { - super(ctx) - // Open the database asynchronously (the parent directory may need creating); - // every backend op awaits `ready` first. Opening synchronously in the ctor - // would force a sync mkdir and block plugin apply. - this.ready = this.openDb(config.path) - this.installWritePath() - } - - private async openDb(path: string): Promise { - if (path !== ':memory:') { - const abs = resolve(path) - await mkdir(dirname(abs), { recursive: true, mode: 0o700 }) - this.db = openDatabase(abs) - } else { - this.db = openDatabase(path) - } - } - - // --- SessionPersistence backend surface (all serialized per session id) --- - - create(meta: SessionMeta): Promise { - const snapshot: SessionMeta = { ...meta } - return this.serialize(snapshot.id, () => this.createCore(snapshot)) - } - - private async createCore(meta: SessionMeta): Promise { - await this.ready - if (this.states.has(meta.id)) { - throw new Error(`session "${meta.id}" already exists in this backend`) - } - if (this.rowFor(meta.id) !== undefined) { - throw new Error(`session "${meta.id}" already has a persisted row; load/resume it instead of creating`) - } - // Lazy: record intent in memory only. No row until the first append, so an - // abandoned (never-appended) session leaves nothing behind and stays absent - // from has()/list(). - this.states.set(meta.id, { meta, cursor: 0, materialized: false }) - } - - // `async` so the synchronous validate/clone below reject (not throw) per the - // Promise contract — callers use `await expect(...).rejects`. - async append(id: SessionId, events: readonly SessionEvent[]): Promise { - // Validate serializability BEFORE cloning so a bad event surfaces the typed - // "non-JSON-serializable" error rather than an opaque DataCloneError from - // structuredClone. Then deep-snapshot the batch HERE, before the op waits - // behind the per-session chain: a caller that passes a live array (e.g. - // session.events) and mutates it — OR mutates an event inside it — before - // the op runs would otherwise have those changes persisted, or advance the - // cursor past what was written. The clone is taken at call time (before the - // first await), matching the JSONL backend. - assertSerializable(events) - const batch = events.map(e => structuredClone(e)) - return this.serialize(id, () => this.appendCore(id, batch)) - } - - private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { - await this.ready - if (events.length === 0) return - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) - - // Contiguity contract: each event's seq must continue the stored log. - for (const [i, event] of events.entries()) { - if (event.seq !== state.cursor + i) { - throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`) - } - } - - // The transaction is the durability + atomicity boundary: materialize the - // sessions row (if lazy) and INSERT every event, or roll back entirely. A - // BEGIN/COMMIT around the batch means a mid-batch failure (a UNIQUE - // violation on a duplicated seq from a concurrent writer) leaves the stored - // log untouched, so the cursor stays truthful and a retry is clean. (A crash - // tail is already gone: load() physically deletes the torn fragment and - // durably closes the interrupted turn before returning, so by the time any - // append runs the stored log is balanced and contiguous.) - const insertEvent = this.db.prepare( - 'INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)', - ) - this.db.exec('BEGIN') - try { - if (!state.materialized) this.writeRow(state.meta) - for (const event of events) { - insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data)) - } - // Bump updatedAt on every append (the mutable summary lives in the row). - const updatedAt = Date.now() - this.db.prepare('UPDATE sessions SET updated_at = ? WHERE id = ?').run(updatedAt, id) - this.db.exec('COMMIT') - state.meta = { ...state.meta, updatedAt } - } catch (error) { - this.db.exec('ROLLBACK') - throw error - } - state.materialized = true - state.cursor += events.length - } - - load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { - return this.serialize(id, () => this.loadCore(id)) - } - - private async loadCore(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { - await this.ready - const row = this.rowFor(id) - if (row === undefined) throw new Error(`session "${id}" not found`) - const meta = rowToMeta(row) - this.assertVersion(meta) - - // Read every stored row ordered by seq, then scan for the preserved prefix: - // the longest seq-contiguous, parseable run, INCLUDING the real events of an - // interrupted final turn after the last turn/end (a turn can be huge — they - // are never truncated). scanRows works off the seq+type COLUMNS for the - // last-turn/end boundary, so a malformed `data` in a torn tail row is - // discarded (not unloadable); only a parse error / seq gap in the COMMITTED - // region (at or before the last turn/end) throws (genuine corruption). - const eventRows = this.db - .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') - .all(id) as unknown as EventRow[] - const { preserved, tornFrom } = scanRows(eventRows) - - // Crash-recovery (mutating load, same as the JSONL backend): if the log ended - // mid-turn, close it DURING load so disk, the returned log, and the cursor all - // agree — both append routes then continue with no special-casing. Synthesize - // the boundary events (a step/end if a step was open, then a - // turn/end {kind:'interrupted'}); the interrupted turn's real events are - // preserved, never truncated (the session-persistence RFC). - const closers = interruptedTurnClosers(preserved) - const balanced = [...preserved, ...closers] - - // Physically repair the stored log inside one transaction: DELETE the torn - // tail fragment (if any), then INSERT the synthetic closers. After COMMIT the - // stored rows == balanced, so the cursor is truthful and the next append - // continues cleanly with no deferred repair. The metadata row stays as-is - // even when preserved.length === 0 (an all-tail crash): the session WAS - // materialized by the partial append, so has()/list() still report it — the - // same as the JSONL backend, whose file likewise survives a first append that - // never reached turn/end. - if (tornFrom !== undefined || closers.length > 0) { - this.db.exec('BEGIN') - try { - if (tornFrom !== undefined) { - this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(id, tornFrom) - } - if (closers.length > 0) { - const insertEvent = this.db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - for (const event of closers) { - insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data)) - } - } - this.db.exec('COMMIT') - } catch (error) { - // The DELETE+INSERT cannot collide (a row at a closer's seq is preserved - // or deleted as torn first); this rolls back a DB-level failure (disk - // full, etc.), unreachable in test. - /* v8 ignore start */ - this.db.exec('ROLLBACK') - throw error - /* v8 ignore stop */ - } - } - - // Record state at the balanced length. The state keeps its OWN copy of the - // meta; the returned value is separate so a consumer mutating loaded.meta - // cannot corrupt the backend's row metadata. - this.states.set(id, { - meta: { ...meta }, - cursor: balanced.length, - materialized: true, - }) - return { meta, events: balanced } - } - - private async adoptLiveStoredPrefix(session: Session, seed: readonly SessionEvent[]): Promise { - await this.ready - const row = this.rowFor(session.header.id) - /* v8 ignore next -- caller checked row existence */ - if (row === undefined) throw new Error(`session "${session.header.id}" not found`) - const meta = rowToMeta(row) - this.assertVersion(meta) - - const rows = this.db - .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') - .all(session.header.id) as unknown as EventRow[] - const { preserved, tornFrom } = scanRows(rows) - if (!seedCoversPrefix(seed, preserved)) { - throw new Error(`session "${session.header.id}" already has a persisted log that does not match this live session (id collision)`) - } - - if (tornFrom !== undefined) { - this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(session.header.id, tornFrom) - } - this.states.set(session.header.id, { - meta: { ...meta }, - cursor: preserved.length, - materialized: true, - owner: session, - }) - const suffix = seed.slice(preserved.length) - if (suffix.length > 0) await this.appendCore(session.header.id, suffix) - } - - async list(): Promise { - await this.ready - // Every metadata row is a materialized session: the row is written only by - // the first append (a created-but-never-appended session has no row), so - // listing all rows is exactly the materialized set. - const rows = this.db - .prepare('SELECT * FROM sessions') - .all() as unknown as SessionRow[] - return rows.map(rowToMeta) - } - - async has(id: SessionId): Promise { - await this.ready - const state = this.states.get(id) - if (state?.materialized) return true - // A metadata row exists iff the session was materialized by a first append. - return this.rowFor(id) !== undefined - } - - delete(id: SessionId): Promise { - return this.serialize(id, () => this.deleteCore(id)) - } - - private async deleteCore(id: SessionId): Promise { - await this.ready - // ON DELETE CASCADE drops the session's events with its row. - this.db.prepare('DELETE FROM sessions WHERE id = ?').run(id) - this.states.delete(id) - } - - update(id: SessionId, summary: Partial): Promise { - return this.serialize(id, () => this.updateCore(id, summary)) - } - - private async updateCore(id: SessionId, summary: Partial): Promise { - await this.ready - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) - const nextMeta: SessionMeta = { ...state.meta, ...summary, updatedAt: summary.updatedAt ?? Date.now() } - // update's only durable effect is the summary fields; the event log is - // untouched. If the row is not materialized yet (a lazy session updated - // before its first append) there is nothing to write — keep the pending - // summary in memory so the materializing append carries it. - if (state.materialized) this.writeRow(nextMeta) - state.meta = nextMeta - } - - // --- row helpers --- - - /** Fetch a session's row, or undefined if absent. */ - private rowFor(id: SessionId): SessionRow | undefined { - const row = this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined - return row - } - - /** - * Insert-or-replace a session's metadata row. The only callers are the first - * materializing `append` and a post-materialization `update`, so writing the - * row IS the materialization (its existence is the signal `has`/`list` read); - * a never-appended session has no row at all. - */ - private writeRow(meta: SessionMeta): void { - this.db.prepare(` - INSERT INTO sessions (id, version, created_at, cwd, parent_session, updated_at, title, first_prompt) - VALUES (?, ?, ?, ?, ?, ?, ?, ?) - ON CONFLICT(id) DO UPDATE SET - version = excluded.version, - created_at = excluded.created_at, - cwd = excluded.cwd, - parent_session = excluded.parent_session, - updated_at = excluded.updated_at, - title = excluded.title, - first_prompt = excluded.first_prompt - `).run( - meta.id, - meta.version, - meta.createdAt, - meta.cwd ?? null, - meta.parentSession ?? null, - meta.updatedAt, - meta.title ?? null, - meta.firstPrompt ?? null, - ) - } - - /** Build a state for a session present in the DB but not yet in memory. */ - private async adopt(id: SessionId): Promise { - await this.loadCore(id) // sets the state; load (serialized) would deadlock - const state = this.states.get(id) - /* v8 ignore next -- loadCore always sets the state for the id */ - if (!state) throw new Error(`failed to adopt session "${id}"`) - return state - } - - private assertVersion(meta: SessionMeta): void { - if (meta.version !== 1) { - throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) - } - } - - /** - * Run `op` after any in-flight operation for the same session id, so writes - * for one session never interleave. Errors do not poison the chain. NOTE: - * serialized public methods must NOT call each other (deadlock); they call - * the unserialized `*Core` helpers instead. - */ - private serialize(id: SessionId, op: () => Promise): Promise { - const prior = this.chains.get(id) ?? Promise.resolve() - const next = prior.then(op, op) - this.chains.set(id, next.then(() => undefined, () => undefined)) - return next - } - - // --- write path (session/event → flush drain) --- - - private installWritePath(): void { - const ctx = this.ctx - - ctx.on('session/created', (session) => { void this.initFor(session) }) - - // Snapshot + buffer every event (the live object is mutable; clone so a - // later in-place mutation cannot rewrite a buffered event). Serializability - // is guaranteed at the source (Session.append), so structuredClone is safe. - ctx.on('session/event', (session, event) => { - let buffer = this.buffers.get(session) - if (!buffer) this.buffers.set(session, buffer = []) - buffer.push(structuredClone(event)) - }) - - ctx.on('session/flush', session => this.flush(session)) - - // Dispose must reach quiescence: await every init + final drain, then close - // the database, BEFORE returning, so no write lands after teardown. - ctx.effect(() => async () => { - let disposeError: unknown - try { - const errors = [ - ...await settledErrors(this.inits.values()), - ...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))), - ...await settledErrors(this.chains.values()), - ] - if (errors.length > 0) { - throw new AggregateError(errors, 'session-persistence-sqlite dispose failed') - } - } catch (error: unknown) { - disposeError = error - throw error - } finally { - try { - await this.ready - this.db.close() - } catch (error: unknown) { - /* v8 ignore next -- open/close failure racing disposal is a defensive teardown edge */ - if (disposeError === undefined) throw error - // Opening/closing the database can only add teardown context here; keep - // the already-captured init/flush/chain AggregateError as the primary - // disposal failure instead of masking it from callers. - } - } - }, 'session-persistence-sqlite write path') - - // HMR: a hot reload does not replay session/created, so seed existing live - // sessions (mirrors dsh-invariants and the JSONL backend). - for (const session of ctx.sessions.list()) void this.initFor(session) - } - - /** Start (once) the async init for a session and remember its promise. */ - private initFor(session: Session): Promise { - const existing = this.inits.get(session) - if (existing) return existing - const seed = session.events.map(e => structuredClone(e)) - const p = this.onCreated(session, seed) - p.catch(() => { /* observed by flush/dispose via the stored promise */ }) - this.inits.set(session, p) - return p - } - - /** - * On session/created: sync the backend's state to a live Session. Cases - * mirror the JSONL backend: - * 1. Already tracked → no-op (or claim ownerless state if the seed matches). - * 2. A row EXISTS and is a seq-aligned PREFIX of the live events → adopt - * (HMR/resume), persisting any live suffix beyond the stored prefix. - * 3. A row EXISTS but is NOT a prefix → reject (id collision). - * 4. No row → a genuinely new session: register meta (lazy) + persist seed. - */ - private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise { - await this.ready - const id = session.header.id - const tracked = this.states.get(id) - if (tracked !== undefined) { - /* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */ - if (tracked.owner === session) return - if (tracked.owner === undefined) { - // Ownerless state from a public create()/load(). The first live session - // claims it ONLY if its seed reproduces the persisted prefix. - if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) { - throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`) - } - tracked.owner = session - const suffix = seed.slice(tracked.cursor) - if (suffix.length > 0) await this.append(id, suffix) - return - } - // Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id - // (never materialized, no pending buffer); else it is a real collision. - const ownerBuffer = this.buffers.get(tracked.owner) - if (!tracked.materialized && !ownerBuffer?.length) { - this.states.delete(id) - } else { - throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`) - } - } - - const row = this.rowFor(id) - if (row !== undefined) { - // Adopt a LIVE prefix without crash-repairing an open turn as interrupted; - // HMR may still append the real completion from the live Session. - await this.serialize(id, () => this.adoptLiveStoredPrefix(session, seed)) - return - } - - // case 4: a genuinely new session. - const meta: SessionMeta = { ...session.header, updatedAt: Date.now() } - await this.create(meta) - const created = this.states.get(id) - /* v8 ignore next -- create() always sets the state for the id */ - if (created !== undefined) created.owner = session - if (seed.length > 0) await this.append(id, seed) - } - - /** The preserved events for a session id (torn tail excluded, turn NOT yet closed). */ - private eventsFor(id: SessionId): SessionEvent[] { - const rows = this.db - .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') - .all(id) as unknown as EventRow[] - // Scan on seq+type columns, parsing `data` only for the preserved prefix (a - // malformed torn tail must not throw here — same as loadCore). Returns the - // preserved events WITHOUT the synthetic closers, so a collision check - // compares a live seed against the real on-disk events, mirroring the JSONL - // backend's scanLog use in onCreated. - return scanRows(rows).preserved - } - - /** Whether a live session's seed reproduces the first `cursor` stored events. */ - private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise { - await this.ready - if (cursor === 0) return true - return seedCoversPrefix(seed, this.eventsFor(id).slice(0, cursor)) - } - - private async flush(session: Session): Promise { - await this.inits.get(session) - await this.serialize(session.header.id, () => this.drain(session)) - } - - /** Drain a session's write buffer to the database. Caller serializes per id. */ - private async drain(session: Session): Promise { - const buffer = this.buffers.get(session) - if (!buffer?.length) return - const batch = buffer.slice() - const state = this.states.get(session.header.id) - /* v8 ignore next -- state is always set by the awaited init before flush */ - const cursor = state?.cursor ?? 0 - const fresh = batch.filter(e => e.seq >= cursor) - if (fresh.length > 0) await this.appendCore(session.header.id, fresh) - buffer.splice(0, batch.length) - } -} - -export default SessionPersistenceSqlite diff --git a/packages/session-persistence-sqlite/tests/sqlite.spec.ts b/packages/session-persistence-sqlite/tests/sqlite.spec.ts deleted file mode 100644 index d6216b7b50..0000000000 --- a/packages/session-persistence-sqlite/tests/sqlite.spec.ts +++ /dev/null @@ -1,781 +0,0 @@ -import { afterEach, describe, expect, it } from 'vitest' -import { Context } from 'cordis' -import { mkdtemp, rm } from 'node:fs/promises' -import { tmpdir } from 'node:os' -import { join } from 'node:path' -import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session' -import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite' -import { openDatabase, scanRows, type EventRow } from '../src/schema.ts' -import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' - -const dirs: string[] = [] -afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) - -async function freshDbPath(): Promise { - const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-')) - dirs.push(dir) - return join(dir, 'sessions.db') -} - -/** A context with the session store + SQLite backend, plus a teardown. */ -async function backend(path = ':memory:'): Promise<{ ctx: Context; dispose: () => Promise }> { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - return { ctx, dispose: () => fiber.dispose() } -} - -// The payoff: the SAME backend-agnostic contract the JSONL backend runs, now -// proving the SQLite backend satisfies identical semantics. -runPersistenceContract('sqlite', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - return { - persistence: ctx.sessionPersistence, - dispose: async () => { await fiber.dispose() }, - } -}) - -describe('scanRows', () => { - // scanRows works off EventRows (data is a JSON string column); build them from - // SessionEvents so the unit tests read in terms of the event vocabulary. - const rows = (events: SessionEvent[]): EventRow[] => - events.map(e => ({ seq: e.seq, type: e.type, time: e.time, data: JSON.stringify(e.data) })) - - it('preserves the full log when it ends exactly on a turn/end (no torn tail)', () => { - const { preserved, tornFrom } = scanRows(rows(oneTurnLog())) - expect(preserved).toEqual(oneTurnLog()) - expect(tornFrom).toBeUndefined() - }) - - it('PRESERVES the real events of an interrupted turn after the last turn/end', () => { - // turn 1 committed (0..5) + a crashed turn 2 (turn/start 6, step/start 7, no - // close): all 8 rows are intact, so the whole prefix is preserved and there - // is no torn fragment to delete. (load() then synthesizes the closers.) - const withOpenTurn: SessionEvent[] = [ - ...oneTurnLog(), - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, - ] - const { preserved, tornFrom } = scanRows(rows(withOpenTurn)) - expect(preserved.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - expect(tornFrom).toBeUndefined() - }) - - it('preserves the contiguous prefix and flags a torn tail at a seq gap', () => { - // A gap after seq 0 (no committed turn/end): seq 0 is the preserved - // interrupted-turn event; the gap bounds it and marks the torn fragment. - const gapped: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing - ] - const { preserved, tornFrom } = scanRows(rows(gapped)) - expect(preserved.map(e => e.seq)).toEqual([0]) - expect(tornFrom).toBe(1) - }) - - it('an empty log preserves nothing and has no torn tail', () => { - expect(scanRows([])).toEqual({ preserved: [] }) - }) - - it('throws on a seq gap inside the committed region (before the last turn/end)', () => { - const gapped: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing - { type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, - ] - expect(() => scanRows(rows(gapped))).toThrow(/seq gap in committed region/) - }) - - it('throws on an unparsable row inside the committed region', () => { - const withCorruptCommitted: EventRow[] = [ - { seq: 0, type: 'turn/start', time: 1, data: '{not json' }, // corrupt, sits before a turn/end - { seq: 1, type: 'turn/end', time: 2, data: JSON.stringify({ turn: 1, reason: { kind: 'completed' } }) }, - ] - expect(() => scanRows(withCorruptCommitted)).toThrow(/unparsable committed event/) - }) - - it('tolerates an unparsable torn-tail row after the last turn/end', () => { - const withCorruptTail: EventRow[] = [ - ...rows(oneTurnLog()), - { seq: 6, type: 'turn/start', time: 7, data: '{not json' }, // torn fragment, no committed turn/end after - ] - const { preserved, tornFrom } = scanRows(withCorruptTail) - expect(preserved).toEqual(oneTurnLog()) - expect(tornFrom).toBe(6) - }) -}) - -describe('SessionPersistenceSqlite: HMR adoption', () => { - it('does not crash-repair an active open turn as interrupted', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - const first = await ctx.plugin(SessionPersistenceSqlite, { path }) - const session = ctx.sessions.create('hmr-open', { meta: { cwd: '/hmr' } }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('step/start', { turn: 1, step: 1 }) - await ctx.parallel('session/flush', session) - - await first.dispose() - const db = openDatabase(path) - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') - .run('hmr-open', 2, 'step/end', 2, '{"torn":') - db.close() - const second = await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('step/end', { turn: 1, step: 1 }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open')) - expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end']) - expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) - await second.dispose() - await ctx.fiber.dispose() - }) -}) - -describe('SessionPersistenceSqlite: durability and crash semantics', () => { - it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => { - const path = await freshDbPath() - const m = meta('crash') - // Run 1: persist a complete turn, then a half-written second turn (no turn/end). - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) - await ctx1.sessionPersistence.create(m) - await ctx1.sessionPersistence.append(m.id, oneTurnLog()) - await ctx1.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, - ]) - await fiber1.dispose() - - // Run 2: load PRESERVES the interrupted turn's real events (a turn can be huge - // — never truncated) and closes the orphaned turn with synthetic boundary - // events: step/end (the step was open) then turn/end {interrupted}. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.type)).toEqual([ - 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1 - 'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real events + synthetic closers - ]) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) - const last = loaded.events.at(-1)! - expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) - - // load durably closed the turn, so the next append continues at the balanced - // length (seq 10) and a reload round-trips identically. - await ctx2.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } }, - ]) - const reloaded = await ctx2.sessionPersistence.load(m.id) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) - await fiber2.dispose() - }) - - it('load() durably closes the interrupted turn: the synthetic closers are on disk after load', async () => { - const path = await freshDbPath() - const m = meta('load-closes') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5 - await b1.dispose() - // Hand-write an interrupted turn (turn/start seq 6, no turn/end). - const db = openDatabase(path) - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)') - .run(m.id, 'turn/start', JSON.stringify({ turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })) - db.close() - - const b2 = await backend(path) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - // turn 2's real turn/start (seq 6) is preserved + a synthetic turn/end (seq 7). - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - expect(loaded.events.at(-1)!.type).toBe('turn/end') - // load() is mutating: the synthetic turn/end MUST be on disk so the stored log - // is balanced and the cursor is truthful (contract: load closes, not defers). - const probe = openDatabase(path) - const stored = probe.prepare('SELECT seq, type FROM events WHERE session_id = ? ORDER BY seq').all(m.id) as { seq: number; type: string }[] - probe.close() - expect(stored.map(r => r.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - expect(stored.at(-1)!.type).toBe('turn/end') - await b2.dispose() - }) - - it('all-tail load: a session whose only turn never closed is preserved and closed on load', async () => { - const path = await freshDbPath() - const m = meta('all-tail') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - // A first turn that NEVER completed: turn/start + user/message, no turn/end. - await b1.ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } } }, - ]) - expect(await b1.ctx.sessionPersistence.has(m.id)).toBe(true) // materialized - await b1.dispose() - - // A fresh backend loads it: the interrupted (only) turn's real events are - // preserved and closed with a synthetic turn/end {interrupted} — NOT - // truncated. The session was materialized, so has()/list() report it present. - const b2 = await backend(path) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'user/message', 'turn/end']) - expect(loaded.events.at(-1)!.type === 'turn/end' && loaded.events.at(-1)!.data).toMatchObject({ reason: { kind: 'interrupted' } }) - expect(await b2.ctx.sessionPersistence.has(m.id)).toBe(true) - expect((await b2.ctx.sessionPersistence.list()).map(x => x.id)).toContain(m.id) - await b2.dispose() - }) - - it('rejects opening a database whose schema version is newer than this build', async () => { - const path = await freshDbPath() - openDatabase(path).close() // stamp user_version = SCHEMA_VERSION - // Bump user_version past what this build supports. - const db = openDatabase(path) - db.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`) - db.close() - expect(() => openDatabase(path)).toThrow(/newer than this build/) - }) - - it('append snapshots the batch: mutating an event after the call does not corrupt the persisted copy', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const m = meta('snapshot') - await ctx.sessionPersistence.create(m) - const batch: SessionEvent[] = [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } } }, - { type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, - ] - const p = ctx.sessionPersistence.append(m.id, batch) - // Mutate the live array AND an event's data AFTER the call but before it - // drains behind the per-session chain. The snapshot taken at call time must - // shield the persisted copy. - ;(batch[1]!.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED' - batch.push({ type: 'user/message', seq: 3, time: 4, data: { content: [{ type: 'text', text: 'injected' }], source: { kind: 'user' } } }) - await p - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events).toHaveLength(3) // the pushed event was not persisted - const um = loaded.events[1] - expect(um?.type === 'user/message' && (um.data.content[0] as { text: string }).text).toBe('original') - await fiber.dispose() - }) - - it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => { - const path = await freshDbPath() - const m = meta('corrupt-tail') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed: seqs 0..5 - await b1.dispose() - - // Hand-insert a torn tail row (seq 6, no closing turn/end) whose `data` is - // invalid JSON. The contract: only a parse error in the COMMITTED region is - // unloadable; a torn tail must be discarded. scanRows finds the last - // turn/end on the seq+type columns (never parsing tail `data`), so the - // unparsable row after it bounds the preserved prefix and is deleted by load. - const db = openDatabase(path) - db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)') - .run(m.id, 'turn/start', '{not valid json') - db.close() - - const b2 = await backend(path) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events).toEqual(oneTurnLog()) // torn tail discarded, committed intact (turn 1 already balanced → no closers) - // load physically deleted the corrupt tail row, so a fresh append continues. - await b2.ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 8, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 9, data: { turn: 2, reason: { kind: 'completed' } } }, - ]) - const reloaded = await b2.ctx.sessionPersistence.load(m.id) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b2.dispose() - }) - - it('append rolls back the whole batch on a mid-batch seq collision (transaction)', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const m = meta('rollback') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5 - - // A batch that re-states an already-stored seq must be rejected and leave - // the stored log unchanged (the UNIQUE (session_id, seq) constraint fires - // inside the transaction → ROLLBACK). - await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toThrow() - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events).toEqual(oneTurnLog()) // unchanged - await fiber.dispose() - }) - - it('persists across separate backend instances over the same file', async () => { - const path = await freshDbPath() - const m = meta('persist', '/proj') - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) - await ctx1.sessionPersistence.create(m) - await ctx1.sessionPersistence.append(m.id, oneTurnLog()) - await ctx1.sessionPersistence.update(m.id, { title: 'T', firstPrompt: 'hi' }) - await fiber1.dispose() - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - expect((await ctx2.sessionPersistence.list()).map(x => x.id)).toContain(m.id) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj', title: 'T', firstPrompt: 'hi' }) - expect(loaded.events).toEqual(oneTurnLog()) - await fiber2.dispose() - }) - - it('rejects an unknown format version on load', async () => { - const path = await freshDbPath() - // Materialize a row with version 2 directly via the real schema. - const db = openDatabase(path) - db.prepare('INSERT INTO sessions (id, version, created_at, updated_at) VALUES (?, ?, ?, ?)') - .run('v2', 2, 1, 1) - db.close() - - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - await expect(ctx.sessionPersistence.load(SessionId('v2'))).rejects.toThrow(/version 2/) - await fiber.dispose() - }) - - it('create rejects a duplicate id (in memory and on a persisted row)', async () => { - const path = await freshDbPath() - const m = meta('dup') - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - await ctx.sessionPersistence.create(m) - // Same in-memory state. - await expect(ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists/) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - await fiber.dispose() - - // A fresh instance over the same file sees the persisted row. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - await expect(ctx2.sessionPersistence.create(m)).rejects.toThrow(/already has a persisted row/) - await fiber2.dispose() - }) - - it('exposes the schema version constant', () => { - expect(SCHEMA_VERSION).toBe(1) - }) -}) - -describe('SessionPersistenceSqlite: write path (session/event → flush)', () => { - function send(session: Session, events: SessionEvent[]): void { - for (const e of events) session.append(e.type, e.data) - } - - it('persists a turn appended through the live session on flush', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const session = ctx.sessions.create('w1') - send(session, oneTurnLog()) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('w1')) - expect(loaded.events.map(e => e.type)).toEqual(oneTurnLog().map(e => e.type)) - await fiber.dispose() - }) - - it('a resumed session does not re-append its seed', async () => { - const path = await freshDbPath() - // Run 1: persist a full turn through the live session. - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) - const s1 = ctx1.sessions.create('resume') - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx1.parallel('session/flush', s1) - await fiber1.dispose() - - // Run 2: reconstruct the live session from the loaded log (seed), then add a - // second turn. The seed must NOT be re-appended (no UNIQUE collision), and - // the second turn continues the seq. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - const { events } = await ctx2.sessionPersistence.load(SessionId('resume')) - const s2 = ctx2.sessions.create('resume', { seed: events }) - s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await ctx2.parallel('session/flush', s2) - const reloaded = await ctx2.sessionPersistence.load(SessionId('resume')) - expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await fiber2.dispose() - }) - - it('HMR: applying the plugin seeds existing live sessions', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const session = ctx.sessions.create('hmr') - for (const e of oneTurnLog()) session.append(e.type, e.data) - // Plugin applied AFTER the session already has events. - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - await ctx.parallel('session/flush', session) - expect(await ctx.sessionPersistence.has(SessionId('hmr'))).toBe(true) - await fiber.dispose() - }) - - it('dispose drains a pending buffer before closing the database', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) - const session = ctx.sessions.create('drain') - for (const e of oneTurnLog()) session.append(e.type, e.data) - // No explicit flush — dispose must drain the buffer. - await fiber.dispose() - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - expect(await ctx2.sessionPersistence.has(SessionId('drain'))).toBe(true) - await fiber2.dispose() - }) - - it('rejects a different live session colliding on a persisted id', async () => { - const path = await freshDbPath() - const ctx1 = new Context() - await ctx1.plugin(SessionStore) - const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) - const s1 = ctx1.sessions.create('collide') - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await ctx1.parallel('session/flush', s1) - await fiber1.dispose() - - // A fresh, unrelated session reusing the id (no seed) must be rejected. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) - const s2 = ctx2.sessions.create('collide') - s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - await expect(ctx2.parallel('session/flush', s2)).rejects.toThrow(/id collision/) - await fiber2.dispose() - }) - - it('update before the first append keeps the summary in memory and the session lazy', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const m = meta('lazy-update') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.update(m.id, { title: 'pending' }) - // Still lazy: no materialized row yet. - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) - // The first append materializes and carries the pending title. - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('pending') - await fiber.dispose() - }) -}) - -describe('SessionPersistenceSqlite: edge cases', () => { - it('append of an empty batch is a no-op', async () => { - const { ctx, dispose } = await backend() - const m = meta('empty-batch') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, []) - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) // still lazy - await dispose() - }) - - it('load rejects a missing session', async () => { - const { ctx, dispose } = await backend() - await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/) - await dispose() - }) - - it('delete of a non-existent session is a no-op', async () => { - const { ctx, dispose } = await backend() - await ctx.sessionPersistence.delete(SessionId('ghost')) - expect(await ctx.sessionPersistence.has(SessionId('ghost'))).toBe(false) - await dispose() - }) - - it('append adopts a session that exists only in the DB (fresh instance)', async () => { - const path = await freshDbPath() - const m = meta('adopt-append') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - // A fresh instance appends a second turn WITHOUT a prior create/load: append - // must adopt the on-disk row (cursor = stored length) and continue the seq. - const b2 = await backend(path) - await b2.ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ]) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b2.dispose() - }) - - it('update adopts a session that exists only in the DB (fresh instance)', async () => { - const path = await freshDbPath() - const m = meta('adopt-update') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - const b2 = await backend(path) - await b2.ctx.sessionPersistence.update(m.id, { title: 'after restart' }) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('after restart') - await b2.dispose() - }) - - it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => { - const path = await freshDbPath() - const m = meta('rollback-insert') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - - // A SECOND backend over the same file loads the session first, so it adopts - // cursor 6 (the committed length) into its OWN in-memory state. - const b2 = await backend(path) - await b2.ctx.sessionPersistence.load(m.id) // cursor 6 in b2 - const turn2: SessionEvent[] = [ - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ] - // b1 commits seq 6..7 first. - await b1.ctx.sessionPersistence.append(m.id, turn2) - // b2 still thinks its cursor is 6, so this batch passes the contiguity check - // but its INSERT of seq 6 hits the UNIQUE (session_id, seq) constraint - // mid-transaction → ROLLBACK + rethrow. - await expect(b2.ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/UNIQUE/) - // b1's turn is intact; b2's rolled-back attempt left nothing extra. - const loaded = await b1.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b1.dispose() - await b2.dispose() - }) - - it('round-trips a header with parentSession (fork lineage)', async () => { - const { ctx, dispose } = await backend() - const m: SessionMeta = { ...meta('child'), parentSession: SessionId('parent') } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.parentSession).toBe(SessionId('parent')) - await dispose() - }) - - it('a fresh live session reusing a previously-loaded id is rejected (ownerless guard)', async () => { - const path = await freshDbPath() - const m = meta('ownerless') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - const b2 = await backend(path) - // load() leaves ownerless state with cursor 6. - await b2.ctx.sessionPersistence.load(m.id) - // A fresh, unrelated live session reusing the id has a shorter/non-matching - // seed → its onCreated must reject rather than graft onto the loaded prefix. - const s = b2.ctx.sessions.create('ownerless') - s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - await expect(b2.ctx.parallel('session/flush', s)).rejects.toThrow(/id collision/) - await b2.dispose() - }) - - it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => { - const path = await freshDbPath() - const m = meta('claim') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - const b2 = await backend(path) - const { events } = await b2.ctx.sessionPersistence.load(m.id) // ownerless, cursor 6 - // A live session seeded with the loaded log PLUS a new turn claims the state - // and persists only the suffix. - const s = b2.ctx.sessions.create('claim', { seed: [ - ...events, - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, - ] }) - await b2.ctx.parallel('session/flush', s) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) - await b2.dispose() - }) - - it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { - const { ctx, dispose } = await backend() - const inits = (ctx.sessionPersistence as unknown as { inits: Map> }).inits - let first!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - first = inner.sessions.create('reuse') - }, { inject: ['sessions'] })) - await inits.get(first) // let the lazy create register the state - await firstFiber.dispose() // disposed before any append → never materialized - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('reuse') - }, { inject: ['sessions'] })) - await expect(inits.get(reuse)).resolves.toBeUndefined() - reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', reuse) - expect(await ctx.sessionPersistence.has(SessionId('reuse'))).toBe(true) - await dispose() - }) - - it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => { - const { ctx, dispose } = await backend() - const inits = (ctx.sessionPersistence as unknown as { inits: Map> }).inits - let first!: Session - const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { - first = inner.sessions.create('buffered') - }, { inject: ['sessions'] })) - await inits.get(first) - // Append a turn but do NOT flush — events sit in the write-behind buffer. - first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - first.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await firstFiber.dispose() // disposed before flush; not materialized, buffer pending - - let reuse!: Session - await ctx.plugin(Object.assign((inner: Context) => { - reuse = inner.sessions.create('buffered') - }, { inject: ['sessions'] })) - await expect(inits.get(reuse)).rejects.toThrow(/already bound to a different live session/) - await dispose() - }) - - it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => { - const { ctx, dispose } = await backend() - const session = ctx.sessions.create('idem') - ctx.emit('session/created', session) // second create event for the same object - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - expect(await ctx.sessionPersistence.has(SessionId('idem'))).toBe(true) - await dispose() - }) - - it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => { - const { ctx, dispose } = await backend() - // create() registers ownerless state with cursor 0 (no events yet). - await ctx.sessionPersistence.create(meta('cursor0')) - // A live session reusing that id, seeded with a turn, claims the ownerless - // state (cursor 0 trivially matches any seed) and persists the whole seed. - const s = ctx.sessions.create('cursor0', { seed: [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }, - ] }) - await ctx.parallel('session/flush', s) - const loaded = await ctx.sessionPersistence.load(SessionId('cursor0')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) - await dispose() - }) - - it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - // The session lives in its OWN fiber so it survives the backend reload. - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-adopt') - }, { inject: ['sessions'] })) - - // Backend instance 1 materializes the session on disk. - const backend1 = await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Hot-reload: dispose instance 1, plug in instance 2 over the SAME file - // while the session stays live. Instance 2 has an empty states map but the - // row is materialized on disk and is a prefix of the live events — it must - // ADOPT (not reject), and a second turn then persists. - await backend1.dispose() - await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow() - - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt')) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) - - it('HMR: adoption persists the live SUFFIX that was ahead of the on-disk prefix', async () => { - const path = await freshDbPath() - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-suffix') - }, { inject: ['sessions'] })) - - const backend1 = await ctx.plugin(SessionPersistenceSqlite, { path }) - session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) - await ctx.parallel('session/flush', session) - - // Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT - // flushing turn 2: it is now ONLY in the live session's events. - await backend1.dispose() - session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) - session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) - - // Instance 2 adopts the on-disk prefix (turn 1) and MUST persist the live - // suffix (turn 2) carried in the session's events. - await ctx.plugin(SessionPersistenceSqlite, { path }) - await ctx.parallel('session/flush', session) - const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix')) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3]) - expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) - await ctx.fiber.dispose() - }) - - it('HMR: a DIFFERENT session colliding with a materialized on-disk id is rejected', async () => { - const path = await freshDbPath() - // Instance 1 materializes a session and disposes. - const b1 = await backend(path) - const s1 = b1.ctx.sessions.create('hmr-collide') - for (const e of oneTurnLog()) s1.append(e.type, e.data) - await b1.ctx.parallel('session/flush', s1) - await b1.dispose() - - // A fresh context with an UNRELATED live session reusing the id meets a - // materialized row that is NOT a prefix of its events → reject. - const ctx = new Context() - await ctx.plugin(SessionStore) - let session!: Session - await ctx.plugin(Object.assign((inner: Context) => { - session = inner.sessions.create('hmr-collide') - }, { inject: ['sessions'] })) - session.append('turn/start', { turn: 9, trigger: { kind: 'message', source: { kind: 'user' } } }) - await ctx.plugin(SessionPersistenceSqlite, { path }) - await expect(ctx.parallel('session/flush', session)).rejects.toThrow(/id collision/) - await ctx.fiber.dispose() - }) -}) diff --git a/packages/session-persistence-sqlite/tsconfig.json b/packages/session-persistence-sqlite/tsconfig.json deleted file mode 100644 index 3595f989bd..0000000000 --- a/packages/session-persistence-sqlite/tsconfig.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../../vendor/schemastery" }, - { "path": "../session" }, - { "path": "../session-persistence" } - ] -} diff --git a/packages/session-persistence/README.md b/packages/session-persistence/README.md index 4da1f00df5..603e525ee0 100644 --- a/packages/session-persistence/README.md +++ b/packages/session-persistence/README.md @@ -1,33 +1,11 @@ -# @deepseek-ai/dsh-session-persistence +# session-persistence/ — persistence capability family -The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, list, and update sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../docs/rfc/implemented/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface. +The durable session-persistence seam and its storage backends. The interface package owns the abstract `SessionPersistence` service and the shared write coordinator; the backends are concrete implementations that register on `ctx.sessionPersistence`. All **product** packages. -The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionMeta`, owned by `dsh-session` and re-exported here. +| Package | Role | ctx key | +|---|---|---| +| `session-persistence/` | Persistence seam + shared write coordinator | `ctx.sessionPersistence` | +| `session-persistence-jsonl/` | JSONL-sidecar persistence backend | (registers `ctx.sessionPersistence`) | +| `session-persistence-sqlite/` | SQLite persistence backend | (registers `ctx.sessionPersistence`) | -## Service API (`ctx.sessionPersistence`) - -| Method | Contract | -|---|---| -| `create(meta): Promise` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). | -| `append(id, events): Promise` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. | -| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. | -| `list(): Promise` | Lightweight listing from metadata, no full-log parse. | -| `has(id)` / `delete(id)` | Existence / removal. A zero-event lazily-materialized session is absent from `has`/`list`. | -| `update(id, summary): Promise` | Update mutable `SessionSummary` fields without touching the append-only log. | - -## Invariants every backend must honor - -- **Append-only; a crashed turn is closed, not truncated.** Committed events (at or below a flushed `turn/end`) are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; `load` preserves them and durably appends synthetic closers (an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}`) to balance the log and keep the rehydrated history a valid provider transcript. Only a never-fully-written torn tail fragment is discarded. -- **Contiguous seq.** `load` rejects a `seq` gap/parse error in the MIDDLE of the log; `append`'s first `seq` must equal the stored next-seq. -- **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable). -- **Durability.** `append` returns only once the batch is durable. - -## Testing backends - -Import `runPersistenceContract` from `tests/contract.ts` and call it with a factory that yields a fresh, empty backend plus a teardown. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics; a backend's own spec adds implementation-specific tests (crash repair, path sanitization) on top. - -Two backends run this suite: `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). Both passing the same contract is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store. - -## Metadata types - -Re-exported from `dsh-session`: `SessionHeader` (immutable: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`), `SessionSummary` (mutable: `updatedAt`, `title?`, `firstPrompt?`), `SessionMeta` (their intersection). +The interface lives at `session-persistence/session-persistence/`; backends are flat siblings. A new storage backend joins here and registers on `ctx.sessionPersistence`. See [session persistence](../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md). diff --git a/packages/session-persistence-jsonl/README.md b/packages/session-persistence/session-persistence-jsonl/README.md similarity index 84% rename from packages/session-persistence-jsonl/README.md rename to packages/session-persistence/session-persistence-jsonl/README.md index ee83eb2927..b8df12e547 100644 --- a/packages/session-persistence-jsonl/README.md +++ b/packages/session-persistence/session-persistence-jsonl/README.md @@ -1,6 +1,6 @@ # @deepseek-ai/dsh-session-persistence-jsonl -The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session plus a small atomic `.summary.json` sidecar for mutable metadata. +The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session. ## On-disk layout @@ -8,7 +8,6 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence / cwd-/ # per-project bucket (or _no-cwd/ when no cwd) .jsonl # header line + one SessionEvent per line (verbatim) - .summary.json # mutable SessionSummary (atomic temp-write + rename) ``` - The first `.jsonl` line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession? }`; every subsequent line is one `SessionEvent` JSON, **verbatim including `assistant/chunk`** so `seq` stays contiguous (`events[i].seq === i`). @@ -22,11 +21,11 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence ## Durability and crash semantics -- **Lazy materialization.** `create(meta)` writes nothing; the `.jsonl` (header + first batch) is written atomically (temp-write + `fsync` + rename) on the first `append`. A created-but-never-appended session leaves nothing on disk and is absent from `has`/`list`. +- **Lazy materialization.** `create(meta)` writes nothing; the `.jsonl` (header + first batch) is written atomically (temp-write + `fsync` + rename) on the first `append`. A created-but-never-appended session leaves nothing on disk and is absent from `list`. - **Append-only.** Committed events (at or below a flushed `turn/end`) are never rewritten. Subsequent appends are line appends at EOF + `fsync`. -- **Crash recovery — close, don't truncate.** A crash can leave a log whose final turn never closed (real events after the last `turn/end`). `load` PRESERVES those events (a turn can be huge — they are real work) and closes the orphaned turn by durably appending synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered (the loop logs the assistant message before running the tools, so a mid-tool crash leaves dangling calls — and `deriveMessages()` would replay an assistant tool-call with no result, which providers reject), then a `step/end` if a step was open, then `turn/end {kind:'interrupted'}`, returning a balanced log. Only a never-fully-written **torn tail fragment** (a final line with no newline / unparseable) is `ftruncate`d away before the closers are written. See [session persistence](../../docs/rfc/implemented/2026-06-14-session-persistence.md). +- **Crash recovery — close, don't truncate.** A crash can leave a log whose final turn never closed (real events after the last `turn/end`). `load` PRESERVES those events (a turn can be huge — they are real work) and closes the orphaned turn by durably appending synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered (the loop logs the assistant message before running the tools, so a mid-tool crash leaves dangling calls — and `deriveMessages()` would replay an assistant tool-call with no result, which providers reject), then a `step/end` if a step was open, then `turn/end {kind:'interrupted'}`, returning a balanced log. Only a never-fully-written **torn tail fragment** (a final line with no newline / unparseable) is `ftruncate`d away before the closers are written. See [session persistence](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md). - **Contiguous-seq.** `load` rejects a mid-log parse error or `seq` gap (unloadable); `append` rejects a batch whose first `seq` does not continue the stored log, and rejects non-JSON-serializable `event.data` naming the offending event type. -- **Format version.** Only v1 is supported; `load` rejects an unknown version. A future format change requires a version bump + migration. +- **Format version.** Only the current `SESSION_FORMAT_VERSION` (v0) is supported; `load` rejects any other version. While the harness is unreleased the on-disk format is pre-release/unstable: a breaking format change is absorbed at v0 (no bump until the first tagged release) and non-current logs are rejected — there is no migration (no persisted user data to preserve). ## Write path diff --git a/packages/session-persistence-jsonl/package.json b/packages/session-persistence/session-persistence-jsonl/package.json similarity index 83% rename from packages/session-persistence-jsonl/package.json rename to packages/session-persistence/session-persistence-jsonl/package.json index 6193af910b..ac18a38838 100644 --- a/packages/session-persistence-jsonl/package.json +++ b/packages/session-persistence/session-persistence-jsonl/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/session-persistence-jsonl/src/format.ts b/packages/session-persistence/session-persistence-jsonl/src/format.ts similarity index 88% rename from packages/session-persistence-jsonl/src/format.ts rename to packages/session-persistence/session-persistence-jsonl/src/format.ts index 1a410d9b75..32258c6a37 100644 --- a/packages/session-persistence-jsonl/src/format.ts +++ b/packages/session-persistence/session-persistence-jsonl/src/format.ts @@ -2,15 +2,15 @@ * On-disk format helpers for the JSONL session-persistence backend: path * sanitization (a {@link SessionId} is an unvalidated branded string, so it * MUST be encoded before use in a path — no traversal, no collision), the - * per-cwd directory layout, header-line (de)serialization, the atomic sidecar - * for mutable summary fields, and the truncation-repair offset computation. + * per-cwd directory layout, header-line (de)serialization, and the + * truncation-repair offset computation. * * @module dsh-session-persistence-jsonl/format */ import { createHash } from 'node:crypto' import { join } from 'node:path' -import type { SessionEvent, SessionHeader, SessionId, SessionMeta } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session' /** * The first line of a session's `.jsonl` file: the immutable @@ -109,11 +109,6 @@ export function logPath(root: string, cwd: string | undefined, id: SessionId): s return join(sessionDir(root, cwd), `${encodeSegment(id)}.jsonl`) } -/** The mutable-summary sidecar path for a session (beside its log). */ -export function sidecarPath(root: string, cwd: string | undefined, id: SessionId): string { - return join(sessionDir(root, cwd), `${encodeSegment(id)}.summary.json`) -} - /** Serialize one event as a JSONL line (no trailing newline). */ export function eventLine(event: SessionEvent): string { return JSON.stringify(event) @@ -138,7 +133,7 @@ export function eventLine(event: SessionEvent): string { * (`Session.append` enforces it): only the final turn can be open, so the * preserved tail is at most one unclosed turn. */ -export function scanLog(buffer: Buffer): { meta: SessionMeta; events: SessionEvent[]; committedBytes: number } { +export function scanLog(buffer: Buffer): { meta: SessionHeader; events: SessionEvent[]; committedBytes: number } { const text = buffer.toString('utf8') // Split into complete (newline-terminated) lines, tracking the byte offset of // each line's end so the truncation point is exact (multi-byte chars make the @@ -233,26 +228,16 @@ export function scanLog(buffer: Buffer): { meta: SessionMeta; events: SessionEve // synthetic closers + new events. const lastPreserved = parsed[preserved.length - 1] const committedBytes = preserved.length > 0 && lastPreserved ? lastPreserved.endByte : headerEntry.endByte - return { meta: metaFrom(headerLine), events: preserved, committedBytes } -} - -/** Build the load-time {@link SessionMeta} from a header line (summary overlaid later). */ -function metaFrom(headerLine: HeaderLine): SessionMeta { - return { - ...fromHeaderLine(headerLine), - updatedAt: headerLine.createdAt, // overlaid by the sidecar in load() - } + return { meta: fromHeaderLine(headerLine), events: preserved, committedBytes } } /** - * Parse just the header line of a log into load-time {@link SessionMeta}, or + * Parse just the header line of a log into a {@link SessionHeader}, or * `undefined` if it is missing/not a header. Used by `list()` to read session * metadata WITHOUT parsing the whole log: a session picker scales with the - * number of sessions, not the total size of every conversation. The summary - * sidecar is overlaid by the caller; `updatedAt` here mirrors `createdAt` until - * then (same as {@link scanLog}'s load-time meta). + * number of sessions, not the total size of every conversation. */ -export function parseHeaderMeta(firstLine: string): SessionMeta | undefined { +export function parseHeaderMeta(firstLine: string): SessionHeader | undefined { let parsed: unknown try { parsed = JSON.parse(firstLine) @@ -260,5 +245,5 @@ export function parseHeaderMeta(firstLine: string): SessionMeta | undefined { return undefined } if (!isHeaderLine(parsed)) return undefined - return metaFrom(parsed) + return fromHeaderLine(parsed) } diff --git a/packages/session-persistence/session-persistence-jsonl/src/index.ts b/packages/session-persistence/session-persistence-jsonl/src/index.ts new file mode 100644 index 0000000000..6de7eafeb3 --- /dev/null +++ b/packages/session-persistence/session-persistence-jsonl/src/index.ts @@ -0,0 +1,381 @@ +/** + * JSONL durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-jsonl`). + * + * One append-only `.jsonl` event log per session (a header line then one + * `SessionEvent` per line, verbatim including `assistant/chunk` so `seq` stays + * contiguous), with lazy materialization (no file until the first `append`), + * atomic first write, and load-time repair of a never-committed crash tail. + * + * The backend supplies ONLY the file-bytes storage primitives (the + * {@link PersistenceBackend} hooks below); all the write-path orchestration + * (the `session/event` → buffer → `session/flush` drain, per-session + * serialization, write cursors, fork-seed persistence, HMR live-adoption, + * crash-repair sequencing, dispose quiescence) lives in the backend-agnostic + * {@link PersistenceCoordinator} this class composes. The four public + * {@link SessionPersistence} methods delegate to the coordinator. + * + * @module @deepseek-ai/dsh-session-persistence-jsonl + */ + +import { Context } from 'cordis' +import z from 'schemastery' +import { open, mkdir, readFile, readdir, link, rm, truncate } from 'node:fs/promises' +import { dirname, resolve } from 'node:path' +import { randomBytes } from 'node:crypto' +import { + SessionPersistence, PersistenceCoordinator, + type PersistenceBackend, type StoredPrefix, +} from '@deepseek-ai/dsh-session-persistence' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' +import { + encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine, +} from './format.ts' + +export interface Config { + /** + * Root directory for all session files. Required (no default): a default of + * `process.cwd()` would scatter session files as the process's cwd changes + * (bash calls, subprocesses). Sessions group under per-cwd subdirectories. + */ + root: string +} + +/** + * Whether `error` is a "no such file/directory" (`ENOENT`) failure — the ONLY + * filesystem error that legitimately means "this session/root is absent" for a + * durable backend. Any OTHER error (`EACCES`, `ENOTDIR`, transient I/O) must + * surface rather than be silently reported as absence. (A NodeJS filesystem + * rejection carries a string `code`.) + */ +function isENOENT(error: unknown): boolean { + return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT' +} + +/** + * The JSONL persistence backend. Load as a plugin; it registers as + * `ctx.sessionPersistence` and (via the coordinator) installs the write-path + * listeners. Its torn-tail marker is the byte offset to truncate the log to. + */ +export class SessionPersistenceJsonl extends SessionPersistence implements PersistenceBackend { + static inject = ['sessions'] + + static Config: z = z.object({ + root: z.string().required(), + }) + + /** + * Backend label for the coordinator's dispose-failure AggregateError and + * effect name. NOTE: this intentionally shadows cordis `Service.name` (which + * the base sets to `'sessionPersistence'`). The service is registered under the + * fixed key the Service constructor captured (`reflect.provide('sessionPersistence', …)`), + * not via `this.name`, so overwriting the instance field with the backend label + * does not affect `ctx.sessionPersistence` resolution — it only relabels the + * dispose diagnostics, which is exactly what {@link PersistenceBackend.name} is for. + */ + override readonly name = 'session-persistence-jsonl' + + private root: string + private coordinator: PersistenceCoordinator + + constructor(ctx: Context, public config: Config) { + super(ctx) + // Resolve the configured root to an ABSOLUTE path ONCE, here. A relative root + // would otherwise re-resolve against `process.cwd()` at every later + // readdir/open — so if any plugin or test changed cwd between create, append, + // and load, one session's files could split across directories. + this.root = resolve(config.root) + this.coordinator = new PersistenceCoordinator(this.ctx, this) + } + + // --- SessionPersistence service surface (delegated to the coordinator) --- + + create(meta: SessionHeader): Promise { + return this.coordinator.create(meta) + } + + append(id: SessionId, events: readonly SessionEvent[]): Promise { + return this.coordinator.append(id, events) + } + + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.coordinator.load(id) + } + + // `list` is BOTH the public service method and the PersistenceBackend hook — + // one method, the bucket walk below. The coordinator adds no orchestration for + // listing (no per-id serialization, no cursor), so it would just call back into + // this same method; routing it through the coordinator would recurse. Defined + // once, in the "PersistenceBackend hooks" section. + + /** + * The per-session init promises, exposed for white-box tests that await a + * specific session's onCreated (there is no public API to await one init). + */ + get inits(): Map> { + return this.coordinator.inits + } + + // --- PersistenceBackend hooks (the file-bytes storage primitives) --- + + /** Read a stored prefix by id across ALL cwd buckets (cwd unknown). */ + async loadStored(id: SessionId): Promise | undefined> { + const file = await this.findLog(id) + if (file === undefined) return undefined + return this.readPrefix(file.path) + } + + /** + * Read a stored prefix SCOPED to `cwd` (HMR live-adoption must not cross cwd). + * `undefined` is the DEFINITE "no-cwd" bucket, NOT "unknown" — a live session + * with no cwd may only adopt a persisted no-cwd log, never a same-id log that + * lives in some other cwd bucket. So this looks at exactly `logPath(cwd)` + * (which maps `undefined` → the `_no-cwd` bucket), never the all-buckets scan. + */ + async loadLive(id: SessionId, cwd: string | undefined): Promise | undefined> { + const path = logPath(this.root, cwd, id) + if (!await this.exists(path)) return undefined + return this.readPrefix(path) + } + + /** + * Read and scan a session's log file into a {@link StoredPrefix}. Folds the + * torn-tail comparison HERE so the `tornMarker` is the byte offset to truncate + * to (or `undefined` when nothing is torn) — the coordinator never sees the + * raw byteLength. + */ + private async readPrefix(path: string): Promise> { + const buffer = await readFile(path) + const { meta, events, committedBytes } = scanLog(buffer) + return { + meta, + events, + ...committedBytes < buffer.byteLength ? { tornMarker: committedBytes } : {}, + } + } + + /** Durably append a batch, lazily materializing the file when not yet present. */ + async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise { + if (isMaterialized) { + await this.appendLines(meta, events) + } else { + await this.materialize(meta, events) + } + } + + /** + * Make a crash repair durable: truncate the torn tail to `tornMarker` bytes (if + * any), then append the synthetic `closers` (if any). Two fsync'd steps — the + * seam does not require this to be atomic. + */ + async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise { + if (tornMarker !== undefined) await this.repair(meta, tornMarker) + if (closers.length > 0) await this.appendLines(meta, closers) + } + + /** List all stored sessions' metadata (header line only — no full-log parse). */ + async list(): Promise { + const metas: SessionHeader[] = [] + for (const dir of await this.listCwdDirs()) { + for (const name of await this.listJsonl(dir)) { + // Read ONLY the header line, not the whole log: a session picker must + // scale with the number of sessions, not the total size of every + // conversation (the log persists every assistant/chunk verbatim). + const first = await this.readFirstLine(`${dir}/${name}`) + if (first === undefined) continue // empty/half-written file + const meta = parseHeaderMeta(first) + if (meta === undefined) continue // not a session header + metas.push(meta) + } + } + return metas + } + + // --- materialization / append / repair (file mechanics) --- + + /** Atomically write the header line + first batch (temp-write, fsync, rename). */ + private async materialize(meta: SessionHeader, events: readonly SessionEvent[]): Promise { + const dir = sessionDir(this.root, meta.cwd) + await mkdir(this.root, { recursive: true, mode: 0o700 }) + await this.syncDir(dirname(this.root)) + await mkdir(dir, { recursive: true, mode: 0o700 }) + await this.syncDir(this.root) + const finalPath = logPath(this.root, meta.cwd, meta.id) + // Never rename over an existing committed log: materialize is the FIRST write + // of a session the backend believes is new. A file here means a different + // session shares this id on disk — reject loudly. (createCore already guards + // the create path, so this is unreachable-in-practice TOCTOU defense.) + /* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */ + if (await this.exists(finalPath)) { + throw new Error(`refusing to materialize "${meta.id}": a log already exists on disk (load/resume it instead)`) + } + const header = JSON.stringify(toHeaderLine(meta)) + const body = events.map(eventLine).join('\n') + const content = header + '\n' + body + '\n' + + const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp` + const handle = await open(tmp, 'wx', 0o600) + try { + await handle.writeFile(content) + await handle.sync() + } finally { + await handle.close() + } + // Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the + // final path already exists, so two processes materializing the same id + // concurrently cannot clobber each other. rename() would silently overwrite. + let linked = false + try { + await link(tmp, finalPath) + linked = true + } finally { + // If link FAILED, the temp is the only reference and must be removed before + // the original error propagates. If it SUCCEEDED, defer temp cleanup to + // AFTER the publish is durable (below) so a temp-rm failure can never reject + // a session whose log already published. + /* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */ + if (!linked) await rm(tmp, { force: true }) + } + // link() succeeded — the log is published. fsync the directory so the new + // entry survives a power loss: the new link is not crash-durable until the + // parent directory's metadata is synced. + await this.syncDir(dir) + // Best-effort temp cleanup: the log is already published and durable, so a + // failure to remove the (now-redundant) temp hard link must NOT reject the + // append. Swallow only the rm failure; nothing else of consequence runs here. + try { + await rm(tmp, { force: true }) + } catch { + /* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */ + } + } + + /** fsync a directory so a just-created/renamed entry inside it is crash-durable. */ + private async syncDir(dir: string): Promise { + const handle = await open(dir, 'r') + try { + await handle.sync() + } finally { + await handle.close() + } + } + + /** + * Append event lines at EOF and fsync. On a write/sync failure AFTER the kernel + * accepted some bytes (ENOSPC, an fsync error), truncate the file back to its + * pre-append size before rethrowing: the cursor is unchanged, so the batch will + * be retried, and without this rollback the retry would append AFTER the partial + * bytes — producing duplicate seqs that make `scanLog` see a gap. + */ + private async appendLines(meta: SessionHeader, events: readonly SessionEvent[]): Promise { + const path = logPath(this.root, meta.cwd, meta.id) + const handle = await open(path, 'a') + try { + const { size: before } = await handle.stat() + try { + await handle.writeFile(events.map(eventLine).join('\n') + '\n') + await handle.sync() + } catch (error) { + // Roll back whatever bytes landed so a retry starts from a clean EOF. + await handle.truncate(before) + await handle.sync() + throw error + } + } finally { + await handle.close() + } + } + + /** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */ + private async repair(meta: SessionHeader, offset: number): Promise { + const path = logPath(this.root, meta.cwd, meta.id) + await truncate(path, offset) + const handle = await open(path, 'r+') + try { + await handle.sync() + } finally { + await handle.close() + } + } + + // --- discovery helpers --- + + /** + * Read the first newline-terminated line of a file without loading the whole + * file. Returns undefined if the file is empty or has no complete first line. + * Reads in bounded chunks so a huge log costs only the header read. + */ + private async readFirstLine(path: string): Promise { + const handle = await open(path, 'r') + try { + const chunks: Buffer[] = [] + const buf = Buffer.alloc(8192) + for (;;) { + const { bytesRead } = await handle.read(buf, 0, buf.length, null) + if (bytesRead === 0) return undefined // EOF with no newline → no complete line + const slice = buf.subarray(0, bytesRead) + const nl = slice.indexOf(0x0a) + if (nl !== -1) { + chunks.push(slice.subarray(0, nl)) + return Buffer.concat(chunks).toString('utf8') + } + chunks.push(Buffer.from(slice)) + } + } finally { + await handle.close() + } + } + + /** + * Find a session's log file by id across ALL cwd buckets — the any-cwd scan + * for `loadStored` (resume identifies a session by id alone). The cwd-scoped + * lookup (`loadLive`) does NOT use this; it goes straight to `logPath(cwd)` so + * a no-cwd session can't match a real-cwd bucket. + */ + private async findLog(id: SessionId): Promise<{ path: string; cwd: string | undefined } | undefined> { + const target = encodeSegment(id) + '.jsonl' + for (const dir of await this.listCwdDirs()) { + const path = `${dir}/${target}` + if (await this.exists(path)) { + // Recover the cwd from the header so the caller has the session's bucket. + const { meta } = scanLog(await readFile(path)) + return { path, cwd: meta.cwd } + } + } + return undefined + } + + /** The cwd-bucket directories under the root (absolute paths). */ + private async listCwdDirs(): Promise { + try { + const entries = await readdir(this.root, { withFileTypes: true }) + return entries.filter(e => e.isDirectory()).map(e => `${this.root}/${e.name}`) + } catch (error) { + // ENOENT = the root has not been created yet → genuinely no sessions. Any + // other error (EACCES, ENOTDIR, transient I/O) must NOT be reported as "no + // sessions" — a durable backend cannot silently pretend state is absent. + if (isENOENT(error)) return [] + throw error + } + } + + private async listJsonl(dir: string): Promise { + const entries = await readdir(dir) + return entries.filter(n => n.endsWith('.jsonl')) + } + + private async exists(path: string): Promise { + try { + const handle = await open(path, 'r') + await handle.close() + return true + } catch (error) { + // Only ENOENT means absent. A permission/I/O error must surface, not be + // collapsed to `false` — otherwise load() reports "not found" and collision + // checks proceed under a false absence assumption. + if (isENOENT(error)) return false + throw error + } + } +} + +export default SessionPersistenceJsonl diff --git a/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts new file mode 100644 index 0000000000..1df70f9c9a --- /dev/null +++ b/packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts @@ -0,0 +1,704 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' +import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts' +import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' + +let root: string +const dirs: string[] = [] + +async function freshRoot(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-')) + dirs.push(dir) + return dir +} + +afterEach(async () => { + for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) +}) + +// Run the shared backend contract against the real JSONL backend. +runPersistenceContract('jsonl', async () => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-')) + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir }) + return { + persistence: ctx.sessionPersistence, + dispose: async () => { + await fiber.dispose() + await rm(dir, { recursive: true, force: true }) + }, + } +}) + +// Run the shared coordinator orchestration suite against the real JSONL backend. +// One temp root is the shared storage scope (two mounted instances over the same +// root = HMR/reload). `corruptTail` appends a partial, newline-less fragment to +// the session's .jsonl past the committed region — a never-committed torn tail +// that drives the coordinator's commitRepair-with-tornMarker branch over real +// file bytes. +runCoordinatorContract('jsonl', async (): Promise => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-coord-')) + return { + mount: async (ctx) => { + const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir }) + return fiber + }, + corruptTail: async (id, cwd) => { + // A half-written record with no trailing newline: scanLog treats it as an + // uncommitted crash fragment and reports committedBytes < byteLength, so + // the coordinator sees a tornMarker to truncate. + await appendFile(logPath(dir, cwd, id), '{"type":"assistant/chunk","seq":8,"ti') + }, + cleanup: async () => { await rm(dir, { recursive: true, force: true }) }, + } +}) + +describe('SessionPersistenceJsonl: format helpers', () => { + it('encodeSegment neutralizes traversal, separators, and absolute paths', () => { + expect(encodeSegment('..')).toBe('~002E~002E') + expect(encodeSegment('.')).toBe('~002E') + expect(encodeSegment('a/b')).toBe('a~002Fb') + expect(encodeSegment('/etc/passwd')).toBe('~002Fetc~002Fpasswd') + expect(encodeSegment('a\u0000b')).toBe('a~0000b') + expect(encodeSegment('plain-ID_1.2')).toBe('plain-ID_1.2') // safe chars pass through + expect(encodeSegment('a~b')).toBe('a~007Eb') // ~ itself is escaped + }) + + it('encodeSegment is injective over UTF-16, incl. lone surrogates', () => { + // Distinct lone surrogates must NOT collide (Buffer.from would normalize + // both to U+FFFD; code-unit escaping keeps them distinct). + const hi = encodeSegment(String.fromCharCode(0xD800)) + const lo = encodeSegment(String.fromCharCode(0xDC00)) + expect(hi).toBe('~D800') + expect(lo).toBe('~DC00') + expect(hi).not.toBe(lo) + // A literal "~002F" input cannot collide with the encoding of "/". + expect(encodeSegment('~002F')).not.toBe(encodeSegment('/')) + }) + + it('encodeSegment rejects an empty id', () => { + expect(() => encodeSegment('')).toThrow(/empty/) + }) +}) + +describe('SessionPersistenceJsonl: durability and crash semantics', () => { + let ctx: Context + beforeEach(async () => { + root = await freshRoot() + ctx = new Context() + await ctx.plugin(SessionStore) + await ctx.plugin(SessionPersistenceJsonl, { root }) + }) + afterEach(async () => { await ctx.fiber.dispose() }) + + it('lazy materialization: create() writes no file until the first append', async () => { + const m = meta('lazy', '/work') + await ctx.sessionPersistence.create(m) + // nothing on disk yet + const dir = sessionDir(root, '/work') + await expect(stat(logPath(root, '/work', m.id))).rejects.toThrow() + expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id) + + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + // now materialized + expect((await stat(logPath(root, '/work', m.id))).isFile()).toBe(true) + expect((await ctx.sessionPersistence.list()).map(h => h.id)).toContain(m.id) + void dir + }) + + it('round-trip is byte-identical (incl. assistant/chunk verbatim)', async () => { + const m = meta('chunks') + const log: SessionEvent[] = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'assistant/chunk', seq: 2, time: 3, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'he' } } }, + { type: 'assistant/chunk', seq: 3, time: 4, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'llo' } } }, + { type: 'assistant/message', seq: 4, time: 5, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] } }, + { type: 'step/end', seq: 5, time: 6, data: { turn: 1, step: 1 } }, + { type: 'turn/end', seq: 6, time: 7, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, log) + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events).toEqual(log) // chunks preserved, contiguous seqs + }) + + it('crash recovery: load preserves the interrupted turn and closes it with a synthetic turn/end {interrupted}', async () => { + const m = meta('crash', '/proj') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5, turn/end at 5 + + // Simulate a crash mid-second-turn: append raw lines that are NOT closed by + // a turn/end (turn/start + step/start are fully written), plus a final + // partial line with no newline (a torn fragment never fully flushed). + const path = logPath(root, '/proj', m.id) + await writeFile(path, [ + JSON.stringify({ type: 'turn/start', seq: 6, time: 8, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }), + JSON.stringify({ type: 'step/start', seq: 7, time: 9, data: { turn: 2, step: 1 } }), + '{"type":"assistant/chunk","seq":8,"ti', // truncated partial line (no newline) + ].join('\n'), { flag: 'a' }) + + // load PRESERVES the interrupted turn's real events (turn/start 6, step/start + // 7) — a turn can be huge, so they must not be truncated — and durably closes + // the orphaned turn with synthetic step/end (8) + turn/end {interrupted} (9). + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) + const last = loaded.events.at(-1)! + expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) + const stepEnd = loaded.events[8]! + expect(stepEnd.type).toBe('step/end') + // the torn seq-8 chunk fragment did not survive + expect(loaded.events.some(e => e.type === 'assistant/chunk' && e.seq === 8)).toBe(false) + + // The next append continues at seq 10 (the balanced length). + const turn3 = [ + { type: 'turn/start', seq: 10, time: 11, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 11, time: 12, data: { turn: 3, reason: { kind: 'completed' } } }, + ] as SessionEvent[] + await ctx.sessionPersistence.append(m.id, turn3) + const reloaded = await ctx.sessionPersistence.load(m.id) + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) + }) + + it('committed events are never rewritten: only the crash tail is repaired', async () => { + const m = meta('append-only') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const before = await readFile(logPath(root, undefined, m.id), 'utf8') + const committedPrefix = before // the whole committed log + + // A crash tail then a repair-append. + await writeFile(logPath(root, undefined, m.id), '\n{"partial', { flag: 'a' }) + await ctx.sessionPersistence.load(m.id) + await ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, + ] as SessionEvent[]) + const after = await readFile(logPath(root, undefined, m.id), 'utf8') + // the committed prefix is byte-for-byte intact at the head of the file + expect(after.startsWith(committedPrefix)).toBe(true) + }) + + it('a failed appendLines truncates partial bytes so a retry has no seq gap', async () => { + const m = meta('truncate-retry') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) // materialized, seqs 0..5 + const sizeBefore = (await stat(logPath(root, undefined, m.id))).size + + // Force the NEXT fsync (inside appendLines) to fail once, AFTER writeFile + // has already put bytes on disk — simulating an ENOSPC/fsync error + // mid-append. The recovery truncate() also fsyncs, so allow that one. + const handle = await (await import('node:fs/promises')).open(logPath(root, undefined, m.id), 'r') + const proto = Object.getPrototypeOf(handle) as { sync: () => Promise } + await handle.close() + const realSync = proto.sync + let failed = false + const spy = vi.spyOn(proto, 'sync').mockImplementation(async function (this: unknown) { + if (!failed) { failed = true; throw new Error('simulated fsync ENOSPC') } + return realSync.call(this) + }) + + const turn2 = [ + { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, + ] as SessionEvent[] + // The append rejects, but the partial bytes are truncated back: the file is + // its pre-append size and the cursor is unchanged. + await expect(ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/ENOSPC/) + expect((await stat(logPath(root, undefined, m.id))).size).toBe(sizeBefore) + spy.mockRestore() + + // The retry now succeeds with NO seq gap — the log is contiguous 0..7. + await ctx.sessionPersistence.append(m.id, turn2) + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + }) + + it('load returns a meta copy: mutating it does not corrupt backend pathing', async () => { + const m = meta('meta-copy', '/proj') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const loaded = await ctx.sessionPersistence.load(m.id) + // A consumer mutates the returned meta's cwd. The backend's stored pathing + // metadata must be unaffected, so a later append still finds the right log. + loaded.meta.cwd = '/evil' + await ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, + ] as SessionEvent[]) + // The append landed in the ORIGINAL /proj log, not beside an /evil path. + const reloaded = await ctx.sessionPersistence.load(m.id) + expect(reloaded.meta.cwd).toBe('/proj') + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + }) + + it('rejects a re-append of an already-stored seq', async () => { + const m = meta('reappend') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toThrow(/seq mismatch/) + }) + + it('path-traversal session ids are neutralized (no escape from root)', async () => { + const evil = SessionId('../../etc/pwn') + const m = { version: 0, id: evil, createdAt: 1 } + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(evil, oneTurnLog()) + // The file lives UNDER root, not at ../../etc. + const all: string[] = [] + async function walk(dir: string): Promise { + for (const e of await readdir(dir, { withFileTypes: true })) { + const p = join(dir, e.name) + if (e.isDirectory()) await walk(p) + else all.push(p) + } + } + await walk(root) + expect(all.length).toBeGreaterThan(0) + expect(all.every(p => p.startsWith(root))).toBe(true) + }) +}) + +describe('SessionPersistenceJsonl: write path (session/event → flush)', () => { + it('concurrent sessions do not cross buffers', async () => { + root = await freshRoot() + const ctx = new Context() + await ctx.plugin(SessionStore) + await ctx.plugin(SessionPersistenceJsonl, { root }) + + const a = ctx.sessions.create(SessionId('sa')) + const b = ctx.sessions.create(SessionId('sb')) + a.append('user/message', { content: [{ type: 'text', text: 'A' }], source: { kind: 'user' } }) + b.append('user/message', { content: [{ type: 'text', text: 'B' }], source: { kind: 'user' } }) + a.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + b.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', a) + await ctx.parallel('session/flush', b) + + const la = await ctx.sessionPersistence.load(SessionId('sa')) + const lb = await ctx.sessionPersistence.load(SessionId('sb')) + expect(JSON.stringify(la.events)).toContain('"A"') + expect(JSON.stringify(la.events)).not.toContain('"B"') + expect(JSON.stringify(lb.events)).toContain('"B"') + expect(JSON.stringify(lb.events)).not.toContain('"A"') + await ctx.fiber.dispose() + }) + +}) + + +describe('SessionPersistenceJsonl: scanLog unit', () => { + it('rejects a header-less / empty log', () => { + expect(() => scanLog(Buffer.from(''))).toThrow() + }) + + it('rejects a corrupt header line', () => { + expect(() => scanLog(Buffer.from('not json\n'))).toThrow(/header/) + }) + + it('rejects a non-session first line', () => { + expect(() => scanLog(Buffer.from('{"type":"event"}\n'))).toThrow(/session header/) + }) + + it('a seq gap after the last turn/end bounds the preserved tail (torn fragment tolerated)', () => { + const log = [ + JSON.stringify({ type: 'session', version: 0, id: 'g', createdAt: 1 }), + JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), + JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1 + ].join('\n') + '\n' + // No committed turn/end, so the gap is a tolerated crash boundary: scanLog + // PRESERVES the contiguous prefix (turn/start seq 0) — real interrupted-turn + // work, not discarded — and stops at the gap. The orphaned open turn is + // closed by loadCore's synthetic turn/end, not here. + expect(scanLog(Buffer.from(log)).events.map(e => e.seq)).toEqual([0]) + }) + + it('rejects a seq gap BEFORE a later committed turn/end (committed data damaged)', () => { + const log = [ + JSON.stringify({ type: 'session', version: 0, id: 'g2', createdAt: 1 }), + JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), + JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1 + JSON.stringify({ type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }), + ].join('\n') + '\n' + // A turn/end exists, so the prefix up to it is committed — but it has a hole. + // Truncating it would silently drop committed data → unloadable. + expect(() => scanLog(Buffer.from(log))).toThrow(/seq gap in committed region/) + }) + + it('rejects a corrupt line BEFORE a later committed turn/end (committed data damaged)', () => { + const log = [ + JSON.stringify({ type: 'session', version: 0, id: 'c', createdAt: 1 }), + '{not json', // corrupt, sits in the committed region (a turn/end follows) + JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }), + ].join('\n') + '\n' + expect(() => scanLog(Buffer.from(log))).toThrow(/unparsable committed event/) + }) + + it('a header-only log (no event lines at all) preserves nothing — committedBytes is the header', () => { + const log = JSON.stringify({ type: 'session', version: 0, id: 'h0', createdAt: 1 }) + '\n' + const scanned = scanLog(Buffer.from(log)) + expect(scanned.events).toEqual([]) + // committedBytes falls back to the header line's end (no preserved events). + expect(scanned.committedBytes).toBe(Buffer.byteLength(log, 'utf8')) + }) + + it('a corrupt line after the last turn/end bounds the preserved tail', () => { + const log = [ + JSON.stringify({ type: 'session', version: 0, id: 'c2', createdAt: 1 }), + JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), + '{not json', // corrupt crash fragment, no turn/end committed + ].join('\n') + '\n' + // The contiguous prefix (turn/start seq 0) is preserved; the corrupt + // fragment after it is the tolerated crash boundary. + expect(scanLog(Buffer.from(log)).events.map(e => e.seq)).toEqual([0]) + }) + + it('tolerates a seq gap AFTER a turn/end (uncommitted tail)', () => { + const log = [ + JSON.stringify({ type: 'session', version: 0, id: 't', createdAt: 1 }), + JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }), + JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }), + JSON.stringify({ type: 'step/start', seq: 9, time: 3, data: { turn: 2, step: 1 } }), // gap in uncommitted tail + ].join('\n') + '\n' + const { events } = scanLog(Buffer.from(log)) + expect(events.map(e => e.seq)).toEqual([0, 1]) // tail dropped + }) +}) + +describe('SessionPersistenceJsonl: edge cases', () => { + let ctx: Context + beforeEach(async () => { + root = await freshRoot() + ctx = new Context() + await ctx.plugin(SessionStore) + await ctx.plugin(SessionPersistenceJsonl, { root }) + }) + afterEach(async () => { await ctx.fiber.dispose() }) + + it('append rejects non-JSON-serializable undefined-producing data', async () => { + const m = meta('undef') + await ctx.sessionPersistence.create(m) + // A value whose JSON.stringify yields undefined (a bare function as data). + const bad = [{ type: 'user/message', seq: 0, time: 1, data: (() => 0) as unknown }] as unknown as SessionEvent[] + await expect(ctx.sessionPersistence.append(m.id, bad)).rejects.toThrow(/non-JSON-serializable/) + }) + + it('create snapshots its meta: mutating the caller object after the call is ignored', async () => { + const m = meta('create-snap', '/orig') + const p = ctx.sessionPersistence.create(m) + // Mutate the caller's meta object immediately after calling create. + m.cwd = '/mutated' + await p + await ctx.sessionPersistence.append(SessionId('create-snap'), oneTurnLog()) + // The log materialized under the ORIGINAL cwd, not the mutated one. + expect((await stat(logPath(root, '/orig', SessionId('create-snap')))).isFile()).toBe(true) + await expect(stat(logPath(root, '/mutated', SessionId('create-snap')))).rejects.toThrow() + }) + + it('list discovers sessions across multiple cwd buckets', async () => { + await ctx.sessionPersistence.create(meta('p1', '/projA')) + await ctx.sessionPersistence.append(SessionId('p1'), oneTurnLog()) + await ctx.sessionPersistence.create(meta('p2', '/projB')) + await ctx.sessionPersistence.append(SessionId('p2'), oneTurnLog()) + await ctx.sessionPersistence.create(meta('p3')) // no cwd → _no-cwd bucket + await ctx.sessionPersistence.append(SessionId('p3'), oneTurnLog()) + + const ids = (await ctx.sessionPersistence.list()).map(x => x.id).sort() + expect(ids).toEqual(['p1', 'p2', 'p3']) + }) + + it('list on an empty root returns nothing', async () => { + expect(await ctx.sessionPersistence.list()).toEqual([]) + }) + + it('list skips empty and non-header .jsonl files (metadata-only read)', async () => { + // A real session… + await ctx.sessionPersistence.create(meta('real', '/p')) + await ctx.sessionPersistence.append(SessionId('real'), oneTurnLog()) + // …alongside two junk files in the _no-cwd bucket: an EMPTY file (readFirstLine + // returns undefined) and a file whose first line is not a session header + // (parseHeaderMeta returns undefined). Both are skipped, not listed. + const bucket = join(root, '_no-cwd') + await mkdir(bucket, { recursive: true }) + await writeFile(join(bucket, 'empty.jsonl'), '') + await writeFile(join(bucket, 'notheader.jsonl'), '{"type":"turn/start"}\n') + await writeFile(join(bucket, 'badjson.jsonl'), 'not json at all\n') + + const ids = (await ctx.sessionPersistence.list()).map(x => x.id).sort() + expect(ids).toEqual(['real']) + }) + + it('list reads a header line longer than the 8KB read chunk', async () => { + // readFirstLine accumulates across reads when the first line exceeds its + // buffer. Plant a valid header whose line is > 8192 bytes (a long extra + // field is tolerated by the header type guard) and confirm list() reads it. + const bucket = join(root, '_no-cwd') + await mkdir(bucket, { recursive: true }) + const bigHeader = JSON.stringify({ type: 'session', version: 0, id: 'big', createdAt: 1, pad: 'x'.repeat(9000) }) + await writeFile(join(bucket, 'big.jsonl'), bigHeader + '\n') + const ids = (await ctx.sessionPersistence.list()).map(x => x.id) + expect(ids).toContain('big') + }) + + it('a DIFFERENT live session object reusing a disposed id gets its own init (no stale cache)', async () => { + // Session A materializes a log under id "reuse". + const sessFiberA = await ctx.plugin(Object.assign((inner: Context) => { + const a = inner.sessions.create(SessionId('reuse'), { meta: { cwd: '/a' } }) + for (const e of oneTurnLog()) a.append(e.type, e.data) + }, { inject: ['sessions'] })) + // Drain A, then dispose ITS fiber (the live session A is gone) while the + // backend stays loaded. + for (const s of ctx.sessions.list()) await ctx.parallel('session/flush', s) + await sessFiberA.dispose() + + // A NEW live Session object reuses id "reuse". The init cache is keyed by + // the Session OBJECT, so this gets its OWN onCreated (not A's stale promise) + // — which detects the on-disk collision and rejects, rather than silently + // appending the new session's events onto A's log under a stale cursor. + const backend = ctx.sessionPersistence as unknown as { inits: Map> } + let b!: Session + await ctx.plugin(Object.assign((inner: Context) => { + b = inner.sessions.create(SessionId('reuse'), { meta: { cwd: '/a' } }) + }, { inject: ['sessions'] })) + await expect(backend.inits.get(b)).rejects.toThrow(/already bound to a different live session|already has a persisted log on disk/) + }) + + it('a NO-CWD live session does NOT cross-cwd-adopt a same-id log from a real cwd bucket (loadLive is scope-exact)', async () => { + // Backend 1: materialize a log under id "x" in the cwd "/w" bucket, then + // dispose the WHOLE backend (so backend 2 mounts with an EMPTY states map — + // the HMR/reload path where onCreated goes through loadLive, not a tracked + // collision). + await ctx.sessionPersistence.create(meta('x', '/w')) + await ctx.sessionPersistence.append(SessionId('x'), oneTurnLog()) + await ctx.fiber.dispose() + + // Backend 2 over the SAME root. A live no-cwd session reuses id "x". Because + // loadLive(id, undefined) is the DEFINITE no-cwd bucket (NOT an all-buckets + // scan), case-2 adoption does NOT match the "/w" log — so it would NOT + // silently graft the no-cwd events onto the "/w" log with a mismatched cwd + // (the bug a non-scope-exact loadLive caused). It falls through to the + // new-session path, where createCore's any-cwd collision probe (loadStored) + // catches the duplicate id and REJECTS — the id is taken in another bucket. + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root }) + const backend = ctx2.sessionPersistence as unknown as { inits: Map> } + let b!: Session + await ctx2.plugin(Object.assign((inner: Context) => { + b = inner.sessions.create(SessionId('x')) // no cwd + }, { inject: ['sessions'] })) + await expect(backend.inits.get(b)).rejects.toThrow(/already has a persisted log on disk/) + + // The "/w" log is untouched — no no-cwd events were grafted onto it, and no + // `_no-cwd` log for "x" was created. + const inW = scanLog(await readFile(logPath(root, '/w', SessionId('x')))) + expect(inW.meta.cwd).toBe('/w') + expect(inW.events).toHaveLength(6) + await expect(stat(logPath(root, undefined, SessionId('x')))).rejects.toThrow() + await ctx2.fiber.dispose() + }) + + it('a seed with matching seq/type/time but DIFFERENT data is rejected (deep prefix compare)', async () => { + // Materialize and load (ownerless, cursor = 6). + await ctx.sessionPersistence.create(meta('divergent', '/a')) + await ctx.sessionPersistence.append(SessionId('divergent'), oneTurnLog()) + await ctx.sessionPersistence.load(SessionId('divergent')) + + const backend = ctx.sessionPersistence as unknown as { inits: Map> } + // A seed that keeps every seq/type/time but mutates a payload must NOT be + // accepted as "the same session" — otherwise drain filters those seqs as + // already persisted and the divergent payload is silently lost. + const tampered = oneTurnLog() + const userMsg = tampered[1] + if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'DIFFERENT' }] + let bad!: Session + await ctx.plugin(Object.assign((inner: Context) => { + bad = inner.sessions.create(SessionId('divergent'), { seed: tampered, meta: { cwd: '/a' } }) + }, { inject: ['sessions'] })) + await expect(backend.inits.get(bad)).rejects.toThrow(/do not match this live session|already has a persisted log/) + }) + + it('a second live session reusing a bound id is rejected', async () => { + // A live session materializes and owns the id. + const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { + const a = inner.sessions.create(SessionId('bound'), { meta: { cwd: '/a' } }) + a.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + a.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + }, { inject: ['sessions'] })) + for (const s of ctx.sessions.list()) await ctx.parallel('session/flush', s) + await firstFiber.dispose() + + const backend = ctx.sessionPersistence as unknown as { inits: Map> } + let second!: Session + await ctx.plugin(Object.assign((inner: Context) => { + second = inner.sessions.create(SessionId('bound'), { meta: { cwd: '/a' } }) + }, { inject: ['sessions'] })) + await expect(backend.inits.get(second)) + .rejects.toThrow(/already bound to a different live session|already has a persisted log|do not match/) + }) + + it('list returns nothing when the root directory does not exist', async () => { + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root: join(root, 'does-not-exist-yet') }) + expect(await ctx2.sessionPersistence.list()).toEqual([]) + await ctx2.fiber.dispose() + }) + + it('list surfaces a non-ENOENT root error (ENOTDIR) instead of reporting no sessions', async () => { + // A durable backend must NOT collapse a storage fault to "no sessions". Point + // the root at a regular FILE: readdir then fails with ENOTDIR, which must + // propagate rather than be swallowed as an empty listing. + const filePath = join(root, 'not-a-dir') + await writeFile(filePath, 'x') + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root: filePath }) + await expect(ctx2.sessionPersistence.list()).rejects.toThrow(/ENOTDIR/) + await ctx2.fiber.dispose() + }) + + it('loadLive surfaces a non-ENOENT lookup error (ENOTDIR) instead of reporting absent', async () => { + // A non-ENOENT error from the per-id open() must surface, not be collapsed to + // "not found" (which would let live-adoption proceed under a false absence + // assumption). A live session's onCreated reaches loadLive(id, cwd) → + // exists(logPath). Make that cwd's bucket DIRECTORY a regular file: open()ing + // `bucket/.jsonl` under it then fails ENOTDIR. + const cwd = '/x' + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root }) + await writeFile(sessionDir(root, cwd), 'x') // bucket path is now a FILE + const backend = ctx2.sessionPersistence as unknown as { inits: Map> } + let s!: Session + await ctx2.plugin(Object.assign((inner: Context) => { + s = inner.sessions.create(SessionId('exists-fault'), { meta: { cwd } }) + }, { inject: ['sessions'] })) + await expect(backend.inits.get(s)).rejects.toThrow(/ENOTDIR/) + await ctx2.fiber.dispose() + }) + + it('append() to a disk-only session adopts it and repairs a crash tail', async () => { + // Persist a session, then corrupt its tail, all through ONE backend. + const m = meta('disk-append', '/d') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + await writeFile(logPath(root, '/d', m.id), '\n{"partial crash', { flag: 'a' }) + + // A FRESH backend with no in-memory state: append directly (no prior load) + // → append must adopt from disk, and the adopt's load schedules a repair + // that the same append then performs before writing. + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root }) + await ctx2.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, + ] as SessionEvent[]) + const loaded = await ctx2.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + await ctx2.fiber.dispose() + }) + + it('a header-only log (open turn, no turn/end) preserves the open turn on load and closes it', async () => { + // A session whose only durable content is an unclosed first turn. scanLog + // preserves the turn/start; loadCore closes it with a synthetic + // turn/end {interrupted} so the returned log is balanced. + const m = meta('open-turn', '/h') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, + ] as SessionEvent[]) + const { events } = await ctx.sessionPersistence.load(m.id) + expect(events.map(e => e.type)).toEqual(['turn/start', 'turn/end']) + const end = events[1]! + expect(end.type === 'turn/end' && end.data.reason).toEqual({ kind: 'interrupted' }) + }) + + + it('createCore rejects an id already on disk under a DIFFERENT cwd bucket', async () => { + // Persist the id under cwd A. + const a = meta('dup-id', '/projA') + await ctx.sessionPersistence.create(a) + await ctx.sessionPersistence.append(a.id, oneTurnLog()) + // A fresh backend creating the SAME id under cwd B must still refuse: load + // identifies by id across all buckets, so a second log would make resume + // nondeterministic. create scans every bucket, not just meta.cwd's. + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root }) + await expect(ctx2.sessionPersistence.create(meta('dup-id', '/projB'))) + .rejects.toThrow(/already has a persisted log on disk/) + await ctx2.fiber.dispose() + }) + + it('flush keeps buffered events when the append fails (no silent loss)', async () => { + root = await freshRoot() + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + await ctx2.plugin(SessionPersistenceJsonl, { root }) + const session = ctx2.sessions.create(SessionId('flush-fail')) + // A full turn lands in the write-behind buffer. + session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + // Make the durable materialize fail on the next flush. + const backend = ctx2.sessionPersistence as unknown as { materialize: (...args: unknown[]) => Promise } + const origMat = backend.materialize.bind(backend) + backend.materialize = () => Promise.reject(new Error('disk full')) + await expect(ctx2.parallel('session/flush', session)).rejects.toThrow(/disk full/) + // The events are STILL buffered (not silently dropped): a retry persists them. + backend.materialize = origMat + await ctx2.parallel('session/flush', session) + const loaded = await ctx2.sessionPersistence.load(SessionId('flush-fail')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) + await ctx2.fiber.dispose() + }) + + it('rejects non-JSON event data: BigInt, function, circular, Map, undefined property', async () => { + const m = meta('serial') + await ctx.sessionPersistence.create(m) + const bad = (extra: unknown) => [{ type: 'user/message', seq: 0, time: 1, data: { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra } }] as unknown as SessionEvent[] + await expect(ctx.sessionPersistence.append(m.id, bad(1n))).rejects.toThrow(/non-JSON-serializable/) + await expect(ctx.sessionPersistence.append(m.id, bad(() => 0))).rejects.toThrow(/non-JSON-serializable/) + await expect(ctx.sessionPersistence.append(m.id, bad(Symbol('s')))).rejects.toThrow(/non-JSON-serializable/) + await expect(ctx.sessionPersistence.append(m.id, bad(new Map()))).rejects.toThrow(/non-JSON-serializable/) + await expect(ctx.sessionPersistence.append(m.id, bad(undefined))).rejects.toThrow(/non-JSON-serializable/) + await expect(ctx.sessionPersistence.append(m.id, bad(Infinity))).rejects.toThrow(/non-JSON-serializable/) + // a circular structure + const circ: Record = {} + circ.self = circ + await expect(ctx.sessionPersistence.append(m.id, bad(circ))).rejects.toThrow(/non-JSON-serializable/) + // The session was never materialized by any of the rejected appends. + expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id) + }) + + it('accepts well-formed JSON values (null, booleans, nested arrays/objects)', async () => { + const m = meta('json-ok') + await ctx.sessionPersistence.create(m) + const ev = [{ type: 'user/message', seq: 0, time: 1, data: { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra: { a: null, b: true, c: [1, 2, { d: 'nested' }] } } }] as unknown as SessionEvent[] + await ctx.sessionPersistence.append(m.id, ev) + expect((await ctx.sessionPersistence.list()).map(h => h.id)).toContain(m.id) + }) + + it('Session.append rejects a non-serializable event at the source (never enters the log)', () => { + const session = ctx.sessions.create(SessionId('reject-bad')) + // Serializability is enforced at the source: Session.append throws on a + // BigInt-bearing event BEFORE it enters session.events, so the durable log + // can never diverge from the live log. The throw surfaces at the caller's + // append site, not asynchronously in a backend flush. + expect(() => { + session.append('user/message', { content: [{ type: 'text', text: 'bad' }], source: { kind: 'user' }, bad: 1n } as never) + }).toThrow(/non-JSON-serializable/) + // The bad event was rejected, so the log stayed empty. + expect(session.events.length).toBe(0) + }) + +}) diff --git a/packages/session-persistence/session-persistence-jsonl/tsconfig.json b/packages/session-persistence/session-persistence-jsonl/tsconfig.json new file mode 100644 index 0000000000..23970f5a57 --- /dev/null +++ b/packages/session-persistence/session-persistence-jsonl/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../core/session" + }, + { + "path": "../../session-persistence/session-persistence" + } + ] +} diff --git a/packages/session-persistence-sqlite/README.md b/packages/session-persistence/session-persistence-sqlite/README.md similarity index 75% rename from packages/session-persistence-sqlite/README.md rename to packages/session-persistence/session-persistence-sqlite/README.md index 2025bd8a80..f397f80445 100644 --- a/packages/session-persistence-sqlite/README.md +++ b/packages/session-persistence/session-persistence-sqlite/README.md @@ -1,20 +1,20 @@ # @deepseek-ai/dsh-session-persistence-sqlite -A SQLite durable session-persistence backend — a second `SessionPersistence` implementation ([session persistence](../../docs/rfc/implemented/2026-06-14-session-persistence.md)), built to validate that the abstract seam and the shared `runPersistenceContract` suite are genuinely backend-agnostic. It satisfies the SAME contract as `dsh-session-persistence-jsonl` (append-only, contiguous-seq, lazy materialization, interrupted-turn close on load), expressed over `node:sqlite` rows instead of file bytes. +A SQLite durable session-persistence backend — a second `SessionPersistence` implementation ([session persistence](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md)), built to validate that the abstract seam and the shared `runPersistenceContract` suite are genuinely backend-agnostic. It satisfies the SAME contract as `dsh-session-persistence-jsonl` (append-only, contiguous-seq, lazy materialization, interrupted-turn close on load), expressed over `node:sqlite` rows instead of file bytes. > **TODO:** this backend talks to `node:sqlite` directly. If a cordis database service (`cordis/db` / a `@cordisjs` SQL driver plugin) is adopted, route through that instead of holding a raw `DatabaseSync` here — the contract surface (`SessionPersistence`) would not change, only the storage driver. ## Storage model -Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). Out-of-log metadata (`SessionMeta`) lives in a `sessions` row, including the mutable `SessionSummary` fields (`updatedAt`, `title`, `firstPrompt`) that `update()` rewrites without touching the event log. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`has`/`list` report exactly the sessions that have a row), so no separate column is needed. +Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). Out-of-log metadata (`SessionHeader`) lives in a `sessions` row. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row), so no separate column is needed. -The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and `journal_mode = WAL`. The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by a newer, incompatible build (higher `user_version`) is rejected rather than opened against an unknown layout. +The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and `journal_mode = WAL`. The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software). ## Contract semantics over rows - **Append = a transaction.** `append` runs `BEGIN`/`COMMIT` around the batch: it materializes the `sessions` row (if still lazy) and INSERTs every event, asserting the contiguous-seq contract first (the first event's `seq` must equal the stored next-seq). A mid-batch failure (a UNIQUE violation on a duplicated seq) rolls back entirely, so the stored log and the in-memory cursor stay consistent. (`load()` already balanced the stored log, so `append` never has to repair a crash tail.) -- **Lazy materialization.** `create()` records intent in memory only — no row is written until the first `append`. A created-but-never-appended session has no `sessions` row, so it is absent from `has()`/`list()` (which report exactly the sessions that have a row). -- **Interrupted-turn close on load.** `load()` reads every stored event ordered by `seq` and finds the longest seq-contiguous, parseable prefix — INCLUDING the real events of an interrupted final turn after the last `turn/end` (the loop only flushes at `turn/end`, so a process killed mid-turn leaves real, fully-written rows past it). A single turn can be huge in a long-horizon task, so those events are **preserved, never truncated**: `load()` CLOSES the orphaned turn by durably appending the minimal synthetic boundary events (an error `tool/result` for every assistant tool call left unanswered, a `step/end` if a step was open, then a `turn/end` carrying `{ kind: 'interrupted' }`), inside one transaction that also DELETEs any never-fully-written torn tail row. `load()` is therefore mutating — after it the stored rows are balanced and the cursor is truthful, so the next `append` continues cleanly. The boundary (last `turn/end`, torn-tail detection) is computed from the `seq`/`type` columns so a malformed `data` in a torn tail row is never parsed (discarded, not unloadable). A parse error or `seq` gap inside the committed region (at or before the last real `turn/end`) makes the session unloadable. A session whose only turn never closed keeps its metadata row and stays present in `has()`/`list()` — the same as the JSONL backend, whose file likewise survives a first append that never reached `turn/end`. +- **Lazy materialization.** `create()` records intent in memory only — no row is written until the first `append`. A created-but-never-appended session has no `sessions` row, so it is absent from `list()` (which reports exactly the sessions that have a row). +- **Interrupted-turn close on load.** `load()` reads every stored event ordered by `seq` and finds the longest seq-contiguous, parseable prefix — INCLUDING the real events of an interrupted final turn after the last `turn/end` (the loop only flushes at `turn/end`, so a process killed mid-turn leaves real, fully-written rows past it). A single turn can be huge in a long-horizon task, so those events are **preserved, never truncated**: `load()` CLOSES the orphaned turn by durably appending the minimal synthetic boundary events (an error `tool/result` for every assistant tool call left unanswered, a `step/end` if a step was open, then a `turn/end` carrying `{ kind: 'interrupted' }`), inside one transaction that also DELETEs any never-fully-written torn tail row. `load()` is therefore mutating — after it the stored rows are balanced and the cursor is truthful, so the next `append` continues cleanly. The boundary (last `turn/end`, torn-tail detection) is computed from the `seq`/`type` columns so a malformed `data` in a torn tail row is never parsed (discarded, not unloadable). A parse error or `seq` gap inside the committed region (at or before the last real `turn/end`) makes the session unloadable. A session whose only turn never closed keeps its metadata row and stays present in `list()` — the same as the JSONL backend, whose file likewise survives a first append that never reached `turn/end`. ## Configuration (schemastery) diff --git a/packages/session-persistence-sqlite/package.json b/packages/session-persistence/session-persistence-sqlite/package.json similarity index 83% rename from packages/session-persistence-sqlite/package.json rename to packages/session-persistence/session-persistence-sqlite/package.json index 463cf683be..b26c69461e 100644 --- a/packages/session-persistence-sqlite/package.json +++ b/packages/session-persistence/session-persistence-sqlite/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/session-persistence/session-persistence-sqlite/src/index.ts b/packages/session-persistence/session-persistence-sqlite/src/index.ts new file mode 100644 index 0000000000..cef61cb071 --- /dev/null +++ b/packages/session-persistence/session-persistence-sqlite/src/index.ts @@ -0,0 +1,244 @@ +/** + * SQLite durable session-persistence backend (`@deepseek-ai/dsh-session-persistence-sqlite`). + * + * A SECOND {@link SessionPersistence} implementation, built to validate that the + * abstract seam + the shared `runPersistenceContract` suite are genuinely + * backend-agnostic: the same append-only / contiguous-seq / lazy-materialization + * / interrupted-turn-close-on-load semantics the JSONL backend expresses over + * file bytes, expressed here over `node:sqlite` rows. Each `SessionEvent` maps + * 1:1 onto a row `(session_id, seq, type, time, data)`. + * + * Like the JSONL backend it supplies ONLY the storage primitives (the + * {@link PersistenceBackend} hooks below — INSERT/DELETE/SELECT inside + * transactions); all the write-path orchestration lives in the backend-agnostic + * {@link PersistenceCoordinator} this class composes. The four public + * {@link SessionPersistence} methods delegate to the coordinator. + * + * @module @deepseek-ai/dsh-session-persistence-sqlite + */ + +import { Context } from 'cordis' +import z from 'schemastery' +import { DatabaseSync } from 'node:sqlite' +import { mkdir } from 'node:fs/promises' +import { dirname, resolve } from 'node:path' +import { + SessionPersistence, PersistenceCoordinator, + type PersistenceBackend, type StoredPrefix, +} from '@deepseek-ai/dsh-session-persistence' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' +import { + openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow, +} from './schema.ts' + +export { SCHEMA_VERSION } from './schema.ts' + +/** Plugin configuration. */ +export interface Config { + /** + * Filesystem path to the SQLite database file. The special value `:memory:` + * opens an in-process database (tests); a file path is created (with parent + * dirs) on construction. + */ + path: string +} + +/** + * The SQLite persistence backend. Load as a plugin; it registers as + * `ctx.sessionPersistence` and (via the coordinator) installs the write-path + * listeners. Its torn-tail marker is the seq to delete from. + */ +export class SessionPersistenceSqlite extends SessionPersistence implements PersistenceBackend { + static inject = ['sessions'] + + static Config: z = z.object({ + path: z.string().required(), + }) + + /** + * Backend label for the coordinator's dispose diagnostics. Intentionally + * shadows cordis `Service.name` (set to `'sessionPersistence'` by the base); + * see the JSONL backend for why this does not affect service resolution. + */ + override readonly name = 'session-persistence-sqlite' + + private db!: DatabaseSync + private ready: Promise + private coordinator: PersistenceCoordinator + + constructor(ctx: Context, public config: Config) { + super(ctx) + // Open the database asynchronously (the parent directory may need creating); + // every hook awaits `ready` first. Opening synchronously would force a sync + // mkdir and block plugin apply. + this.ready = this.openDb(config.path) + this.coordinator = new PersistenceCoordinator(this.ctx, this) + } + + private async openDb(path: string): Promise { + if (path !== ':memory:') { + const abs = resolve(path) + await mkdir(dirname(abs), { recursive: true, mode: 0o700 }) + this.db = openDatabase(abs) + } else { + this.db = openDatabase(path) + } + } + + // --- SessionPersistence service surface (delegated to the coordinator) --- + + create(meta: SessionHeader): Promise { + return this.coordinator.create(meta) + } + + append(id: SessionId, events: readonly SessionEvent[]): Promise { + return this.coordinator.append(id, events) + } + + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.coordinator.load(id) + } + + // `list` is BOTH the public service method and the PersistenceBackend hook — + // one method (the SELECT below). The coordinator adds no orchestration for + // listing, so routing it through the coordinator would just recurse. Defined + // once, in the "PersistenceBackend hooks" section. + + /** + * The per-session init promises, exposed for white-box tests that await a + * specific session's onCreated (there is no public API to await one init). + */ + get inits(): Map> { + return this.coordinator.inits + } + + // --- PersistenceBackend hooks (the SQLite storage primitives) --- + + /** Read a stored prefix by id (ids are globally unique — no scope to scan). */ + loadStored(id: SessionId): Promise | undefined> { + return this.readPrefix(id) + } + + /** Read a stored prefix; `cwd` is ignored (the id is globally unique in SQLite). */ + loadLive(id: SessionId, _cwd: string | undefined): Promise | undefined> { + return this.readPrefix(id) + } + + /** + * Read a session's row + ordered events into a {@link StoredPrefix}. The + * torn-tail marker is the seq from which a never-committed tail must be deleted + * (`scanRows` already returns it as `number | undefined`). + */ + private async readPrefix(id: SessionId): Promise | undefined> { + await this.ready + const row = this.rowFor(id) + if (row === undefined) return undefined + const meta = rowToMeta(row) + const eventRows = this.db + .prepare('SELECT seq, type, time, data FROM events WHERE session_id = ? ORDER BY seq') + .all(id) as unknown as EventRow[] + const { preserved, tornFrom } = scanRows(eventRows) + return { meta, events: preserved, ...tornFrom !== undefined ? { tornMarker: tornFrom } : {} } + } + + /** + * Durably append a batch in ONE transaction: materialize the sessions row (if + * lazy) and INSERT every event, or roll back entirely. The transaction is the + * atomicity + durability boundary, so a mid-batch failure (a UNIQUE violation + * on a duplicated seq) leaves the stored log untouched. + */ + async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise { + await this.ready + const insertEvent = this.db.prepare( + 'INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)', + ) + this.db.exec('BEGIN') + try { + if (!isMaterialized) this.writeRow(meta) + for (const event of events) { + insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data)) + } + this.db.exec('COMMIT') + } catch (error) { + this.db.exec('ROLLBACK') + throw error + } + } + + /** + * Make a crash repair durable in ONE transaction: DELETE the torn tail (from + * `tornMarker`) and INSERT the synthetic `closers`. After COMMIT the stored rows + * == the balanced log. + */ + async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise { + await this.ready + this.db.exec('BEGIN') + try { + if (tornMarker !== undefined) { + this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(meta.id, tornMarker) + } + if (closers.length > 0) { + const insertEvent = this.db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') + for (const event of closers) { + insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data)) + } + } + this.db.exec('COMMIT') + } catch (error) { + // The DELETE+INSERT cannot collide (a row at a closer's seq is preserved or + // deleted as torn first); this rolls back a DB-level failure (disk full, + // etc.), unreachable in test. + /* v8 ignore start */ + this.db.exec('ROLLBACK') + throw error + /* v8 ignore stop */ + } + } + + /** List all materialized sessions' metadata (every row is a materialized session). */ + async list(): Promise { + await this.ready + const rows = this.db + .prepare('SELECT * FROM sessions') + .all() as unknown as SessionRow[] + return rows.map(rowToMeta) + } + + /** Close the database handle (awaited by the coordinator's dispose, post-drain). */ + async close(): Promise { + await this.ready + this.db.close() + } + + // --- row helpers --- + + /** Fetch a session's row, or undefined if absent. */ + private rowFor(id: SessionId): SessionRow | undefined { + return this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined + } + + /** + * Insert-or-replace a session's metadata row. The only caller is the first + * materializing `appendBatch`, so writing the row IS the materialization (its + * existence is the signal `list` reads). + */ + private writeRow(meta: SessionHeader): void { + this.db.prepare(` + INSERT INTO sessions (id, version, created_at, cwd, parent_session) + VALUES (?, ?, ?, ?, ?) + ON CONFLICT(id) DO UPDATE SET + version = excluded.version, + created_at = excluded.created_at, + cwd = excluded.cwd, + parent_session = excluded.parent_session + `).run( + meta.id, + meta.version, + meta.createdAt, + meta.cwd ?? null, + meta.parentSession ?? null, + ) + } +} + +export default SessionPersistenceSqlite diff --git a/packages/session-persistence-sqlite/src/schema.ts b/packages/session-persistence/session-persistence-sqlite/src/schema.ts similarity index 84% rename from packages/session-persistence-sqlite/src/schema.ts rename to packages/session-persistence/session-persistence-sqlite/src/schema.ts index 247079c3a8..8238cba30c 100644 --- a/packages/session-persistence-sqlite/src/schema.ts +++ b/packages/session-persistence/session-persistence-sqlite/src/schema.ts @@ -8,20 +8,20 @@ */ import { DatabaseSync } from 'node:sqlite' -import type { SessionEvent, SessionId, SessionMeta } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' /** * The on-disk schema version. Bumped only on a breaking change to the table * layout; orthogonal to a session's own `version` (which versions the EVENT * vocabulary, stored per session in the `sessions` row). */ -export const SCHEMA_VERSION = 1 +export const SCHEMA_VERSION = 2 /** - * A row of the `sessions` table — the out-of-log metadata (`SessionMeta`). The - * row's EXISTENCE is the materialization signal: it is written only by the + * A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}). + * The row's EXISTENCE is the materialization signal: it is written only by the * first `append` (lazy materialization), so a created-but-never-appended - * session has no row and is absent from `has`/`list`, mirroring the JSONL + * session has no row and is absent from `list`, mirroring the JSONL * backend's "no file until first append". */ export interface SessionRow { @@ -30,9 +30,6 @@ export interface SessionRow { created_at: number cwd: string | null parent_session: string | null - updated_at: number - title: string | null - first_prompt: string | null } /** An `events` table row: one `SessionEvent` mapped 1:1 (`data` is JSON text). */ @@ -51,10 +48,11 @@ export interface EventRow { * * The table-layout version is persisted in SQLite's `PRAGMA user_version` and * checked on open: a fresh database (user_version 0) is stamped with the - * current {@link SCHEMA_VERSION}; an existing database with a NEWER version - * (written by a future, incompatible build) is rejected rather than opened - * against a layout this build does not understand. (An older-but-compatible - * version would be migrated here when migrations exist; v1 has none.) + * current {@link SCHEMA_VERSION}; an existing database whose version is NOT the + * current one (written by a different, incompatible build — older or newer) is + * REJECTED rather than opened against a layout this build does not understand. + * There are no migrations: v1 had a different `sessions` layout and is not + * upgraded in place. */ export function openDatabase(path: string): DatabaseSync { const db = new DatabaseSync(path) @@ -62,9 +60,9 @@ export function openDatabase(path: string): DatabaseSync { db.exec('PRAGMA journal_mode = WAL') // `PRAGMA user_version` always returns exactly one row { user_version }. const { user_version: onDisk } = db.prepare('PRAGMA user_version').get() as { user_version: number } - if (onDisk > SCHEMA_VERSION) { + if (onDisk !== 0 && onDisk !== SCHEMA_VERSION) { db.close() - throw new Error(`session database at "${path}" has schema version ${onDisk}, newer than this build supports (${SCHEMA_VERSION})`) + throw new Error(`session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`) } if (onDisk === 0) { // Fresh (or pre-versioning) database: stamp the current layout version. @@ -78,10 +76,7 @@ export function openDatabase(path: string): DatabaseSync { version INTEGER NOT NULL, created_at INTEGER NOT NULL, cwd TEXT, - parent_session TEXT, - updated_at INTEGER NOT NULL, - title TEXT, - first_prompt TEXT + parent_session TEXT ) STRICT `) db.exec(` @@ -97,17 +92,14 @@ export function openDatabase(path: string): DatabaseSync { return db } -/** Reconstruct the full {@link SessionMeta} from a `sessions` row. */ -export function rowToMeta(row: SessionRow): SessionMeta { +/** Reconstruct the {@link SessionHeader} from a `sessions` row. */ +export function rowToMeta(row: SessionRow): SessionHeader { return { version: row.version, id: row.id as SessionId, createdAt: row.created_at, - updatedAt: row.updated_at, ...row.cwd !== null ? { cwd: row.cwd } : {}, ...row.parent_session !== null ? { parentSession: row.parent_session as SessionId } : {}, - ...row.title !== null ? { title: row.title } : {}, - ...row.first_prompt !== null ? { firstPrompt: row.first_prompt } : {}, } } diff --git a/packages/session-persistence/session-persistence-sqlite/tests/sqlite.spec.ts b/packages/session-persistence/session-persistence-sqlite/tests/sqlite.spec.ts new file mode 100644 index 0000000000..2bc9c59643 --- /dev/null +++ b/packages/session-persistence/session-persistence-sqlite/tests/sqlite.spec.ts @@ -0,0 +1,373 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite' +import { openDatabase, scanRows, type EventRow } from '../src/schema.ts' +import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' + +const dirs: string[] = [] +afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) + +async function freshDbPath(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-')) + dirs.push(dir) + return join(dir, 'sessions.db') +} + +/** A context with the session store + SQLite backend, plus a teardown. */ +async function backend(path = ':memory:'): Promise<{ ctx: Context; dispose: () => Promise }> { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(SessionPersistenceSqlite, { path }) + return { ctx, dispose: () => fiber.dispose() } +} + +// The payoff: the SAME backend-agnostic contract the JSONL backend runs, now +// proving the SQLite backend satisfies identical semantics. +runPersistenceContract('sqlite', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) + return { + persistence: ctx.sessionPersistence, + dispose: async () => { await fiber.dispose() }, + } +}) + +// Run the shared coordinator orchestration suite against the real SQLite backend. +// A FILE-backed db (not :memory:) is the shared storage scope so two mounted +// instances see the same rows (HMR/reload). `corruptTail` INSERTs a row past the +// committed seq whose `data` is invalid JSON — a never-committed torn tail that +// drives the coordinator's commitRepair-with-tornMarker branch over real db rows. +runCoordinatorContract('sqlite', async (): Promise => { + const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-coord-')) + const path = join(dir, 'sessions.db') + return { + mount: async ctx => ctx.plugin(SessionPersistenceSqlite, { path }), + corruptTail: async (id) => { + // A row past the committed region whose `data` does not parse: scanRows + // bounds the preserved prefix at it and returns its seq as tornFrom, which + // the backend surfaces to the coordinator as the tornMarker to delete from. + const db = openDatabase(path) + const next = (db.prepare('SELECT COALESCE(MAX(seq), -1) + 1 AS n FROM events WHERE session_id = ?') + .get(id) as { n: number }).n + db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)') + .run(id, next, 'assistant/chunk', 99, '{not valid json') + db.close() + }, + cleanup: async () => { await rm(dir, { recursive: true, force: true }) }, + } +}) + +describe('scanRows', () => { + // scanRows works off EventRows (data is a JSON string column); build them from + // SessionEvents so the unit tests read in terms of the event vocabulary. + const rows = (events: SessionEvent[]): EventRow[] => + events.map(e => ({ seq: e.seq, type: e.type, time: e.time, data: JSON.stringify(e.data) })) + + it('preserves the full log when it ends exactly on a turn/end (no torn tail)', () => { + const { preserved, tornFrom } = scanRows(rows(oneTurnLog())) + expect(preserved).toEqual(oneTurnLog()) + expect(tornFrom).toBeUndefined() + }) + + it('PRESERVES the real events of an interrupted turn after the last turn/end', () => { + // turn 1 committed (0..5) + a crashed turn 2 (turn/start 6, step/start 7, no + // close): all 8 rows are intact, so the whole prefix is preserved and there + // is no torn fragment to delete. (load() then synthesizes the closers.) + const withOpenTurn: SessionEvent[] = [ + ...oneTurnLog(), + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, + ] + const { preserved, tornFrom } = scanRows(rows(withOpenTurn)) + expect(preserved.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + expect(tornFrom).toBeUndefined() + }) + + it('preserves the contiguous prefix and flags a torn tail at a seq gap', () => { + // A gap after seq 0 (no committed turn/end): seq 0 is the preserved + // interrupted-turn event; the gap bounds it and marks the torn fragment. + const gapped: SessionEvent[] = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing + ] + const { preserved, tornFrom } = scanRows(rows(gapped)) + expect(preserved.map(e => e.seq)).toEqual([0]) + expect(tornFrom).toBe(1) + }) + + it('an empty log preserves nothing and has no torn tail', () => { + expect(scanRows([])).toEqual({ preserved: [] }) + }) + + it('throws on a seq gap inside the committed region (before the last turn/end)', () => { + const gapped: SessionEvent[] = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing + { type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + expect(() => scanRows(rows(gapped))).toThrow(/seq gap in committed region/) + }) + + it('throws on an unparsable row inside the committed region', () => { + const withCorruptCommitted: EventRow[] = [ + { seq: 0, type: 'turn/start', time: 1, data: '{not json' }, // corrupt, sits before a turn/end + { seq: 1, type: 'turn/end', time: 2, data: JSON.stringify({ turn: 1, reason: { kind: 'completed' } }) }, + ] + expect(() => scanRows(withCorruptCommitted)).toThrow(/unparsable committed event/) + }) + + it('tolerates an unparsable torn-tail row after the last turn/end', () => { + const withCorruptTail: EventRow[] = [ + ...rows(oneTurnLog()), + { seq: 6, type: 'turn/start', time: 7, data: '{not json' }, // torn fragment, no committed turn/end after + ] + const { preserved, tornFrom } = scanRows(withCorruptTail) + expect(preserved).toEqual(oneTurnLog()) + expect(tornFrom).toBe(6) + }) +}) + +describe('SessionPersistenceSqlite: durability and crash semantics', () => { + it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => { + const path = await freshDbPath() + const m = meta('crash') + // Run 1: persist a complete turn, then a half-written second turn (no turn/end). + const ctx1 = new Context() + await ctx1.plugin(SessionStore) + const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) + await ctx1.sessionPersistence.create(m) + await ctx1.sessionPersistence.append(m.id, oneTurnLog()) + await ctx1.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, + ]) + await fiber1.dispose() + + // Run 2: load PRESERVES the interrupted turn's real events (a turn can be huge + // — never truncated) and closes the orphaned turn with synthetic boundary + // events: step/end (the step was open) then turn/end {interrupted}. + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) + const loaded = await ctx2.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.type)).toEqual([ + 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1 + 'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real events + synthetic closers + ]) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) + const last = loaded.events.at(-1)! + expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) + + // load durably closed the turn, so the next append continues at the balanced + // length (seq 10) and a reload round-trips identically. + await ctx2.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } }, + ]) + const reloaded = await ctx2.sessionPersistence.load(m.id) + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) + await fiber2.dispose() + }) + + it('load() durably closes the interrupted turn: the synthetic closers are on disk after load', async () => { + const path = await freshDbPath() + const m = meta('load-closes') + const b1 = await backend(path) + await b1.ctx.sessionPersistence.create(m) + await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5 + await b1.dispose() + // Hand-write an interrupted turn (turn/start seq 6, no turn/end). + const db = openDatabase(path) + db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)') + .run(m.id, 'turn/start', JSON.stringify({ turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })) + db.close() + + const b2 = await backend(path) + const loaded = await b2.ctx.sessionPersistence.load(m.id) + // turn 2's real turn/start (seq 6) is preserved + a synthetic turn/end (seq 7). + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + expect(loaded.events.at(-1)!.type).toBe('turn/end') + // load() is mutating: the synthetic turn/end MUST be on disk so the stored log + // is balanced and the cursor is truthful (contract: load closes, not defers). + const probe = openDatabase(path) + const stored = probe.prepare('SELECT seq, type FROM events WHERE session_id = ? ORDER BY seq').all(m.id) as { seq: number; type: string }[] + probe.close() + expect(stored.map(r => r.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + expect(stored.at(-1)!.type).toBe('turn/end') + await b2.dispose() + }) + + it('all-tail load: a session whose only turn never closed is preserved and closed on load', async () => { + const path = await freshDbPath() + const m = meta('all-tail') + const b1 = await backend(path) + await b1.ctx.sessionPersistence.create(m) + // A first turn that NEVER completed: turn/start + user/message, no turn/end. + await b1.ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } } }, + ]) + await b1.dispose() + + // A fresh backend loads it: the interrupted (only) turn's real events are + // preserved and closed with a synthetic turn/end {interrupted} — NOT + // truncated. The session was materialized, so list() reports it present. + const b2 = await backend(path) + const loaded = await b2.ctx.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'user/message', 'turn/end']) + expect(loaded.events.at(-1)!.type === 'turn/end' && loaded.events.at(-1)!.data).toMatchObject({ reason: { kind: 'interrupted' } }) + expect((await b2.ctx.sessionPersistence.list()).map(x => x.id)).toContain(m.id) + await b2.dispose() + }) + + it('rejects opening a database whose schema version is not the current build (newer OR older)', async () => { + const path = await freshDbPath() + openDatabase(path).close() // stamp user_version = SCHEMA_VERSION + // Bump user_version past what this build supports. + const dbNewer = openDatabase(path) + dbNewer.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`) + dbNewer.close() + expect(() => openDatabase(path)).toThrow(/incompatible with this build/) + + // A stale OLDER version (e.g. a pre-summary-drop v1 DB) is also rejected — + // we do not migrate (unreleased software, no backward-compat). + const olderPath = await freshDbPath() + openDatabase(olderPath).close() + const dbOlder = openDatabase(olderPath) + dbOlder.exec('PRAGMA user_version = 1') + dbOlder.close() + expect(() => openDatabase(olderPath)).toThrow(/incompatible with this build/) + }) + + it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => { + const path = await freshDbPath() + const m = meta('corrupt-tail') + const b1 = await backend(path) + await b1.ctx.sessionPersistence.create(m) + await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed: seqs 0..5 + await b1.dispose() + + // Hand-insert a torn tail row (seq 6, no closing turn/end) whose `data` is + // invalid JSON. The contract: only a parse error in the COMMITTED region is + // unloadable; a torn tail must be discarded. scanRows finds the last + // turn/end on the seq+type columns (never parsing tail `data`), so the + // unparsable row after it bounds the preserved prefix and is deleted by load. + const db = openDatabase(path) + db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)') + .run(m.id, 'turn/start', '{not valid json') + db.close() + + const b2 = await backend(path) + const loaded = await b2.ctx.sessionPersistence.load(m.id) + expect(loaded.events).toEqual(oneTurnLog()) // torn tail discarded, committed intact (turn 1 already balanced → no closers) + // load physically deleted the corrupt tail row, so a fresh append continues. + await b2.ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 8, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 9, data: { turn: 2, reason: { kind: 'completed' } } }, + ]) + const reloaded = await b2.ctx.sessionPersistence.load(m.id) + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + await b2.dispose() + }) + + it('append rolls back the whole batch on a mid-batch seq collision (transaction)', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) + const m = meta('rollback') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5 + + // A batch that re-states an already-stored seq must be rejected and leave + // the stored log unchanged (the UNIQUE (session_id, seq) constraint fires + // inside the transaction → ROLLBACK). + await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toThrow() + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events).toEqual(oneTurnLog()) // unchanged + await fiber.dispose() + }) + + it('persists across separate backend instances over the same file', async () => { + const path = await freshDbPath() + const m = meta('persist', '/proj') + const ctx1 = new Context() + await ctx1.plugin(SessionStore) + const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) + await ctx1.sessionPersistence.create(m) + await ctx1.sessionPersistence.append(m.id, oneTurnLog()) + await fiber1.dispose() + + const ctx2 = new Context() + await ctx2.plugin(SessionStore) + const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) + expect((await ctx2.sessionPersistence.list()).map(x => x.id)).toContain(m.id) + const loaded = await ctx2.sessionPersistence.load(m.id) + expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj' }) + expect(loaded.events).toEqual(oneTurnLog()) + await fiber2.dispose() + }) + + it('exposes the schema version constant', () => { + expect(SCHEMA_VERSION).toBe(2) + }) +}) + +describe('SessionPersistenceSqlite: edge cases', () => { + it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => { + const path = await freshDbPath() + const m = meta('rollback-insert') + const b1 = await backend(path) + await b1.ctx.sessionPersistence.create(m) + await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) + + // A SECOND backend over the same file loads the session first, so it adopts + // cursor 6 (the committed length) into its OWN in-memory state. + const b2 = await backend(path) + await b2.ctx.sessionPersistence.load(m.id) // cursor 6 in b2 + const turn2: SessionEvent[] = [ + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, + ] + // b1 commits seq 6..7 first. + await b1.ctx.sessionPersistence.append(m.id, turn2) + // b2 still thinks its cursor is 6, so this batch passes the contiguity check + // but its INSERT of seq 6 hits the UNIQUE (session_id, seq) constraint + // mid-transaction → ROLLBACK + rethrow. + await expect(b2.ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/UNIQUE/) + // b1's turn is intact; b2's rolled-back attempt left nothing extra. + const loaded = await b1.ctx.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + await b1.dispose() + await b2.dispose() + }) + + it('HMR: a DIFFERENT session colliding with a materialized on-disk id is rejected', async () => { + const path = await freshDbPath() + // Instance 1 materializes a session and disposes. + const b1 = await backend(path) + const s1 = b1.ctx.sessions.create(SessionId('hmr-collide')) + for (const e of oneTurnLog()) s1.append(e.type, e.data) + await b1.ctx.parallel('session/flush', s1) + await b1.dispose() + + // A fresh context with an UNRELATED live session reusing the id meets a + // materialized row that is NOT a prefix of its events → reject. + const ctx = new Context() + await ctx.plugin(SessionStore) + let session!: Session + await ctx.plugin(Object.assign((inner: Context) => { + session = inner.sessions.create(SessionId('hmr-collide')) + }, { inject: ['sessions'] })) + session.append('turn/start', { turn: 9, trigger: { kind: 'message', source: { kind: 'user' } } }) + await ctx.plugin(SessionPersistenceSqlite, { path }) + await expect(ctx.parallel('session/flush', session)).rejects.toThrow(/id collision/) + await ctx.fiber.dispose() + }) +}) diff --git a/packages/session-persistence/session-persistence-sqlite/tsconfig.json b/packages/session-persistence/session-persistence-sqlite/tsconfig.json new file mode 100644 index 0000000000..23970f5a57 --- /dev/null +++ b/packages/session-persistence/session-persistence-sqlite/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../core/session" + }, + { + "path": "../../session-persistence/session-persistence" + } + ] +} diff --git a/packages/session-persistence/session-persistence/README.md b/packages/session-persistence/session-persistence/README.md new file mode 100644 index 0000000000..928e7b033d --- /dev/null +++ b/packages/session-persistence/session-persistence/README.md @@ -0,0 +1,51 @@ +# @deepseek-ai/dsh-session-persistence + +The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface. + +The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here. + +## Service API (`ctx.sessionPersistence`) + +| Method | Contract | +|---|---| +| `create(meta): Promise` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). | +| `append(id, events): Promise` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. | +| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. | +| `list(): Promise` | Lightweight listing from metadata, no full-log parse. A zero-event lazily-materialized session is absent from `list`. | + +## Invariants every backend must honor + +- **Append-only; a crashed turn is closed, not truncated.** Committed events (at or below a flushed `turn/end`) are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; `load` preserves them and durably appends synthetic closers (an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}`) to balance the log and keep the rehydrated history a valid provider transcript. Only a never-fully-written torn tail fragment is discarded. +- **Contiguous seq.** `load` rejects a `seq` gap/parse error in the MIDDLE of the log; `append`'s first `seq` must equal the stored next-seq. +- **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable). +- **Durability.** `append` returns only once the batch is durable. + +## The write coordinator + +The two first-party backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the in-memory bookkeeping (per-id state, write-behind buffers, per-id serialization chains, per-session init promises), the `session/event` → buffer → `session/flush` drain, lazy materialization, crash-tail repair on load, the four `session/created` adoption cases (new / HMR-adopt / collision / ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives differed (write bytes vs. INSERT rows). + +`PersistenceCoordinator` owns that orchestration once. A first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements the small `PersistenceBackend` hook interface, and delegates its four public service methods to the coordinator. This keeps the duplicated, correctness-heavy orchestration in a single place (it used to receive the same fixes twice). + +The `PersistenceBackend` hooks (the only seam between the coordinator and storage): + +| Hook | Role | +|---|---| +| `name` | Backend label for the dispose-failure `AggregateError`. | +| `loadStored(id)` | Read a stored prefix by id, scanning ANY storage scope. Used by resume/load and, via `!== undefined`, the create-collision probe. Returns an opaque `tornMarker` iff a torn tail must be truncated. | +| `loadLive(id, cwd)` | Read a stored prefix SCOPED to `cwd` (HMR live-adoption must only adopt a log at the SAME cwd; a same-id log elsewhere is a collision, not a resume). A globally-unique-id backend ignores `cwd`. | +| `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. | +| `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined` — a marker may be falsy, e.g. seq/offset `0`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). | +| `list()` | List all stored metadata. | +| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. | + +The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). The public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator. See [the write-coordinator RFC](../../../docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md). + +## Testing backends + +Import `runPersistenceContract` from `tests/contract.ts` (the public-API contract) and `runCoordinatorContract` from `tests/coordinator-contract.ts` (the shared write-path orchestration: adoption, HMR, collision, dispose-drain, crash-tail repair) and call each with a fixture for your backend. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics AND the same orchestration, so a backend's own spec is left with only storage-mechanics tests (path sanitization, fsync rollback; schema version, transaction rollback) on top. + +Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store. + +## Metadata types + +Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`). diff --git a/packages/session-persistence/package.json b/packages/session-persistence/session-persistence/package.json similarity index 80% rename from packages/session-persistence/package.json rename to packages/session-persistence/session-persistence/package.json index bd84fd1826..ed6c80dfd9 100644 --- a/packages/session-persistence/package.json +++ b/packages/session-persistence/session-persistence/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/session-persistence/session-persistence/src/coordinator.ts b/packages/session-persistence/session-persistence/src/coordinator.ts new file mode 100644 index 0000000000..7c7b044ee1 --- /dev/null +++ b/packages/session-persistence/session-persistence/src/coordinator.ts @@ -0,0 +1,549 @@ +/** + * The backend-agnostic write-path orchestration shared by every first-party + * {@link SessionPersistence} backend. + * + * Every durable backend needs the same orchestration: the in-memory bookkeeping + * (the per-id state, the write-behind buffers, the per-id serialization chains, + * the per-session init promises), the `session/event` → buffer → `session/flush` + * drain, lazy materialization, crash-tail repair on load, the four + * `session/created` adoption cases (new / HMR-adopt / collision / + * ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives are + * backend-specific (file bytes for `dsh-session-persistence-jsonl`, `node:sqlite` + * rows for `dsh-session-persistence-sqlite`). {@link PersistenceCoordinator} owns + * the orchestration; a backend supplies the storage primitives as a small + * {@link PersistenceBackend} hook object. + * + * The abstract {@link SessionPersistence} service's public API is independent of + * this: a backend IS a `SessionPersistence` (its four public methods delegate to + * a coordinator it composes), so a third-party backend MAY implement the service + * directly without using the coordinator at all. + * + * See the write-coordinator RFC (docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md) + * for the design rationale (composition over inheritance, the opaque torn marker). + * + * @module @deepseek-ai/dsh-session-persistence/coordinator + */ + +import { Context } from 'cordis' +import { interruptedTurnClosers, SESSION_FORMAT_VERSION } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' +import { assertSerializable, seedCoversPrefix } from './index.ts' + +/** + * A stored session's durable prefix as read back from a backend: its + * {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix, + * and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must + * be truncated before further writes. + * + * The coordinator NEVER inspects `tornMarker`'s value — it only tests + * `!== undefined` (is there a tail to repair?) and passes the value back to + * {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker + * type: the JSONL backend uses the byte offset to truncate to, the SQLite + * backend uses the seq to delete from (both happen to be `number`). + */ +export interface StoredPrefix { + meta: SessionHeader + events: SessionEvent[] + tornMarker?: TornMarker +} + +/** + * The storage seam between {@link PersistenceCoordinator} and a concrete + * backend: the minimal set of durable primitives the orchestration calls. A + * backend implements these (over files, rows, an object store, …); the + * coordinator supplies everything else (buffering, serialization, cursors, + * adoption, crash repair sequencing, dispose quiescence). + * + * @typeParam TornMarker - the backend's opaque torn-tail repair token (see + * {@link StoredPrefix}). The coordinator treats it as fully opaque. + */ +export interface PersistenceBackend { + /** Human-readable backend name, used in the dispose-failure AggregateError. */ + readonly name: string + + /** + * Read a stored prefix by id, scanning ANY storage scope (for JSONL: every + * cwd bucket). Returns `undefined` if no stored artifact exists. Used by + * resume/load, and — via `!== undefined` — by the create-collision probe. + * The returned `tornMarker` is present iff there is a torn tail to truncate. + */ + loadStored(id: SessionId): Promise | undefined> + + /** + * Read a stored prefix SCOPED to `cwd`. Deliberately distinct from + * {@link loadStored}: HMR live-adoption must only adopt a persisted log at the + * SAME cwd as the live session (a same-id log at a different cwd is a + * collision, not a resume) — conflating the two reintroduces a cross-cwd + * adoption bug. For a globally-unique-id backend (SQLite) `cwd` is ignored. + */ + loadLive(id: SessionId, cwd: string | undefined): Promise | undefined> + + /** + * Durably append a CONTIGUOUS batch, lazily materializing the session first + * when `!isMaterialized`. The materialize-write and the first event batch MUST + * commit ATOMICALLY (a crash between them must not leave a materialized-but- + * empty session). Returns once the batch is durable. + */ + appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise + + /** + * Make a crash repair durable: truncate the torn tail (iff + * `tornMarker !== undefined`) and append `closers` (iff any). NOT required to + * be atomic — a file backend may truncate-then-append in two fsync'd steps. + * Used by load (truncate + synthetic closers) and by live-adoption (truncate + * only, `closers = []`). + */ + commitRepair(meta: SessionHeader, tornMarker: TornMarker | undefined, closers: readonly SessionEvent[]): Promise + + /** List all stored (materialized) sessions' metadata. */ + list(): Promise + + /** + * Optional lifecycle teardown (e.g. close a database handle). Awaited by the + * coordinator's dispose effect AFTER the quiescence drain. A stateless file + * backend omits it. + */ + close?(): Promise +} + +/** Per-session write state held by the coordinator's in-memory bookkeeping. */ +interface SessionState { + meta: SessionHeader + /** The next seq the backend expects to append (the stored log length). */ + cursor: number + /** + * Whether the backend has physically written this session (a JSONL file / + * SQLite row exists). `create()` registers state LAZILY — cursor 0, + * materialized false, nothing on disk — so an empty session leaves no + * artifact and the FIRST `appendBatch` writes the header + its events in ONE + * transaction (the "a row exists ⇔ it has events" invariant `list` + * relies on; a separate up-front materialize could crash leaving a row with + * zero events). The flag is the only signal that distinguishes a session + * registered-but-never-written from one durably present, which the reclaim + * path needs (an abandoned id with no artifact AND no buffered events is free + * to reuse; a materialized one is a real collision). + */ + materialized: boolean + /** + * The live Session this state was bound to via `onCreated`, if any. State + * created through the public `create()`/`load()` API has no owner; state bound + * to a live session lets `onCreated` reject a second, unrelated session on the + * same id (a collision) instead of silently no-opping. + */ + owner?: Session +} + +/** Collect the rejection reasons from a set of promises (none-throwing). */ +async function settledErrors(promises: Iterable>): Promise { + const settled = await Promise.allSettled([...promises]) + const errors: unknown[] = [] + for (const result of settled) { + if (result.status === 'rejected') errors.push(result.reason) + } + return errors +} + +/** + * Owns the backend-agnostic session write-path orchestration. A backend + * constructs one (`new PersistenceCoordinator(ctx, this)`), implements + * {@link PersistenceBackend}, and delegates its four public service methods to + * the matching coordinator methods. + * + * All per-id operations are serialized (a per-id promise chain) so concurrent + * flushes / a flush racing a load never interleave storage writes. The + * constructor installs the write-path listeners and the dispose effect. + * + * @typeParam TornMarker - the backend's opaque torn-tail repair token. + */ +export class PersistenceCoordinator { + /** Backend bookkeeping keyed by session id (NOT the live Session object). */ + private states = new Map() + /** Write-behind buffers keyed by the live Session (write path). */ + private buffers = new Map() + /** + * Per-session serialization: every operation chains onto the prior one for the + * same id, so writes for one session never interleave. Keyed by session id. + */ + private chains = new Map>() + /** + * Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not + * its id: a disposed fiber's session can be replaced by a different live + * Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache + * would hand the new object the old object's init promise. + * + * Public (readonly) so a backend can expose it for white-box tests that await + * a specific session's init (there is no public API to await one init); the + * coordinator itself only ever mutates it internally. + */ + readonly inits = new Map>() + + constructor(private ctx: Context, private backend: PersistenceBackend) { + this.installWritePath() + } + + // --- public surface (the backend's service methods delegate here) --- + + /** + * Register a new session's metadata (lazy: no physical write until the first + * {@link append}). Rejects if the id is already tracked or already persisted. + */ + create(meta: SessionHeader): Promise { + // Snapshot the metadata at call time: the op runs later (behind the + // per-session chain) and the snapshot is stored as the lazy state, so keeping + // the caller's object by reference would let a later mutation of `id`/`cwd` + // register under one key but materialize under a different path/header. + const snapshot: SessionHeader = { ...meta } + return this.serialize(snapshot.id, () => this.createCore(snapshot)) + } + + private async createCore(meta: SessionHeader): Promise { + // Do NOT clobber an existing session: the SessionId IS the identity. + if (this.states.has(meta.id)) { + throw new Error(`session "${meta.id}" already exists in this backend`) + } + // A persisted artifact under this id (in ANY scope) blocks creation: load/ + // resume identify a session by id alone, so a second artifact would make + // resume nondeterministic. + if (await this.backend.loadStored(meta.id) !== undefined) { + throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`) + } + // Pure lazy: record intent only. No artifact until the first append. + this.states.set(meta.id, { meta, cursor: 0, materialized: false }) + } + + // `async` so the synchronous validate/clone below reject (not throw) per the + // Promise contract — callers use `await expect(...).rejects`. + /** + * Durably persist a batch of events. Honors the append-only and contiguous-seq + * contracts; rejects non-JSON-serializable `event.data`. + */ + async append(id: SessionId, events: readonly SessionEvent[]): Promise { + // Validate serializability BEFORE cloning so a bad event surfaces the typed + // error rather than an opaque DataCloneError from structuredClone. + assertSerializable(events) + // Deep-snapshot the batch HERE, before the op waits behind the per-session + // chain: a caller that mutates a live array (e.g. session.events) — or an + // event inside it — before the op runs would otherwise have those changes + // persisted. The clone is taken synchronously (at call time). + const batch = events.map(e => structuredClone(e)) + return this.serialize(id, () => this.appendCore(id, batch)) + } + + private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { + if (events.length === 0) return + let state = this.states.get(id) + if (state === undefined) state = await this.adopt(id) // calls loadCore, not load + + // Contiguity contract: each event's seq must continue the stored log. + for (const [i, event] of events.entries()) { + if (event.seq !== state.cursor + i) { + throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`) + } + } + + await this.backend.appendBatch(state.meta, events, state.materialized) + // The durable write is the transaction: mark materialized + advance the + // cursor as soon as it commits (uniform across backends). + state.materialized = true + state.cursor += events.length + } + + /** + * Reload a session: its {@link SessionHeader} plus the event log up to the last + * durable checkpoint, with any interrupted final turn durably closed (synthetic + * boundary events) during load. + */ + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.serialize(id, () => this.loadCore(id)) + } + + private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + const stored = await this.backend.loadStored(id) + if (stored === undefined) throw new Error(`session "${id}" not found`) + const { meta, events, tornMarker } = stored + this.assertVersion(meta) + + // Crash-recovery: if the log ended mid-turn (real, preserved events but no + // closing turn/end), close it durably DURING load so disk, the returned log, + // and the cursor all agree. The interrupted turn's real events are preserved, + // never truncated (a turn can be huge — the session-persistence RFC); only a + // never-fully-written torn tail fragment is discarded. + const closers = interruptedTurnClosers(events) + const balanced = [...events, ...closers] + + // Make the repair durable (truncate the torn tail + append the synthetic + // closers) BEFORE recording state — commitRepair takes `meta` directly, so + // there is no state-path ordering dependency (uniform across backends). + if (tornMarker !== undefined || closers.length > 0) { + await this.backend.commitRepair(meta, tornMarker, closers) + } + // The state keeps its OWN copy of the meta; the returned value is separate so + // a consumer mutating loaded.meta cannot corrupt the backend's metadata. + this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true }) + return { meta, events: balanced } + } + + // NOTE: there is deliberately no coordinator `list()`. Listing needs none of + // the coordinator's orchestration (no per-id serialization, no cursor, no + // in-memory state) — it is a pure read of stored metadata. A backend's public + // `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it + // through the coordinator would only forward to that same hook, so the + // coordinator stays out of the listing path entirely. + + // --- per-id serialization + adoption helpers --- + + /** + * Run `op` after any in-flight operation for the same session id, so writes for + * one session never interleave. Errors do not poison the chain. NOTE: serialized + * public methods must NOT call each other (deadlock); they call the unserialized + * `*Core` helpers instead. + */ + private serialize(id: SessionId, op: () => Promise): Promise { + const prior = this.chains.get(id) ?? Promise.resolve() + const next = prior.then(op, op) + // Keep the chain alive but swallow this op's rejection for the NEXT waiter + // (the caller still sees the real rejection via `next`). + this.chains.set(id, next.then(() => undefined, () => undefined)) + return next + } + + /** Build a state for a session discovered in storage but not yet in memory. */ + private async adopt(id: SessionId): Promise { + // loadCore (NOT load) — adopt runs inside an already-serialized op, so + // re-entering the chain via the public load() would deadlock. + await this.loadCore(id) + const state = this.states.get(id) + /* v8 ignore next -- loadCore always sets the state for the id */ + if (!state) throw new Error(`failed to adopt session "${id}"`) + return state + } + + private assertVersion(meta: SessionHeader): void { + if (meta.version !== SESSION_FORMAT_VERSION) { + throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v${SESSION_FORMAT_VERSION} is supported)`) + } + } + + // --- write path (session/event → flush drain) --- + + private installWritePath(): void { + const ctx = this.ctx + + // Capture the header on creation; persist a fork's seed once. Record the init + // promise so flush/dispose can await it (onCreated is async). + ctx.on('session/created', (session) => { void this.initFor(session) }) + + // Snapshot + buffer every event (the live object is mutable; clone so a later + // in-place mutation cannot rewrite a buffered event). Serializability is + // guaranteed at the source (Session.append), so structuredClone is safe. + ctx.on('session/event', (session, event) => { + let buffer = this.buffers.get(session) + if (!buffer) this.buffers.set(session, buffer = []) + buffer.push(structuredClone(event)) + }) + + // Drain to the backend at the durability checkpoint. + ctx.on('session/flush', session => this.flush(session)) + + // Dispose must reach quiescence: await every init + final drain BEFORE + // returning, then close the backend's own resources (AFTER the drain), so no + // write lands after teardown and a close failure never MASKS a drain error. + ctx.effect(() => async () => { + let disposeError: unknown + try { + const errors = [ + ...await settledErrors(this.inits.values()), + ...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))), + ...await settledErrors(this.chains.values()), + ] + if (errors.length > 0) { + throw new AggregateError(errors, `${this.backend.name} dispose failed`) + } + } catch (error: unknown) { + disposeError = error + throw error + } finally { + try { + await this.backend.close?.() + } catch (closeError: unknown) { + // A close failure can only add teardown context; keep the already- + // captured drain AggregateError as the primary failure rather than + // masking it. Only surface the close error if the drain succeeded. + /* v8 ignore start -- close failure racing disposal is a defensive teardown edge */ + if (disposeError === undefined) throw closeError + /* v8 ignore stop */ + } + } + }, `${this.backend.name} write path`) + + // HMR: a hot reload does not replay session/created, so seed existing live + // sessions (mirrors dsh-invariants). + for (const session of ctx.sessions.list()) void this.initFor(session) + } + + /** Start (once) the async init for a session and remember its promise. */ + private initFor(session: Session): Promise { + const existing = this.inits.get(session) + if (existing) return existing + // Snapshot the seed SYNCHRONOUSLY — initFor runs inside the `session/created` + // emit, before any later `append` adds non-seed events. A clone freezes it + // against later mutation of the live event objects. + const seed = session.events.map(e => structuredClone(e)) + const p = this.onCreated(session, seed) + // Attach a no-op rejection handler so a failing init does not surface as an + // unhandled rejection if no flush observes `p` before it rejects. The REAL + // error is still delivered: flush/dispose await the same `p` from the map. + p.catch(() => { /* observed by flush/dispose via the stored promise */ }) + this.inits.set(session, p) + return p + } + + /** + * Whether a live session's `seed` reproduces the first `cursor` persisted + * events. A `cursor` of 0 (nothing persisted yet) trivially matches. Used when + * a live session claims ownerless state left by a prior `load()`/`create()`. + */ + private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise { + if (cursor === 0) return true + const stored = await this.backend.loadStored(id) + /* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */ + if (stored === undefined) return false + return seedCoversPrefix(seed, stored.events.slice(0, cursor)) + } + + /** + * On session/created: sync the backend's in-memory state to a live Session. + * + * Cases, by whether this backend tracks the id and whether an artifact exists: + * 1. Already tracked → no-op (or claim ownerless state if the seed matches, + * or reclaim a truly-abandoned id, else reject as a collision). + * 2. Not tracked, an artifact EXISTS at this cwd and is a seq-aligned PREFIX + * of the live events → ADOPT it (HMR/reload), persisting any live suffix. + * 3. Not tracked, an artifact EXISTS but is NOT a prefix → REJECT (collision). + * 4. Not tracked and NO artifact → a genuinely new session: register meta + * (lazy) and persist its seed once. + */ + private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise { + const id = session.header.id + const tracked = this.states.get(id) + if (tracked !== undefined) { + // case 1: already tracked. + /* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */ + if (tracked.owner === session) return + if (tracked.owner === undefined) { + // Ownerless state from the public create()/load() API. The FIRST live + // session claims it — but ONLY if BOTH the cwd scope and the seed match. + // The cwd guard mirrors case-2's cwd-scoped loadLive(): a same-id + // ownerless artifact at a DIFFERENT cwd is a collision, not a claim + // (claiming it would append the live cwd's events under the stored + // header's cwd, the exact cross-cwd corruption the loadLive scope + // prevents). The seed guard then ensures the live events reproduce the + // persisted prefix (else a fresh, unrelated session reusing the id would + // have its seq 0..cursor-1 events filtered as already-written and + // grafted on). + if (tracked.meta.cwd !== session.header.cwd) { + throw new Error(`session "${id}" is already persisted at a different cwd (persisted: ${String(tracked.meta.cwd)}, live: ${String(session.header.cwd)}) (id collision)`) + } + if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) { + throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`) + } + tracked.owner = session + // Persist the seed SUFFIX beyond the persisted prefix. Constructor seed + // events never emit session/event, so the buffer never sees them. + const suffix = seed.slice(tracked.cursor) + if (suffix.length > 0) await this.append(id, suffix) + return + } + // Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id + // (never materialized, no pending buffer); else it is a real collision. + const ownerBuffer = this.buffers.get(tracked.owner) + if (!tracked.materialized && !ownerBuffer?.length) { + this.states.delete(id) + } else { + throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`) + } + } + + // case 2/3: an artifact at THIS cwd is adopted as a live prefix (or rejected + // as a collision inside adoptLivePrefix). cwd-scoped (loadLive), never + // any-scope: a same-id artifact at a different cwd is a collision, not a + // resume. + const live = await this.backend.loadLive(id, session.header.cwd) + if (live !== undefined) { + // Do NOT route through loadCore(): that crash-repairs open turns as + // interrupted, which is wrong for HMR while the live Session is still the + // authority and may append the real step/turn end later. + await this.serialize(id, () => this.adoptLivePrefix(session, seed, live)) + return + } + + // case 4: a genuinely new session. Register its meta (lazy), then persist its + // seed (events present at creation time) once. + const meta: SessionHeader = { ...session.header } + await this.create(meta) + // Bind this state to the live session so a later DIFFERENT session reusing + // the id is detected as a collision (case 1) rather than silently no-opped. + const created = this.states.get(id) + /* v8 ignore next -- create() always sets the state for the id */ + if (created !== undefined) created.owner = session + if (seed.length > 0) await this.append(id, seed) + } + + /** + * Adopt a stored prefix as a live session's history (HMR/reload): verify the + * seed covers the stored prefix, truncate any torn tail (NOT the open turn — + * the live Session is still the authority), bind ownership, and persist the + * live suffix that was ahead of the stored prefix. + */ + private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix): Promise { + const { meta, events, tornMarker } = stored + this.assertVersion(meta) + if (!seedCoversPrefix(seed, events)) { + throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`) + } + // Truncate-only repair (no closers): the open turn is NOT closed here. + if (tornMarker !== undefined) await this.backend.commitRepair(meta, tornMarker, []) + this.states.set(session.header.id, { + meta: { ...meta }, + cursor: events.length, + materialized: true, + owner: session, + }) + const suffix = seed.slice(events.length) + if (suffix.length > 0) await this.appendCore(session.header.id, suffix) + } + + private async flush(session: Session): Promise { + // Wait for the session's init (onCreated) so the state/cursor and any + // fork-seed persistence are in place before draining. Awaiting the same + // promise initFor stored also surfaces an init failure (e.g. a collision) + // here, where the caller of session/flush observes it. + await this.inits.get(session) + // Serialize the WHOLE drain (read cursor → append → splice) on the per-session + // chain so two concurrent flushes cannot both read the same cursor and + // seq-mismatch on the second append. + await this.serialize(session.header.id, () => this.drain(session)) + } + + /** Drain a session's write buffer to the backend. Caller serializes this per id. */ + private async drain(session: Session): Promise { + const buffer = this.buffers.get(session) + if (!buffer?.length) return + // Copy WITHOUT removing: the buffer is the only durable-pending copy of these + // events. Drain it only AFTER the append commits; events pushed during the + // await sit past batch.length and survive the prefix splice, so a + // retry/dispose re-drains the rest. + const batch = buffer.slice() + const state = this.states.get(session.header.id) + // Only append events at or beyond the write cursor (a resumed session's seed + // is already stored). flush awaits the init above, which always sets state, + // so the `?? 0` fallback is a defensive guard that never fires in practice. + /* v8 ignore next -- state is always set by the awaited init before flush */ + const cursor = state?.cursor ?? 0 + const fresh = batch.filter(e => e.seq >= cursor) + // appendCore (NOT the serialized append) — drain already runs inside the + // per-session chain, so re-entering via append() would deadlock. + if (fresh.length > 0) await this.appendCore(session.header.id, fresh) + buffer.splice(0, batch.length) + } +} diff --git a/packages/session-persistence/src/index.ts b/packages/session-persistence/session-persistence/src/index.ts similarity index 84% rename from packages/session-persistence/src/index.ts rename to packages/session-persistence/session-persistence/src/index.ts index 0ce1e25146..a9ffd11792 100644 --- a/packages/session-persistence/src/index.ts +++ b/packages/session-persistence/session-persistence/src/index.ts @@ -1,7 +1,7 @@ /** * The durable session-persistence seam (`ctx.sessionPersistence`): an abstract * service defining WHAT a persistence backend does — durably store, reload, - * list, and update sessions — without saying HOW. Implementations subclass + * and list sessions — without saying HOW. Implementations subclass * {@link SessionPersistence} and register themselves as the * `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl` * (an append-only JSONL log per session) is the first and @@ -15,7 +15,7 @@ * parallel "persisted message" type the log must be converted to and from * (faithful to the event-sourced model: the log is the single source of * truth). Metadata that is NOT replayable conversation state (format version, - * cwd, lineage) travels separately as {@link SessionMeta}, which is owned by + * cwd, lineage) travels separately as {@link SessionHeader}, which is owned by * `dsh-session` and re-exported here. * * @module @deepseek-ai/dsh-session-persistence @@ -23,10 +23,14 @@ import { Context, Service } from 'cordis' import { isJsonValue } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' // Re-export the metadata vocabulary so consumers import it from the seam. -export type { SessionHeader, SessionSummary, SessionMeta } from '@deepseek-ai/dsh-session' +export type { SessionHeader } from '@deepseek-ai/dsh-session' + +// The backend-agnostic write-path orchestration first-party backends compose. +export { PersistenceCoordinator } from './coordinator.ts' +export type { PersistenceBackend, StoredPrefix } from './coordinator.ts' declare module 'cordis' { interface Context { @@ -99,10 +103,10 @@ export abstract class SessionPersistence extends Service { /** * Register a new session's metadata. A backend MAY defer the physical write * until the first {@link append} (lazy materialization), in which case a - * created-but-never-appended session is absent from {@link has}/{@link list} + * created-but-never-appended session is absent from {@link list} * — abandoned sessions leave nothing behind. */ - abstract create(meta: SessionMeta): Promise + abstract create(meta: SessionHeader): Promise /** * Durably persist a batch of events (called from the write-behind drain at @@ -114,7 +118,7 @@ export abstract class SessionPersistence extends Service { abstract append(id: SessionId, events: readonly SessionEvent[]): Promise /** - * Reload a session: its {@link SessionMeta} plus the event log up to the last + * Reload a session: its {@link SessionHeader} plus the event log up to the last * durable checkpoint. Returns `meta` AND `events` so the live session is * reconstructed with its `cwd`/lineage, not just its log. * @@ -135,24 +139,10 @@ export abstract class SessionPersistence extends Service { * unloadable (reject). Rejects an unknown format `version`. See the session-persistence RFC for * the crash-recovery contract. */ - abstract load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> + abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> /** Lightweight listing from metadata, without a full-log parse. */ - abstract list(): Promise - - /** Whether a session is durably present (materialized). */ - abstract has(id: SessionId): Promise - - /** Remove a session and all its persisted artifacts. */ - abstract delete(id: SessionId): Promise - - /** - * Update mutable metadata ({@link SessionSummary}: `updatedAt`, `title`, - * `firstPrompt`) WITHOUT touching the append-only event log. A backend - * stores the summary beside the log (a sidecar file, a header row) and - * rewrites only it. - */ - abstract update(id: SessionId, summary: Partial): Promise + abstract list(): Promise } export default SessionPersistence diff --git a/packages/session-persistence/tests/contract.ts b/packages/session-persistence/session-persistence/tests/contract.ts similarity index 83% rename from packages/session-persistence/tests/contract.ts rename to packages/session-persistence/session-persistence/tests/contract.ts index 73f6f653bd..a0f0e7bfa0 100644 --- a/packages/session-persistence/tests/contract.ts +++ b/packages/session-persistence/session-persistence/tests/contract.ts @@ -8,9 +8,9 @@ * @module @deepseek-ai/dsh-session-persistence/tests/contract */ -import { describe, expect, it, vi } from 'vitest' -import { SessionId } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session' +import { describe, expect, it } from 'vitest' +import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import { CallId } from '@deepseek-ai/dsh-llm' import type { SessionPersistence } from '../src/index.ts' @@ -20,13 +20,12 @@ export interface ContractBackend { dispose: () => Promise } -/** Build a minimal {@link SessionMeta} for a session id. */ -export function meta(id: string, cwd?: string): SessionMeta { +/** Build a minimal {@link SessionHeader} for a session id. */ +export function meta(id: string, cwd?: string): SessionHeader { return { - version: 1, + version: SESSION_FORMAT_VERSION, id: SessionId(id), createdAt: 1000, - updatedAt: 1000, ...cwd !== undefined ? { cwd } : {}, } } @@ -58,7 +57,7 @@ export function runPersistenceContract(name: string, make: () => Promise Promise { + it('list() excludes a created-but-never-appended (zero-event) session', async () => { const { persistence, dispose } = await make() try { await persistence.create(meta('empty')) - expect(await persistence.has(SessionId('empty'))).toBe(false) expect((await persistence.list()).map(m => m.id)).not.toContain(SessionId('empty')) } finally { await dispose() } }) - it('has()/list() include a session once it has events', async () => { + it('list() includes a session once it has events', async () => { const { persistence, dispose } = await make() try { const m = meta('s2') await persistence.create(m) await persistence.append(m.id, oneTurnLog()) - expect(await persistence.has(m.id)).toBe(true) expect((await persistence.list()).map(x => x.id)).toContain(m.id) } finally { await dispose() @@ -228,45 +225,5 @@ export function runPersistenceContract(name: string, make: () => Promise { - const { persistence, dispose } = await make() - try { - const m = meta('s6') - await persistence.create(m) - await persistence.append(m.id, oneTurnLog()) - expect(await persistence.has(m.id)).toBe(true) - await persistence.delete(m.id) - expect(await persistence.has(m.id)).toBe(false) - } finally { - await dispose() - } - }) - - it('update mutates summary fields without touching the event log', async () => { - const { persistence, dispose } = await make() - try { - const m = meta('s7') - const log = oneTurnLog() - await persistence.create(m) - await persistence.append(m.id, log) - const beforeUpdate = (await persistence.load(m.id)).meta.updatedAt - vi.useFakeTimers() - vi.setSystemTime(beforeUpdate + 1_000) - try { - await persistence.update(m.id, { title: 'My session', firstPrompt: 'hi' }) - } finally { - vi.useRealTimers() - } - - const loaded = await persistence.load(m.id) - expect(loaded.meta.title).toBe('My session') - expect(loaded.meta.firstPrompt).toBe('hi') - expect(loaded.meta.updatedAt).toBe(beforeUpdate + 1_000) - expect(loaded.events).toEqual(log) // log untouched - } finally { - await dispose() - } - }) }) } diff --git a/packages/session-persistence/session-persistence/tests/coordinator-contract.ts b/packages/session-persistence/session-persistence/tests/coordinator-contract.ts new file mode 100644 index 0000000000..431d02b4cb --- /dev/null +++ b/packages/session-persistence/session-persistence/tests/coordinator-contract.ts @@ -0,0 +1,775 @@ +/** + * Reusable ORCHESTRATION suite for any backend that composes a + * {@link PersistenceCoordinator}. Where {@link runPersistenceContract} (in + * contract.ts) pins the public read/write SEMANTICS, this suite pins the + * coordinator's WRITE-PATH ORCHESTRATION — the behavior that is identical across + * every first-party backend because it lives in the shared coordinator, not in + * the storage primitives: the `session/created` → `session/event` → + * `session/flush` → dispose drain, lazy materialization, fork-seed persistence, + * the four `onCreated` adoption cases (new / HMR-adopt / collision / + * ownerless-claim), crash-tail repair on load, and dispose-time quiescence. + * + * A backend imports {@link runCoordinatorContract} and calls it with a + * {@link CoordinatorFixture} factory that knows how to (a) mount the REAL + * backend plugin on a {@link Context} over a SHARED storage scope (so HMR/reload + * tests can dispose one instance and mount another over the same bytes/rows), + * and (b) inject a never-committed torn tail for one session + * ({@link CoordinatorFixture.corruptTail}) so the through-coordinator torn-tail + * repair branch is exercised against real storage. The suite drives everything + * through the PUBLIC {@link SessionPersistence} API + the cordis SessionStore + * write path — never the storage primitives directly — so it runs unchanged for + * every backend (memory / jsonl / sqlite). + * + * Each scenario lives here once and runs once per backend through the fixture; + * the per-backend specs keep ONLY their storage-mechanics tests. + * + * @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract + */ + +import { describe, expect, it } from 'vitest' +import { Context, type Fiber } from 'cordis' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import type { SessionPersistence } from '../src/index.ts' +import { meta, oneTurnLog } from './contract.ts' + +/** + * The backend-specific capabilities the orchestration suite needs beyond the + * public service API. A fresh fixture is created per test (isolated storage); + * the suite mounts/disposes backend instances on it and cleans it up at the end. + */ +export interface CoordinatorFixture { + /** + * Mount the REAL backend plugin (via `ctx.plugin`, the Loader path) on `ctx`, + * over THIS fixture's shared storage scope. Returns the plugin fiber so the + * suite can dispose a single instance (HMR/reload) while the storage — and any + * still-live session in another fiber — survives. The caller has already + * mounted `SessionStore` on `ctx`. + */ + mount: (ctx: Context) => Promise + + /** + * Inject a NEVER-COMMITTED torn tail into the backend's storage for `id` at + * the given `cwd` (the cwd the session was created with): a half-written + * record past the committed region (JSONL: a partial line with no newline; + * SQLite: a row with invalid `data` JSON past the committed seq). This drives + * the coordinator's `loadCore` `tornMarker !== undefined` → `commitRepair` + * branch against real storage. + * + * OMITTED by a backend that structurally has no torn tails (memory): the + * torn-tail scenario then self-skips (asserted explicitly in the suite). + */ + corruptTail?: (id: SessionId, cwd: string | undefined) => Promise + + /** Tear down the storage scope (remove the temp dir / file). */ + cleanup: () => Promise +} + +/** A constant absolute cwd; jsonl keys directories off it, memory/sqlite ignore it. */ +const WORK = '/w' +const OTHER = '/other' + +/** The per-session init map a backend exposes for white-box init awaits. */ +function inits(persistence: SessionPersistence): Map> { + return (persistence as unknown as { inits: Map> }).inits +} + +/** Append a whole event log to a live session, event by event (drives session/event). */ +function send(session: Session, events: readonly SessionEvent[]): void { + for (const e of events) session.append(e.type, e.data) +} + +/** A live session created inside its OWN fiber, so it survives a backend reload. */ +async function liveSessionInFiber( + ctx: Context, id: string, cwd: string | undefined, +): Promise { + let session!: Session + await ctx.plugin(Object.assign((inner: Context) => { + session = inner.sessions.create(SessionId(id), cwd !== undefined ? { meta: { cwd } } : undefined) + }, { inject: ['sessions'] })) + return session +} + +/** + * Run the coordinator orchestration suite against a backend. `makeFixture()` + * MUST return a fresh fixture (isolated storage) each call. + */ +export function runCoordinatorContract(name: string, makeFixture: () => Promise): void { + describe(`PersistenceCoordinator orchestration: ${name}`, () => { + /** Mount SessionStore + a backend instance on a fresh context over the fixture's storage. */ + async function freshCtx(fix: CoordinatorFixture): Promise<{ ctx: Context; fiber: Fiber }> { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await fix.mount(ctx) + return { ctx, fiber } + } + + // --- write path: live session → flush → reload --- + + it('persists a live session driven through the store, surviving reload', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const session = ctx.sessions.create(SessionId('live'), { meta: { cwd: WORK } }) + send(session, oneTurnLog()) + await ctx.parallel('session/flush', session) + + const loaded = await ctx.sessionPersistence.load(SessionId('live')) + expect(loaded.events).toHaveLength(6) + expect(loaded.meta.cwd).toBe(WORK) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const session = ctx.sessions.create(SessionId('mutate'), { meta: { cwd: WORK } }) + const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } }) + // Mutate the live event object AFTER it was buffered by session/event. + ;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED' + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + const loaded = await ctx.sessionPersistence.load(SessionId('mutate')) + const first = loaded.events[0] + expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original') + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('append snapshots the batch: mutating the caller array/events after the call is ignored', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = meta('snapshot', WORK) + await ctx.sessionPersistence.create(m) + const events = oneTurnLog() // seqs 0..5 + const userMsg = events[1] // the user/message event + const p = ctx.sessionPersistence.append(m.id, events) + // Mutate the caller's array AND an event object after the call but before + // the queued op runs: the snapshot taken at call time must shield the copy. + events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }) + if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }] + await p + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6 + const persisted = JSON.stringify(loaded.events) + expect(persisted).toContain('hi') // original content + expect(persisted).not.toContain('MUTATED') + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- fork / resume --- + + it('fork: a seeded new session persists its seed once (no double-write on a no-op flush)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const seed = oneTurnLog() + // A fork: a brand-new id whose seed came from elsewhere. + const forked = ctx.sessions.create(SessionId('forked'), { seed, meta: { cwd: WORK } }) + await inits(ctx.sessionPersistence).get(forked) // onCreated persisted the seed + const loaded = await ctx.sessionPersistence.load(SessionId('forked')) + expect(loaded.events).toEqual(seed) + // A flush with no NEW events must not double-write. + await ctx.parallel('session/flush', forked) + const reloaded = await ctx.sessionPersistence.load(SessionId('forked')) + expect(reloaded.events).toEqual(seed) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('resume: a re-created session seeded with the loaded log does not re-append its seed and continues the seq', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + // First lifecycle: persist a session through the store. + const s1 = first.ctx.sessions.create(SessionId('resumed'), { meta: { cwd: WORK } }) + send(s1, oneTurnLog()) + await first.ctx.parallel('session/flush', s1) + } finally { + await first.fiber.dispose() + } + + // Second lifecycle: a NEW backend instance + a session re-created with the + // same id SEEDED with the loaded events. onCreated adopts the stored log + // (does not re-persist the seed); a new turn appends at seq 6. + const second = await freshCtx(fix) + try { + const loaded = await second.ctx.sessionPersistence.load(SessionId('resumed')) + const s2 = second.ctx.sessions.create(SessionId('resumed'), { seed: loaded.events, meta: { cwd: WORK } }) + await inits(second.ctx.sessionPersistence).get(s2) // let onCreated adopt + s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) + s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) + await second.ctx.parallel('session/flush', s2) + + const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed')) + // 6 original + 2 new, contiguous, no duplicated seed. + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + // --- HMR --- + + it('HMR: applying the plugin seeds existing live sessions', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + // A session exists BEFORE the persistence plugin is applied. + const session = ctx.sessions.create(SessionId('pre-existing'), { meta: { cwd: WORK } }) + session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + + const fiber = await fix.mount(ctx) + try { + // The plugin seeded it on apply; a subsequent flush persists its events. + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing')) + expect(loaded.events.length).toBeGreaterThanOrEqual(2) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR: dispose drains remaining buffers', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await fix.mount(ctx) + const session = await liveSessionInFiber(ctx, 'drain', WORK) + session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + // No explicit flush — dispose must drain. + await fiber.dispose() + + // A fresh backend instance reads what the disposed one drained. + const second = await freshCtx(fix) + try { + const loaded = await second.ctx.sessionPersistence.load(SessionId('drain')) + expect(loaded.events.length).toBeGreaterThanOrEqual(2) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + // The session lives in its OWN fiber so it survives the backend reload. + const session = await liveSessionInFiber(ctx, 'hmr-adopt', WORK) + try { + // Backend instance 1 materializes the session. + const backend1 = await fix.mount(ctx) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + // Hot-reload: dispose instance 1, mount instance 2 over the SAME storage + // while the session stays live. Instance 2 has an empty states map but the + // log is materialized and is a prefix of the live events — it must ADOPT + // (not reject). A second turn appended after reload then persists. + await backend1.dispose() + await fix.mount(ctx) + session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) + await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow() + + const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt')) + expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) + } finally { + await ctx.fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR: adoption persists the live SUFFIX that was ahead of the stored prefix', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + const session = await liveSessionInFiber(ctx, 'hmr-suffix', WORK) + try { + // Instance 1 flushes turn 1. + const backend1 = await fix.mount(ctx) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + // Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT + // flushing turn 2: it is now ONLY in the live session's events; the new + // backend never buffered it via session/event. + await backend1.dispose() + session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('turn/end', { turn: 2, reason: { kind: 'completed' } }) + + // Instance 2 adopts the stored prefix (turn 1) and MUST also persist the + // live suffix (turn 2) carried in the session's events. + await fix.mount(ctx) + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3]) + expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2) + } finally { + await ctx.fiber.dispose() + await fix.cleanup() + } + }) + + it('HMR adoption does NOT crash-repair an active open turn as interrupted (truncate without closers)', async () => { + const fix = await makeFixture() + const ctx = new Context() + await ctx.plugin(SessionStore) + const session = await liveSessionInFiber(ctx, 'hmr-open', WORK) + try { + const first = await fix.mount(ctx) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + await ctx.parallel('session/flush', session) + + // Crash-tail a torn fragment past the (open) committed turn, then reload. + await first.dispose() + if (fix.corruptTail) await fix.corruptTail(SessionId('hmr-open'), WORK) + const second = await fix.mount(ctx) + // The live session is still the authority: it appends the REAL step/turn + // end. Adoption must truncate the torn tail but NOT synthesize closers. + session.append('step/end', { turn: 1, step: 1 }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + + const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open')) + expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end']) + expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } }) + await second.dispose() + } finally { + await ctx.fiber.dispose() + await fix.cleanup() + } + }) + + // --- collision / id reuse --- + + it('a NEW live session colliding on a persisted id is rejected, not silently adopted', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + const s1 = first.ctx.sessions.create(SessionId('collide'), { meta: { cwd: WORK } }) + send(s1, oneTurnLog()) + await first.ctx.parallel('session/flush', s1) + } finally { + await first.fiber.dispose() + } + + // A FRESH backend + a NEW live session with the same id but NO explicit + // resume. onCreated treats it as new; create() rejects because a log already + // exists. The rejection surfaces via the init promise (flush awaits it). + const second = await freshCtx(fix) + try { + const s2 = second.ctx.sessions.create(SessionId('collide'), { meta: { cwd: WORK } }) + s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + await expect(inits(second.ctx.sessionPersistence).get(s2)) + .rejects.toThrow(/already has a persisted log|id collision/) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // A live session created then disposed BEFORE its first append: cursor 0, + // never materialized. A new live session reusing the id must reclaim it. + let firstSession!: Session + const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { + firstSession = inner.sessions.create(SessionId('abandoned'), { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await inits(ctx.sessionPersistence).get(firstSession) // register the lazy state + await firstFiber.dispose() // disposed before any append → never materialized + + let reuse!: Session + await ctx.plugin(Object.assign((inner: Context) => { + reuse = inner.sessions.create(SessionId('abandoned'), { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await expect(inits(ctx.sessionPersistence).get(reuse)).resolves.toBeUndefined() + reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', reuse) + const loaded = await ctx.sessionPersistence.load(SessionId('abandoned')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1]) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + let first!: Session + const firstFiber = await ctx.plugin(Object.assign((inner: Context) => { + first = inner.sessions.create(SessionId('buffered'), { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await inits(ctx.sessionPersistence).get(first) + // Append a turn but do NOT flush — events sit in the write-behind buffer. + first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + first.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await firstFiber.dispose() // disposed before flush; not materialized, buffer pending + + let reuse!: Session + await ctx.plugin(Object.assign((inner: Context) => { + reuse = inner.sessions.create(SessionId('buffered'), { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await expect(inits(ctx.sessionPersistence).get(reuse)).rejects.toThrow(/already bound to a different live session/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const session = ctx.sessions.create(SessionId('idem'), { meta: { cwd: WORK } }) + session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + // Re-emit session/created for the SAME live session (idempotent initFor). + ctx.emit('session/created', session) + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('idem')) + expect(loaded.events).toHaveLength(2) // not doubled + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- ownerless-state claim (public create()/load() then a live session arrives) --- + + it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // create() registers ownerless state with cursor 0 (lazy, nothing persisted). + await ctx.sessionPersistence.create(meta('lazy-claim', WORK)) + // A live session with that id arrives and claims it (cursor 0 matches + // trivially), persisting its seed. + const live = ctx.sessions.create(SessionId('lazy-claim'), { seed: oneTurnLog(), meta: { cwd: WORK } }) + await expect(inits(ctx.sessionPersistence).get(live)).resolves.toBeUndefined() + const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Materialize a log, then load() it WITHOUT a live session — ownerless + // state, cursor at the persisted length. + await ctx.sessionPersistence.create(meta('preview', WORK)) + await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog()) + await ctx.sessionPersistence.load(SessionId('preview')) + + // A FRESH (empty-seed) live session reusing that id must be rejected: its + // seq 0..cursor-1 events would otherwise be filtered as already-persisted. + let fresh!: Session + await ctx.plugin(Object.assign((inner: Context) => { + fresh = inner.sessions.create(SessionId('preview'), { meta: { cwd: WORK } }) + }, { inject: ['sessions'] })) + await expect(inits(ctx.sessionPersistence).get(fresh)) + .rejects.toThrow(/do not match this live session|already has a persisted log|id collision/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Materialize and load (ownerless, cursor = 6). + await ctx.sessionPersistence.create(meta('claim', WORK)) + await ctx.sessionPersistence.append(SessionId('claim'), oneTurnLog()) + const { events } = await ctx.sessionPersistence.load(SessionId('claim')) + + // A live session SEEDED with the loaded log PLUS a new turn claims the + // ownerless state and persists only the suffix. + const cont = ctx.sessions.create(SessionId('claim'), { seed: [ + ...events, + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, + ], meta: { cwd: WORK } }) + await inits(ctx.sessionPersistence).get(cont) + const loaded = await ctx.sessionPersistence.load(SessionId('claim')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a live session at a DIFFERENT cwd cannot claim cursor-0 ownerless state (cwd scope)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // create() registers ownerless state at cwd /a (cursor 0 — claims would + // otherwise match trivially on the seed). + await ctx.sessionPersistence.create(meta('wrong-cwd-claim', OTHER)) + // A live session reusing the id but at cwd WORK must NOT claim it — the + // cwd scope is the fence (without it, WORK events would append under the + // OTHER header). Rejected as a collision. + const live = ctx.sessions.create(SessionId('wrong-cwd-claim'), { seed: oneTurnLog(), meta: { cwd: WORK } }) + await expect(inits(ctx.sessionPersistence).get(live)).rejects.toThrow(/different cwd|id collision/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a live session at a DIFFERENT cwd cannot claim loaded-prefix ownerless state (cwd scope)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Materialize + load at cwd OTHER (ownerless, cursor = 6). + await ctx.sessionPersistence.create(meta('wrong-cwd-load', OTHER)) + await ctx.sessionPersistence.append(SessionId('wrong-cwd-load'), oneTurnLog()) + const { events } = await ctx.sessionPersistence.load(SessionId('wrong-cwd-load')) + // A live session whose SEED matches the loaded prefix but whose cwd is + // WORK must still be rejected — the cwd guard runs before the seed check. + const live = ctx.sessions.create(SessionId('wrong-cwd-load'), { seed: events, meta: { cwd: WORK } }) + await expect(inits(ctx.sessionPersistence).get(live)).rejects.toThrow(/different cwd|id collision/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('a no-cwd ownerless state cannot be claimed by a live session WITH a cwd (cwd scope, undefined side)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Ownerless state created WITHOUT a cwd (the no-cwd bucket). + await ctx.sessionPersistence.create(meta('no-cwd-state')) + // A live session reusing the id but WITH cwd WORK is a cwd mismatch + // (undefined vs WORK) and must be rejected. + const live = ctx.sessions.create(SessionId('no-cwd-state'), { seed: oneTurnLog(), meta: { cwd: WORK } }) + await expect(inits(ctx.sessionPersistence).get(live)).rejects.toThrow(/different cwd|id collision/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- append adopts a storage-only session (fresh instance, no prior create/load) --- + + it('append adopts a storage-only session (fresh instance) and continues the seq', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + const m = meta('adopt-append', WORK) + await first.ctx.sessionPersistence.create(m) + await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) + } finally { + await first.fiber.dispose() + } + + // A fresh instance appends a second turn WITHOUT a prior create/load: append + // must adopt the stored session (cursor = stored length) and continue. + const second = await freshCtx(fix) + try { + await second.ctx.sessionPersistence.append(SessionId('adopt-append'), [ + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, + ]) + const loaded = await second.ctx.sessionPersistence.load(SessionId('adopt-append')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7]) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + // --- small public-API edges that the coordinator owns uniformly --- + + it('append of an empty batch is a no-op (stays lazy)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = meta('empty-batch', WORK) + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, []) + expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('load rejects a missing session', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('create rejects a duplicate id (in memory and on a persisted log)', async () => { + const fix = await makeFixture() + const first = await freshCtx(fix) + try { + const m = meta('dup', WORK) + await first.ctx.sessionPersistence.create(m) + // Same in-memory state. + await expect(first.ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists in this backend/) + await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) + } finally { + await first.fiber.dispose() + } + + // A fresh instance over the same storage sees the persisted log. + const second = await freshCtx(fix) + try { + await expect(second.ctx.sessionPersistence.create(meta('dup', WORK))) + .rejects.toThrow(/already has a persisted log on disk/) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + + it('rejects an unknown format version on load (assertVersion)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = { version: 99, id: SessionId('v99'), createdAt: 1, cwd: WORK } + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/version/) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('round-trips a header with parentSession (fork lineage)', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + const m = { version: SESSION_FORMAT_VERSION, id: SessionId('forked-child'), createdAt: 1, cwd: WORK, parentSession: SessionId('the-parent') } + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.meta.parentSession).toBe('the-parent') + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + it('flush before init resolves uses cursor 0', async () => { + const fix = await makeFixture() + const { ctx, fiber } = await freshCtx(fix) + try { + // Append directly to a live session and flush IMMEDIATELY, before the + // async onCreated init has necessarily set state (exercises the + // state-undefined cursor path). + const session = ctx.sessions.create(SessionId('flush-nostate'), { meta: { cwd: WORK } }) + session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.parallel('session/flush', session) + const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate')) + expect(loaded.events).toHaveLength(2) + } finally { + await fiber.dispose() + await fix.cleanup() + } + }) + + // --- crash-tail repair THROUGH the coordinator (real storage torn tail) --- + + it('torn-tail load: a never-committed tail is truncated and the open turn closed during load (commitRepair w/ tornMarker)', async () => { + const fix = await makeFixture() + if (!fix.corruptTail) { + // A memory-style store has no torn tails (every write is atomic in RAM), + // so there is no tornMarker path to exercise. Assert that explicitly + // instead of silently skipping, then bail. + expect(fix.corruptTail).toBeUndefined() + await fix.cleanup() + return + } + const first = await freshCtx(fix) + try { + const m = meta('torn', WORK) + await first.ctx.sessionPersistence.create(m) + await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed 0..5 (balanced) + // A second turn whose real events are durable but never closed (open turn). + await first.ctx.sessionPersistence.append(m.id, [ + { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, + ]) + } finally { + await first.fiber.dispose() + } + // Inject a torn fragment past the committed region (never-committed tail). + await fix.corruptTail(SessionId('torn'), WORK) + + // A FRESH instance loads: the torn tail is truncated (tornMarker !== + // undefined) AND the open turn 2 is closed with synthetic step/end + + // turn/end {interrupted} — commitRepair runs with BOTH a torn marker and + // closers. The preserved real events (0..7) are never truncated. + const second = await freshCtx(fix) + try { + const loaded = await second.ctx.sessionPersistence.load(SessionId('torn')) + expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) + expect(loaded.events.map(e => e.type)).toEqual([ + 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1 + 'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real + synthetic closers + ]) + const last = loaded.events.at(-1)! + expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' }) + + // The repair is durable: the next append continues at the balanced length + // (seq 10) and a reload round-trips identically. + await second.ctx.sessionPersistence.append(SessionId('torn'), [ + { type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } }, + { type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } }, + ]) + const reloaded = await second.ctx.sessionPersistence.load(SessionId('torn')) + expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) + } finally { + await second.fiber.dispose() + await fix.cleanup() + } + }) + }) +} diff --git a/packages/session-persistence/session-persistence/tests/persistence.spec.ts b/packages/session-persistence/session-persistence/tests/persistence.spec.ts new file mode 100644 index 0000000000..4e4cf67822 --- /dev/null +++ b/packages/session-persistence/session-persistence/tests/persistence.spec.ts @@ -0,0 +1,196 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import SessionStore, { SessionId, isJsonValue } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' +import { + SessionPersistence, PersistenceCoordinator, assertSerializable, seedCoversPrefix, + type PersistenceBackend, type StoredPrefix, +} from '../src/index.ts' +import { runPersistenceContract, meta, oneTurnLog } from './contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract.ts' + +/** The durable store shape: materialized sessions only (no lazy entries). */ +type MemoryStore = Map + +/** Optional plugin config: an EXTERNAL store shared across backend instances. */ +interface MemoryConfig { store?: MemoryStore } + +/** + * A trivial in-memory {@link SessionPersistence} that composes a + * {@link PersistenceCoordinator} over a dependency-free `Map`-backed + * {@link PersistenceBackend}. It is BOTH the coordinator's reference vehicle + * (the simplest possible storage — a `Map` with no torn + * tails, so `tornMarker` is always undefined) and the cover for the abstract + * base's constructor + service registration. The real durable backends are + * `@deepseek-ai/dsh-session-persistence-jsonl` / `-sqlite`. + * + * The store can be supplied via config so two backend instances share one Map — + * the in-RAM analogue of two backends over the same file/db, which the + * coordinator orchestration suite's HMR/reload tests need (a fresh instance with + * an empty in-memory states map adopting an already-materialized session). + */ +class MemoryPersistence extends SessionPersistence implements PersistenceBackend { + static inject = ['sessions'] + + override readonly name = 'session-persistence-memory' + + /** The whole durable store: materialized sessions only (no lazy entries). */ + private store: MemoryStore + private coordinator: PersistenceCoordinator + + constructor(ctx: Context, config?: MemoryConfig) { + super(ctx) + // Assign the store BEFORE constructing the coordinator: the coordinator's + // constructor installs the write path and synchronously seeds existing live + // sessions (onCreated → loadLive → this.store), so store must exist first. + this.store = config?.store ?? new Map() + this.coordinator = new PersistenceCoordinator(this.ctx, this) + } + + // --- service surface (delegated to the coordinator) --- + + create(m: SessionHeader): Promise { + return this.coordinator.create(m) + } + + append(id: SessionId, events: readonly SessionEvent[]): Promise { + return this.coordinator.append(id, events) + } + + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { + return this.coordinator.load(id) + } + + /** White-box accessor: await a specific session's onCreated init. */ + get inits(): Map> { + return this.coordinator.inits + } + + // --- PersistenceBackend hooks (the Map storage primitives) --- + + // A Map-backed store has no torn tails, so `tornMarker` is never set. Ids are + // globally unique, so loadStored and loadLive are identical (cwd is ignored). + async loadStored(id: SessionId): Promise | undefined> { + const entry = this.store.get(id) + if (!entry) return undefined + return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) } + } + + loadLive(id: SessionId, _cwd: string | undefined): Promise | undefined> { + return this.loadStored(id) + } + + async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise { + // Defense-in-depth: the coordinator already validates serializability, but a + // durable store must reject non-JSON data at its own boundary too. + for (const e of events) { + if (!isJsonValue(e.data)) throw new Error(`event "${e.type}" carries non-JSON-serializable data`) + } + const existing = this.store.get(m.id) + if (!existing) { + // First batch: `_isMaterialized` is false (the coordinator only omits + // materialization on the first batch); writing the entry IS the materialization. + this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] }) + } else { + existing.events.push(...structuredClone(events) as SessionEvent[]) + } + } + + async commitRepair(m: SessionHeader, _tornMarker: undefined, closers: readonly SessionEvent[]): Promise { + // No torn tails in a Map store, so `_tornMarker` is always undefined; only the + // synthetic closers are appended (the same DELETE+INSERT a DB backend does, + // minus the truncate). + const entry = this.store.get(m.id) + /* v8 ignore next -- commitRepair only runs for a materialized (stored) session */ + if (!entry) return + if (closers.length > 0) entry.events.push(...structuredClone(closers) as SessionEvent[]) + } + + async list(): Promise { + return [...this.store.values()].map(e => structuredClone(e.meta)) + } +} + +// Run the shared contract against the in-memory backend. +runPersistenceContract('memory', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(MemoryPersistence) + return { + persistence: ctx.sessionPersistence, + dispose: async () => { await fiber.dispose() }, + } +}) + +// Run the shared coordinator orchestration suite against the in-memory backend. +// A per-fixture Map is the shared "storage", so two mounted instances see the +// same materialized sessions (HMR/reload). `corruptTail` is OMITTED: a Map store +// writes atomically in RAM and has no torn tails, so the suite's torn-tail test +// self-skips (and asserts the omission). The real torn-tail repair branch is +// covered by the jsonl/sqlite fixtures, which CAN inject one. +runCoordinatorContract('memory', async (): Promise => { + const store: MemoryStore = new Map() + return { + mount: async ctx => ctx.plugin(MemoryPersistence, { store }), + cleanup: async () => { store.clear() }, + } +}) + +describe('SessionPersistence service registration', () => { + it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(MemoryPersistence) + expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence) + + await fiber.dispose() + expect(ctx.sessionPersistence).toBeUndefined() + }) + + it('round-trips through the registered service instance', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const fiber = await ctx.plugin(MemoryPersistence) + const m = meta('reg') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const loaded = await ctx.sessionPersistence.load(m.id) + expect(loaded.events).toHaveLength(6) + await fiber.dispose() + }) +}) + +describe('shared persistence helpers', () => { + it('accepts a seed that reproduces the persisted prefix exactly', () => { + const log = oneTurnLog() + expect(seedCoversPrefix(log, log.slice(0, 3))).toBe(true) + expect(seedCoversPrefix(log, [])).toBe(true) + }) + + it('rejects a prefix longer than the seed', () => { + const log = oneTurnLog() + expect(seedCoversPrefix(log.slice(0, 2), log)).toBe(false) + }) + + it('rejects a same-envelope event with mutated data', () => { + const log = oneTurnLog() + const tampered = structuredClone(log) + const event = tampered[1]! + tampered[1] = { + ...event, + data: { ...event.data, content: [{ type: 'text', text: 'tampered' }] }, + } as SessionEvent + expect(seedCoversPrefix(tampered, log.slice(0, 2))).toBe(false) + }) + + it('accepts JSON-serializable event data', () => { + expect(() => { assertSerializable(oneTurnLog()) }).not.toThrow() + }) + + it('rejects non-JSON-serializable event data with type and seq context', () => { + const bad = [ + { type: 'user/message', seq: 0, time: 1, data: { content: 1n } }, + ] as unknown as SessionEvent[] + expect(() => { assertSerializable(bad) }).toThrow(/"user\/message".*seq 0/) + }) +}) diff --git a/packages/session-persistence/session-persistence/tsconfig.json b/packages/session-persistence/session-persistence/tsconfig.json new file mode 100644 index 0000000000..e817086a6a --- /dev/null +++ b/packages/session-persistence/session-persistence/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../core/session" + } + ] +} diff --git a/packages/session-persistence/tests/persistence.spec.ts b/packages/session-persistence/tests/persistence.spec.ts deleted file mode 100644 index 5c0a29131f..0000000000 --- a/packages/session-persistence/tests/persistence.spec.ts +++ /dev/null @@ -1,141 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { Context } from 'cordis' -import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' -import { SessionPersistence, assertSerializable, seedCoversPrefix } from '../src/index.ts' -import { runPersistenceContract, meta, oneTurnLog } from './contract.ts' - -/** - * A minimal in-memory {@link SessionPersistence} used to (a) cover the abstract - * base's constructor + service registration and (b) validate the reusable - * contract suite itself. The real durable backend is - * `@deepseek-ai/dsh-session-persistence-jsonl`. - */ -class MemoryPersistence extends SessionPersistence { - private store = new Map() - private pending = new Map() - - async create(m: SessionMeta): Promise { - // Lazy: record the intended meta, but stay absent from has/list until the - // first append materializes the session. - this.pending.set(m.id, m) - } - - async append(id: SessionId, events: readonly SessionEvent[]): Promise { - const existing = this.store.get(id) - const nextSeq = existing ? existing.events.length : 0 - if (events.length > 0 && events[0]!.seq !== nextSeq) { - throw new Error(`append seq mismatch for "${id}": expected ${nextSeq}, got ${events[0]!.seq}`) - } - for (let i = 0; i < events.length; i++) { - const e = events[i]! - if (e.seq !== nextSeq + i) throw new Error(`non-contiguous seq in batch for "${id}" at index ${i}`) - if (!isJsonValue(e.data)) { - throw new Error(`event "${e.type}" carries non-JSON-serializable data`) - } - } - if (!existing) { - const m = this.pending.get(id) - if (!m) throw new Error(`append before create for "${id}"`) - this.store.set(id, { meta: m, events: structuredClone(events) as SessionEvent[] }) - } else { - existing.events.push(...structuredClone(events) as SessionEvent[]) - } - } - - async load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { - const entry = this.store.get(id) - if (!entry) throw new Error(`session "${id}" not found`) - // Honor the crash-recovery contract: if the stored log ends mid-turn, close - // the orphaned turn durably with synthetic boundary events and continue from - // the balanced length. - const closers = interruptedTurnClosers(entry.events) - if (closers.length > 0) entry.events.push(...structuredClone(closers)) - return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) } - } - - async list(): Promise { - return [...this.store.values()].map(e => structuredClone(e.meta)) - } - - async has(id: SessionId): Promise { - return this.store.has(id) - } - - async delete(id: SessionId): Promise { - this.store.delete(id) - this.pending.delete(id) - } - - async update(id: SessionId, summary: Partial): Promise { - const entry = this.store.get(id) - if (entry) Object.assign(entry.meta, summary, { updatedAt: summary.updatedAt ?? Date.now() }) - } -} - -// Run the shared contract against the in-memory backend. -runPersistenceContract('memory', async () => { - const ctx = new Context() - const fiber = await ctx.plugin(MemoryPersistence) - return { - persistence: ctx.sessionPersistence, - dispose: async () => { await fiber.dispose() }, - } -}) - -describe('SessionPersistence service registration', () => { - it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => { - const ctx = new Context() - const fiber = await ctx.plugin(MemoryPersistence) - expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence) - - await fiber.dispose() - expect(ctx.sessionPersistence).toBeUndefined() - }) - - it('round-trips through the registered service instance', async () => { - const ctx = new Context() - const fiber = await ctx.plugin(MemoryPersistence) - const m = meta('reg') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events).toHaveLength(6) - await fiber.dispose() - }) -}) - -describe('shared persistence helpers', () => { - it('accepts a seed that reproduces the persisted prefix exactly', () => { - const log = oneTurnLog() - expect(seedCoversPrefix(log, log.slice(0, 3))).toBe(true) - expect(seedCoversPrefix(log, [])).toBe(true) - }) - - it('rejects a prefix longer than the seed', () => { - const log = oneTurnLog() - expect(seedCoversPrefix(log.slice(0, 2), log)).toBe(false) - }) - - it('rejects a same-envelope event with mutated data', () => { - const log = oneTurnLog() - const tampered = structuredClone(log) - const event = tampered[1]! - tampered[1] = { - ...event, - data: { ...event.data, content: [{ type: 'text', text: 'tampered' }] }, - } as SessionEvent - expect(seedCoversPrefix(tampered, log.slice(0, 2))).toBe(false) - }) - - it('accepts JSON-serializable event data', () => { - expect(() => { assertSerializable(oneTurnLog()) }).not.toThrow() - }) - - it('rejects non-JSON-serializable event data with type and seq context', () => { - const bad = [ - { type: 'user/message', seq: 0, time: 1, data: { content: 1n } }, - ] as unknown as SessionEvent[] - expect(() => { assertSerializable(bad) }).toThrow(/"user\/message".*seq 0/) - }) -}) diff --git a/packages/session-persistence/tsconfig.json b/packages/session-persistence/tsconfig.json deleted file mode 100644 index 727294a720..0000000000 --- a/packages/session-persistence/tsconfig.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../session" } - ] -} diff --git a/packages/session/tsconfig.json b/packages/session/tsconfig.json deleted file mode 100644 index e226412a53..0000000000 --- a/packages/session/tsconfig.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../llm" } - ] -} diff --git a/packages/support/README.md b/packages/support/README.md new file mode 100644 index 0000000000..52f7f6fe25 --- /dev/null +++ b/packages/support/README.md @@ -0,0 +1,11 @@ +# support/ — dev/test/example infrastructure + +Packages that exist to serve development, testing, and the examples rather than to ship as product API. They are real workspace packages (typed, tested, under the coverage gate), but they carry **lower compatibility expectations**: they may change or be removed when the development need behind them does, without the deprecation care a product package would warrant. + +| Package | Role | ctx key | +|---|---|---| +| `invariants/` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) | +| `ui-stdio/` | Minimal stdio (readline) UI plugin: renders `agent/*` events, feeds stdin lines to the agent | (drives `ctx.agents`) | +| `llm-replay/` | Record/replay adapter: short-circuits `llm/stream` from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) | + +`invariants` runs only in dev mode (contract checks, not runtime behavior). `ui-stdio` and `llm-replay` were extracted from the examples for reuse and to bring them under the per-file coverage gate; they back the demos and the snapshot test tier. A package graduates OUT of `support/` into a product group only when it gains documented product consumers. diff --git a/packages/invariants/README.md b/packages/support/invariants/README.md similarity index 96% rename from packages/invariants/README.md rename to packages/support/invariants/README.md index 094b7e13a0..a08ccf01a7 100644 --- a/packages/invariants/README.md +++ b/packages/support/invariants/README.md @@ -44,7 +44,7 @@ On any violation it throws `InvariantError` (`code: 'INVARIANT'`). ## Why runtime, not deep-readonly types -A `DeepReadonly` is high type-noise across every log consumer, and a plugin can cast straight through it. A dev-mode freeze plus these assertions catch real corruption at zero production cost and zero type noise. The always-on half of that defense — cloning derived messages so request/adapter mutation can't reach back into the log — lives in `dsh-session`'s `deriveMessages`. This package is the dev-mode tripwire. See [dev-mode invariants](../../docs/rfc/implemented/2026-06-11-dev-invariants-over-deep-readonly.md). +A `DeepReadonly` is high type-noise across every log consumer, and a plugin can cast straight through it. A dev-mode freeze plus these assertions catch real corruption at zero production cost and zero type noise. The always-on half of that defense — cloning derived messages so request/adapter mutation can't reach back into the log — lives in `dsh-session`'s `deriveMessages`. This package is the dev-mode tripwire. See [dev-mode invariants](../../../docs/rfc/implemented/architecture/2026-06-11-dev-invariants-over-deep-readonly.md). ## Seeded sessions diff --git a/packages/invariants/package.json b/packages/support/invariants/package.json similarity index 83% rename from packages/invariants/package.json rename to packages/support/invariants/package.json index 7508d3e7d8..97d6160b03 100644 --- a/packages/invariants/package.json +++ b/packages/support/invariants/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/invariants/src/index.ts b/packages/support/invariants/src/index.ts similarity index 98% rename from packages/invariants/src/index.ts rename to packages/support/invariants/src/index.ts index b201e1ae43..804093380f 100644 --- a/packages/invariants/src/index.ts +++ b/packages/support/invariants/src/index.ts @@ -21,6 +21,7 @@ import type { Context } from 'cordis' import { HarnessError } from '@deepseek-ai/dsh-llm' +import type { CallId } from '@deepseek-ai/dsh-llm' import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent' import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' @@ -65,7 +66,7 @@ interface SessionTrace { * Tool-call ids issued in the OPEN step awaiting a result. Cleared at * `step/end` — a result must arrive in the same step as its call. */ - pendingCalls: Set + pendingCalls: Set } /** @@ -191,8 +192,8 @@ function checkEvent(trace: SessionTrace, event: SessionEvent): void { // turn as its commit/replay boundary (the JSONL backend treats anything // after the last turn/end as a crash tail), so a bare event between turns is // silently dropped on reload. The loop records queued user messages after - // turn/start, an idle agent.inject() wraps its context/message in a one-shot - // turn, and usage/error are only appended inside an open turn. A `default` + // turn/start, and an idle agent.inject() wraps its context/message in a + // one-shot turn. A `default` // (not an enumerated list) is deliberate: SessionEventMap is // merge-extensible, so a PLUGIN-added event type appended while idle must // also fail here rather than fall through and be dropped on resume. diff --git a/packages/invariants/tests/invariants.spec.ts b/packages/support/invariants/tests/invariants.spec.ts similarity index 97% rename from packages/invariants/tests/invariants.spec.ts rename to packages/support/invariants/tests/invariants.spec.ts index 1086a21bfb..b9add0479f 100644 --- a/packages/invariants/tests/invariants.spec.ts +++ b/packages/support/invariants/tests/invariants.spec.ts @@ -2,7 +2,7 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' import { CallId } from '@deepseek-ai/dsh-llm' import type { Agent } from '@deepseek-ai/dsh-agent' -import SessionStore from '@deepseek-ai/dsh-session' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import * as Invariants from '@deepseek-ai/dsh-invariants' import { InvariantError } from '@deepseek-ai/dsh-invariants' @@ -95,14 +95,12 @@ describe('session-log invariants', () => { .toThrow(/outside any open turn/) }) - it('rejects usage/error and plugin-added events appended outside any open turn', async () => { + it('rejects steering and plugin-added events appended outside any open turn', async () => { const { ctx } = await setup({ freeze: false }) const session = ctx.sessions.create() - // usage and error are turn-scoped: outside a turn they would land past the + // steering/message is turn-scoped: outside a turn it would land past the // commit boundary and be dropped on resume (the turn-enclosure RFC). - expect(() => session.append('usage', { turn: 1, step: 1, usage: { inputTokens: 1, outputTokens: 1 } })) - .toThrow(/outside any open turn/) - expect(() => session.append('error', { turn: 1, step: 1, message: 'boom' })) + expect(() => session.append('steering/message', { turn: 1, content: [{ type: 'text', text: 'go' }], source: { kind: 'user' } })) .toThrow(/outside any open turn/) // A PLUGIN-added (merge-extensible) event type is caught by the default too. expect(() => session.append('compaction/marker' as never, { foo: 'bar' } as never)) @@ -156,7 +154,7 @@ describe('session-log invariants', () => { session.append('step/start', { turn: 1, step: 1 }) session.append('tool/call', { turn: 1, step: 1, callId: CallId('c1'), name: 'echo', arguments: '{}' }) session.append('step/end', { turn: 1, step: 1 }) - session.append('turn/end', { turn: 1, reason: { kind: 'error', message: 'boom' } }) + session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, message: 'boom' } }) }).not.toThrow() }) @@ -175,8 +173,8 @@ describe('session-log invariants', () => { it('tracks turns per session independently', async () => { const { ctx } = await setup({ freeze: false }) - const a = ctx.sessions.create('a') - const b = ctx.sessions.create('b') + const a = ctx.sessions.create(SessionId('a')) + const b = ctx.sessions.create(SessionId('b')) a.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) // b is a fresh session — its own turn/start must not see a's open turn. expect(() => b.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })).not.toThrow() diff --git a/packages/support/invariants/tsconfig.json b/packages/support/invariants/tsconfig.json new file mode 100644 index 0000000000..8dca14c786 --- /dev/null +++ b/packages/support/invariants/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + }, + { + "path": "../../core/agent" + } + ] +} diff --git a/packages/support/llm-replay/README.md b/packages/support/llm-replay/README.md new file mode 100644 index 0000000000..91230cc113 --- /dev/null +++ b/packages/support/llm-replay/README.md @@ -0,0 +1,36 @@ +# @deepseek-ai/dsh-llm-replay + +A replay LLM plugin for keyless snapshot tests. It installs a single `llm/stream` waterfall listener that short-circuits the waterfall (never calls `next()`) and yields model streams reconstructed from a recorded **session JSONL** fixture — so a test can boot the real agent against a fixed model transcript with no API key. + +Its consumer is the ACP snapshot harness in `examples/acp-agent`, which loads this plugin (via `cordis.snapshot.yml`) in place of a real LLM adapter. The package exists so its derive/parse/replay logic falls under the per-file 100% coverage gate on `packages/*/src` (the same logic, while it lived under `examples/`, was outside the gate). + +## How the fixture works + +The fixture IS the persisted session log (`/session.jsonl`). Its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call's chunk sequence (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`", done by the snapshot harness — this plugin does not record. + +Two failure modes are not reconstructable from `assistant/chunk` alone — a pure throw before any chunk (e.g. an HTTP 401, where the log holds only a `turn/end {error}` and no chunks) and a cancel/hang (timing, not chunk content). A scenario that needs those supplies an optional sidecar (`/replay.override.json`: a `ReplayEntry[]`) that REPLACES the derived script. + +## Config + +| Key | Type | Default | Notes | +|---|---|---|---| +| `file` | string | `$DSH_SNAPSHOT_FILE` | Path to the per-scenario `session.jsonl` fixture. Required (config or env). | +| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | Optional path to a `ReplayEntry[]` sidecar that replaces the derived script. | + +```yaml +- id: llm-replay + name: '@deepseek-ai/dsh-llm-replay' + # file/overrideFile default to $DSH_SNAPSHOT_FILE / $DSH_SNAPSHOT_OVERRIDE, + # set by the snapshot harness per scenario. +``` + +## Exports + +- `installLlmReplay(ctx, config)` — install the `llm/stream` listener; returns the disposer (HMR safety). Use this in tests to drive replay without the Loader or env vars. +- `loadReplayScript(config)` — resolve the `ReplayEntry[]` for a scenario (sidecar override if present, else derived from the JSONL; fail-loud if the fixture is missing). +- `deriveReplayScript(events)` / `parseSessionLog(text)` — the pure helpers that turn a recorded session log into a script. A derived group must end in a `finish` chunk; a group without one is the fingerprint of a thrown `stream()` and must instead be expressed via an override sidecar. +- Types `ReplayEntry` / `ReplayConfig` / `Config`. + +## Plugin export shape + +Named `name` / `inject` / `Config` / `apply`, with **no default export**: the cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray default would collapse the module to the bare function and drop the `inject` namespace (see [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)). diff --git a/packages/support/llm-replay/package.json b/packages/support/llm-replay/package.json new file mode 100644 index 0000000000..ce57ea18ef --- /dev/null +++ b/packages/support/llm-replay/package.json @@ -0,0 +1,34 @@ +{ + "name": "@deepseek-ai/dsh-llm-replay", + "description": "Replay LLM plugin: short-circuits llm/stream with model chunks reconstructed from a recorded session JSONL (keyless snapshot tests)", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/support/llm-replay/src/index.ts b/packages/support/llm-replay/src/index.ts new file mode 100644 index 0000000000..67fcf09697 --- /dev/null +++ b/packages/support/llm-replay/src/index.ts @@ -0,0 +1,249 @@ +/** + * Replay LLM plugin for snapshot tests. + * + * Installs a single `llm/stream` waterfall listener that short-circuits the + * waterfall (never calls `next()`) and yields model streams reconstructed from + * a recorded **session JSONL** fixture — so a snapshot test can boot the real + * agent against a fixed model transcript with no API key. See + * docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md. + * + * The fixture IS the persisted session log (`/session.jsonl`): its + * `assistant/chunk` events carry every {@link StreamChunk}, so grouping them by + * `(turn, step)` reconstructs each `stream()` call's chunk sequence (one model + * call per loop step — see packages/core/agent-loop/src/loop.ts). Recording is + * therefore "run the real agent once and harvest the `.jsonl`", done by the + * snapshot harness — this plugin does not record. + * + * Two failure modes are NOT reconstructable from `assistant/chunk` alone — a + * pure throw before any chunk (e.g. an HTTP 401: the log holds only a + * `turn/end {error}`, no chunks) and a cancel/hang (timing, not chunk content). + * A scenario that needs those supplies an optional sidecar + * (`/replay.override.json`: a `ReplayEntry[]`) that REPLACES the + * derived script. + * + * It lives in its own package (not under `examples/`) so its derive/parse/ + * replay logic falls under the per-file 100% coverage gate on package `src` + * trees — its tests previously lived under `examples/`, which the gate does + * not measure, leaving these branches (clean chunks / mid-stream throw / hang) + * unguarded. Its consumer is the ACP snapshot harness in `examples/acp-agent`, + * which loads it (via `cordis.snapshot.yml`) in place of a real LLM adapter. + * + * Plugin export shape: named `name`/`inject`/`Config`/`apply`, NO default + * export (the cordis Loader's `unwrapExports` does `exports.default ?? exports`, + * so a stray default would drop the namespace — see docs/postmortem/0001). + * + * @module @deepseek-ai/dsh-llm-replay + */ + +import { existsSync, readFileSync } from 'node:fs' +import type { Context } from 'cordis' +import type { SessionEvent } from '@deepseek-ai/dsh-session' +import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' +import { LlmError, assertNever } from '@deepseek-ai/dsh-llm' + +/** + * One recorded model call. A discriminated union (not a bare `StreamChunk[]`) + * so it can faithfully replay BOTH branches of the documented LLM failure + * contract — an adapter may THROW from `stream()` or end with a `finish` error + * chunk — plus a `hang` marker for cancellation scenarios (mirrors the + * `MockAdapter` `hang` support in packages/core/agent-loop/tests). + * + * A `throw` entry carries any `chunks` the adapter emitted BEFORE it threw, so + * a mid-stream transport failure (partial output then `STREAM_CLOSED`) replays + * the partial chunks first and only then throws — exactly what the agent loop + * saw live (it may already have emitted partial assistant chunks). + * + * The normal/finish-terminated cases are DERIVED from the session JSONL + * ({@link deriveReplayScript}); only the throw and hang cases need a + * hand-authored sidecar entry (a thrown stream leaves no terminal `finish` in + * the log, so it cannot be derived as `chunks`). + */ +export type ReplayEntry = + | { kind: 'chunks'; chunks: StreamChunk[] } + | { kind: 'throw'; chunks: StreamChunk[]; message: string; code: string; status?: number } + | { kind: 'hang' } + +/** Resolved plugin configuration. */ +export interface ReplayConfig { + /** Path to the per-scenario `session.jsonl` fixture (the recorded log). */ + file: string + /** + * Optional path to a `ReplayEntry[]` sidecar that REPLACES the derived + * script. Used by the two scenarios not expressible as `assistant/chunk` + * (pure throw-before-chunk, cancel/hang). Absent for normal scenarios. + */ + overrideFile?: string +} + +/** + * Parse a session `.jsonl` buffer into its event list. Line 0 is the session + * header (a `{type:'session',…}` record), every subsequent non-empty line is a + * {@link SessionEvent}. The header is skipped; malformed lines fail loud. + */ +export function parseSessionLog(text: string): SessionEvent[] { + const lines = text.split('\n').filter(line => line.trim().length > 0) + const events: SessionEvent[] = [] + // Skip line 0 (the header). A reader distinguishes it by its `type:'session'` + // tag; we simply drop the first line, which the JSONL backend guarantees is + // the header. + for (let i = 1; i < lines.length; i++) { + const parsed: unknown = JSON.parse(lines[i] as string) + events.push(parsed as SessionEvent) + } + return events +} + +/** + * Reconstruct the per-`stream()` replay script from a recorded session log. + * + * The agent loop makes exactly one `ctx.llm.stream()` call per step and appends + * every chunk as an `assistant/chunk` event tagged with the current + * `(turn, step)`. Grouping those events by `(turn, step)` in log order + * therefore yields one `{kind:'chunks'}` entry per model call, in call order. + * + * A group is only valid if it ends in a `finish` chunk — the adapter contract + * guarantees a successful (or finish-error) stream terminates with `finish`, + * and the loop relies on it. A group WITHOUT a terminal `finish` is the + * fingerprint of a *thrown* `stream()` (the loop recorded the prefix chunks, + * then an `error`/`turn/end`, but no `finish`): such a stream cannot be + * faithfully replayed as `{kind:'chunks'}` (that would look like a clean stop), + * so deriving it is an error — the scenario must supply a `replay.override.json` + * sidecar with an explicit `throw` (or `hang`) entry instead. {@link + * deriveReplayScript} throws, naming the offending `(turn, step)`, so a missing + * override fails loud rather than silently replaying a thrown call as success. + */ +export function deriveReplayScript(events: SessionEvent[]): ReplayEntry[] { + const script: ReplayEntry[] = [] + let currentKey: string | undefined + let current: StreamChunk[] = [] + const close = (key: string | undefined, chunks: StreamChunk[]): void => { + if (chunks.length === 0) return + if (chunks[chunks.length - 1]?.type !== 'finish') { + throw new Error( + `llm-replay: model call ${key} ended without a finish chunk (a thrown stream); ` + + 'this scenario needs a replay.override.json sidecar', + ) + } + script.push({ kind: 'chunks', chunks }) + } + for (const event of events) { + if (event.type !== 'assistant/chunk') continue + const { turn, step, chunk } = event.data + const key = `${turn}/${step}` + if (key !== currentKey) { + // A new (turn, step) — i.e. a new stream() call. Close the previous one + // (skip the initial empty buffer before any chunk has been seen). + close(currentKey, current) + currentKey = key + current = [] + } + current.push(chunk) + } + close(currentKey, current) + return script +} + +/** + * Build the replay script for a scenario: the sidecar override if present, + * otherwise the script derived from the recorded session JSONL. Fail-loud if + * the JSONL fixture is missing (the scenario was never recorded) — never + * silently returns an empty script, so a coverage hole can't masquerade as a + * passing replay. + */ +export function loadReplayScript(config: ReplayConfig): ReplayEntry[] { + if (config.overrideFile !== undefined && existsSync(config.overrideFile)) { + const parsed: unknown = JSON.parse(readFileSync(config.overrideFile, 'utf8')) + if (!Array.isArray(parsed)) { + throw new Error(`llm-replay: override is not a JSON array: ${config.overrideFile}`) + } + return parsed as ReplayEntry[] + } + if (!existsSync(config.file)) { + throw new Error(`llm-replay: fixture not found: ${config.file} — run \`pnpm run test:snapshot:record\` first`) + } + return deriveReplayScript(parseSessionLog(readFileSync(config.file, 'utf8'))) +} + +/** Yield a recorded stream back, honoring abort like a real adapter. */ +async function* replayEntry(entry: ReplayEntry, signal: AbortSignal | undefined): AsyncIterable { + switch (entry.kind) { + case 'chunks': + for (const chunk of entry.chunks) { + if (signal?.aborted) throw new Error('aborted') + yield chunk + } + return + case 'throw': + // Replay the THROW branch of the LLM contract: emit whatever the adapter + // streamed before it threw (so the loop sees the same partial output it + // saw live), then throw the recorded error (e.g. a provider 401, or a + // mid-stream STREAM_CLOSED after partial chunks). + for (const chunk of entry.chunks) { + if (signal?.aborted) throw new Error('aborted') + yield chunk + } + throw new LlmError(entry.message, entry.code, entry.status) + case 'hang': + // Replay a stream that stalls until cancelled (mirrors MockAdapter): one + // chunk, then wait for abort and surface it as the consumer expects. + yield { type: 'block-start', index: 0, blockType: 'text' } + yield { type: 'text-delta', index: 0, text: 'partial' } + await new Promise((_resolve, reject) => { + if (signal?.aborted) { reject(new Error('aborted')); return } + signal?.addEventListener('abort', () => { reject(new Error('aborted')) }, { once: true }) + }) + /* v8 ignore next -- unreachable: the hang promise only ever rejects (on abort), never resolves; control never reaches here */ + return + default: + // Closed local union: an unknown kind means malformed (hand-edited or + // drifted) sidecar data — fail loud with a runtime diagnostic. + return assertNever(entry, 'llm-replay replay entry') + } +} + +/** + * Install the replay `llm/stream` listener on `ctx`. Returns the listener + * disposer (so a fiber dispose removes it — HMR safety). Exported separately + * from {@link apply} so unit tests can drive it without the Loader or env vars. + * + * Replay is POSITIONAL: the Nth `stream()` call serves the Nth script entry. + * This is deterministic only with at most one model stream in flight at a time; + * the snapshot harness runs one ACP session per scenario to guarantee that. The + * cursor is advanced synchronously at listener-invocation time (not lazily + * inside the generator) so call ORDER, not iteration order, fixes the mapping. + */ +export function installLlmReplay(ctx: Context, config: ReplayConfig): () => void { + const entries = loadReplayScript(config) + let cursor = 0 + return ctx.on('llm/stream', (options: GenerateOptions, _next) => { + const index = cursor++ + const entry: ReplayEntry | undefined = entries[index] + return (async function* () { + if (entry === undefined) { + throw new Error( + `llm-replay: script exhausted — requested model call #${index + 1} but the fixture has only ${entries.length}; re-record the scenario`, + ) + } + yield* replayEntry(entry, options.signal) + })() + }) +} + +export const name = 'llm-replay' +export const inject = ['llm'] + +export interface Config { + /** Override the fixture path; defaults to `$DSH_SNAPSHOT_FILE`. */ + file?: string + /** Override the sidecar path; defaults to `$DSH_SNAPSHOT_OVERRIDE`. */ + overrideFile?: string +} + +export function apply(ctx: Context, config: Config = {}): void { + const file = config.file ?? process.env.DSH_SNAPSHOT_FILE + if (file === undefined || file.length === 0) { + throw new Error('llm-replay: a fixture path is required (Config.file or $DSH_SNAPSHOT_FILE)') + } + const overrideFile = config.overrideFile ?? process.env.DSH_SNAPSHOT_OVERRIDE + installLlmReplay(ctx, overrideFile === undefined || overrideFile.length === 0 ? { file } : { file, overrideFile }) +} diff --git a/packages/support/llm-replay/tests/llm-replay.spec.ts b/packages/support/llm-replay/tests/llm-replay.spec.ts new file mode 100644 index 0000000000..925881273a --- /dev/null +++ b/packages/support/llm-replay/tests/llm-replay.spec.ts @@ -0,0 +1,421 @@ +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import type { SessionEvent } from '@deepseek-ai/dsh-session' +import LlmService, { GenerateOptions, LlmAdapter, StreamChunk } from '@deepseek-ai/dsh-llm' +import { + type ReplayEntry, + apply, + deriveReplayScript, + inject, + installLlmReplay, + loadReplayScript, + name, + parseSessionLog, +} from '../src/index.ts' + +/** + * Unit tests for the replay llm/stream plugin. These drive the listener through + * the REAL LlmService waterfall (not a hand-rolled stub) so they verify the + * actual seam the snapshot harness depends on, plus the pure + * derive/parse/load helpers that turn a recorded session JSONL into a script. + */ + +const TEXT_CHUNKS: StreamChunk[] = [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'text-delta', index: 0, text: 'hi' }, + { type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } }, + { type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } }, + { type: 'finish', reason: { kind: 'stop' } }, +] + +/** Build a minimal session-JSONL string: a header line + the given events. */ +function sessionJsonl(events: SessionEvent[]): string { + const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) + return [header, ...events.map(e => JSON.stringify(e))].join('\n') + '\n' +} + +/** A SessionEvent of type assistant/chunk for (turn, step). */ +function chunkEvent(seq: number, turn: number, step: number, chunk: StreamChunk): SessionEvent { + return { type: 'assistant/chunk', seq, time: 0, data: { turn, step, chunk } } +} + +let dir: string +let file: string + +beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), 'llm-replay-spec-')) + file = join(dir, 'session.jsonl') +}) + +afterEach(() => { + rmSync(dir, { recursive: true, force: true }) +}) + +async function drain(iter: AsyncIterable): Promise { + const out: StreamChunk[] = [] + for await (const chunk of iter) out.push(chunk) + return out +} + +describe('parseSessionLog', () => { + it('skips the header line and parses each event', () => { + const events = [chunkEvent(1, 1, 1, TEXT_CHUNKS[0] as StreamChunk)] + expect(parseSessionLog(sessionJsonl(events))).toEqual(events) + }) + + it('ignores blank lines', () => { + const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) + const ev = chunkEvent(1, 1, 1, TEXT_CHUNKS[0] as StreamChunk) + expect(parseSessionLog(`${header}\n\n${JSON.stringify(ev)}\n\n`)).toEqual([ev]) + }) +}) + +describe('deriveReplayScript', () => { + it('groups assistant/chunk by (turn, step) into one entry per stream() call', () => { + const events: SessionEvent[] = TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c)) + expect(deriveReplayScript(events)).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) + }) + + it('produces one entry per distinct (turn, step), in log order', () => { + const callA = TEXT_CHUNKS + const callB: StreamChunk[] = [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'text-delta', index: 0, text: 'two' }, + { type: 'finish', reason: { kind: 'stop' } }, + ] + let seq = 1 + const events: SessionEvent[] = [ + ...callA.map(c => chunkEvent(seq++, 1, 1, c)), + ...callB.map(c => chunkEvent(seq++, 1, 2, c)), // same turn, next step + ] + expect(deriveReplayScript(events)).toEqual([ + { kind: 'chunks', chunks: callA }, + { kind: 'chunks', chunks: callB }, + ]) + }) + + it('separates calls across turns too', () => { + let seq = 1 + const events: SessionEvent[] = [ + ...TEXT_CHUNKS.map(c => chunkEvent(seq++, 1, 1, c)), + ...TEXT_CHUNKS.map(c => chunkEvent(seq++, 2, 1, c)), // new turn, step resets to 1 + ] + expect(deriveReplayScript(events)).toHaveLength(2) + }) + + it('ignores non-assistant/chunk events', () => { + let seq = 1 + const events: SessionEvent[] = [ + { type: 'turn/start', seq: seq++, time: 0, data: { turn: 1, trigger: { kind: 'continuation' } } }, + ...TEXT_CHUNKS.map(c => chunkEvent(seq++, 1, 1, c)), + { type: 'turn/end', seq: seq++, time: 0, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + expect(deriveReplayScript(events)).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) + }) + + it('returns an empty script for a log with no assistant/chunk events', () => { + expect(deriveReplayScript([])).toEqual([]) + }) + + it('keeps a finish-error chunk in the derived entry (replays naturally)', () => { + const errChunks: StreamChunk[] = [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'finish', reason: { kind: 'error', message: 'boom', code: 'X' } }, + ] + const events = errChunks.map((c, i) => chunkEvent(i + 1, 1, 1, c)) + expect(deriveReplayScript(events)).toEqual([{ kind: 'chunks', chunks: errChunks }]) + }) + + it('throws on a group that lacks a terminal finish chunk (a thrown stream)', () => { + // A thrown stream(): prefix chunks logged, then turn/end (error reason), NO finish. + const events: SessionEvent[] = [ + chunkEvent(1, 1, 1, { type: 'block-start', index: 0, blockType: 'text' }), + chunkEvent(2, 1, 1, { type: 'text-delta', index: 0, text: 'par' }), + { type: 'turn/end', seq: 3, time: 0, data: { turn: 1, reason: { kind: 'error', step: 1, message: 'x' } } }, + ] + expect(() => deriveReplayScript(events)).toThrow(/without a finish chunk.*replay\.override\.json/s) + }) + + it('names the offending (turn, step) when a group is incomplete', () => { + const events: SessionEvent[] = [ + chunkEvent(1, 2, 3, { type: 'block-start', index: 0, blockType: 'text' }), + ] + expect(() => deriveReplayScript(events)).toThrow(/2\/3/) + }) +}) + +describe('loadReplayScript', () => { + it('derives from the session JSONL when no override is present', () => { + writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8') + expect(loadReplayScript({ file })).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) + }) + + it('uses the sidecar override when present, ignoring the JSONL', () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + const override: ReplayEntry[] = [{ kind: 'throw', chunks: [], message: '401', code: 'AUTH', status: 401 }] + writeFileSync(overrideFile, JSON.stringify(override), 'utf8') + expect(loadReplayScript({ file, overrideFile })).toEqual(override) + }) + + it('falls back to the JSONL when the override path is set but absent', () => { + writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8') + expect(loadReplayScript({ file, overrideFile: join(dir, 'nope.json') })) + .toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) + }) + + it('fails loud when the fixture is missing', () => { + expect(() => loadReplayScript({ file: join(dir, 'absent.jsonl') })).toThrow(/fixture not found/) + }) + + it('throws when the override is not a JSON array', () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, '{"not":"array"}', 'utf8') + expect(() => loadReplayScript({ file, overrideFile })).toThrow(/not a JSON array/) + }) +}) + +describe('installLlmReplay (through the real waterfall)', () => { + function writeLog(...calls: StreamChunk[][]): void { + let seq = 1 + const events: SessionEvent[] = [] + calls.forEach((chunks, step) => { + for (const c of chunks) events.push(chunkEvent(seq++, 1, step + 1, c)) + }) + writeFileSync(file, sessionJsonl(events), 'utf8') + } + + it('serves derived chunks back, short-circuiting the adapter', async () => { + writeLog(TEXT_CHUNKS) + const ctx = new Context() + await ctx.plugin(LlmService) + // No adapter registered for 'm' — replay must not reach it. + installLlmReplay(ctx, { file }) + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS) + }) + + it('serves the Nth call the Nth derived entry (positional)', async () => { + const second: StreamChunk[] = [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'text-delta', index: 0, text: 'two' }, + { type: 'finish', reason: { kind: 'stop' } }, + ] + writeLog(TEXT_CHUNKS, second) + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file }) + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS) + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(second) + }) + + it('replays a sidecar throw-entry as an LlmError with code/status, after its prefix chunks', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + const partial: StreamChunk[] = [{ type: 'block-start', index: 0, blockType: 'text' }] + writeFileSync(overrideFile, JSON.stringify([ + { kind: 'throw', chunks: partial, message: 'unauthorized', code: 'AUTH', status: 401 }, + ]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file, overrideFile }) + + const seen: StreamChunk[] = [] + await expect((async () => { + for await (const c of ctx.llm.stream({ model: 'm', messages: [] })) seen.push(c) + })()).rejects.toMatchObject({ message: 'unauthorized', code: 'AUTH', status: 401 }) + expect(seen).toEqual(partial) + }) + + it('replays a sidecar hang-entry that surfaces abort when the signal fires', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file, overrideFile }) + + const controller = new AbortController() + const iterator = ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal })[Symbol.asyncIterator]() + // Deterministically consume the two pre-hang chunks (no sleep), then abort + // and assert the next pull rejects — event-driven, per the no-sleeps rule. + expect((await iterator.next()).value).toMatchObject({ type: 'block-start' }) + expect((await iterator.next()).value).toMatchObject({ type: 'text-delta' }) + controller.abort() + await expect(iterator.next()).rejects.toThrow('aborted') + }) + + it('fails loud when the script is exhausted', async () => { + writeLog(TEXT_CHUNKS) + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file }) + await drain(ctx.llm.stream({ model: 'm', messages: [] })) + await expect(drain(ctx.llm.stream({ model: 'm', messages: [] }))).rejects.toThrow(/exhausted/) + }) + + it('aborts mid-replay when the signal is already set', async () => { + writeLog(TEXT_CHUNKS) + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file }) + const controller = new AbortController() + controller.abort() + await expect(drain(ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal }))) + .rejects.toThrow('aborted') + }) + + it('removes the waterfall listener when the owning fiber is disposed (HMR safety)', async () => { + writeLog(TEXT_CHUNKS, TEXT_CHUNKS) + const ctx = new Context() + await ctx.plugin(LlmService) + + // A real adapter to fall through to AFTER dispose, proving the listener is gone. + class FallthroughAdapter extends LlmAdapter { + async * stream(_options: GenerateOptions): AsyncIterable { + yield { type: 'finish', reason: { kind: 'stop' } } + } + } + ctx.llm.registerAdapter(['m'], new FallthroughAdapter()) + + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + installLlmReplay(inner, { file }) + }, { inject: ['llm'] })) + + // While installed, replay short-circuits to the derived fixture ('hi'). + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS) + + await fiber.dispose() + // After dispose the listener is gone; the call reaches the real adapter. + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))) + .toEqual([{ type: 'finish', reason: { kind: 'stop' } }]) + }) + + it('throws on a malformed sidecar entry kind (the assertNever guard)', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + // A kind the union does not know — hand-edited/drifted sidecar data. + writeFileSync(overrideFile, JSON.stringify([{ kind: 'bogus' }]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file, overrideFile }) + await expect(drain(ctx.llm.stream({ model: 'm', messages: [] }))) + .rejects.toThrow(/llm-replay replay entry/) + }) + + it('rejects a hang entry when the signal fires DURING the wait (abort listener path)', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file, overrideFile }) + const controller = new AbortController() + const iterator = ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal })[Symbol.asyncIterator]() + // Consume the two pre-hang chunks, then start the third pull so the generator + // is parked inside the await (signal NOT yet aborted — exercises the + // addEventListener('abort') registration), and only THEN abort. + expect((await iterator.next()).value).toMatchObject({ type: 'block-start' }) + expect((await iterator.next()).value).toMatchObject({ type: 'text-delta' }) + const pending = iterator.next() + await new Promise(r => setImmediate(r)) + controller.abort() + await expect(pending).rejects.toThrow('aborted') + }) + + it('aborts mid-replay of a throw-entry prefix when the signal is set', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + const partial: StreamChunk[] = [{ type: 'block-start', index: 0, blockType: 'text' }] + writeFileSync(overrideFile, JSON.stringify([ + { kind: 'throw', chunks: partial, message: 'unauthorized', code: 'AUTH', status: 401 }, + ]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file, overrideFile }) + const controller = new AbortController() + controller.abort() + // Already aborted: the throw-entry's prefix loop surfaces 'aborted' before + // it can reach the recorded LlmError. + await expect(drain(ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal }))) + .rejects.toThrow('aborted') + }) + + it('surfaces an already-aborted signal on a hang entry before waiting', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + installLlmReplay(ctx, { file, overrideFile }) + const controller = new AbortController() + controller.abort() + // The two pre-hang chunks still flow; the abort surfaces at the await. + const iterator = ctx.llm.stream({ model: 'm', messages: [], signal: controller.signal })[Symbol.asyncIterator]() + await iterator.next() + await iterator.next() + await expect(iterator.next()).rejects.toThrow('aborted') + }) +}) + +describe('apply (the plugin entry)', () => { + const ORIG = { file: process.env.DSH_SNAPSHOT_FILE, override: process.env.DSH_SNAPSHOT_OVERRIDE } + afterEach(() => { + if (ORIG.file === undefined) delete process.env.DSH_SNAPSHOT_FILE + else process.env.DSH_SNAPSHOT_FILE = ORIG.file + if (ORIG.override === undefined) delete process.env.DSH_SNAPSHOT_OVERRIDE + else process.env.DSH_SNAPSHOT_OVERRIDE = ORIG.override + }) + + it('exposes the namespace plugin shape (name/inject, no default export)', () => { + expect(name).toBe('llm-replay') + expect(inject).toEqual(['llm']) + }) + + it('installs replay from an explicit config.file', async () => { + writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8') + const ctx = new Context() + await ctx.plugin(LlmService) + apply(ctx, { file }) + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS) + }) + + it('falls back to $DSH_SNAPSHOT_FILE / $DSH_SNAPSHOT_OVERRIDE when config is empty', async () => { + writeFileSync(file, sessionJsonl([]), 'utf8') + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([{ kind: 'chunks', chunks: TEXT_CHUNKS }]), 'utf8') + process.env.DSH_SNAPSHOT_FILE = file + process.env.DSH_SNAPSHOT_OVERRIDE = overrideFile + const ctx = new Context() + await ctx.plugin(LlmService) + apply(ctx) + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS) + }) + + it('uses only the file when no override path is configured or in the env', async () => { + writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8') + process.env.DSH_SNAPSHOT_FILE = file + delete process.env.DSH_SNAPSHOT_OVERRIDE + const ctx = new Context() + await ctx.plugin(LlmService) + apply(ctx) + expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS) + }) + + it('throws when no fixture path is given by config or env', async () => { + delete process.env.DSH_SNAPSHOT_FILE + const ctx = new Context() + await ctx.plugin(LlmService) + expect(() => { apply(ctx, {}) }).toThrow(/a fixture path is required/) + }) + + it('treats an empty-string fixture path as missing', async () => { + delete process.env.DSH_SNAPSHOT_FILE + const ctx = new Context() + await ctx.plugin(LlmService) + expect(() => { apply(ctx, { file: '' }) }).toThrow(/a fixture path is required/) + }) +}) diff --git a/packages/support/llm-replay/tsconfig.json b/packages/support/llm-replay/tsconfig.json new file mode 100644 index 0000000000..95245937ec --- /dev/null +++ b/packages/support/llm-replay/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + } + ] +} diff --git a/packages/support/ui-stdio/README.md b/packages/support/ui-stdio/README.md new file mode 100644 index 0000000000..b65fd4d8e1 --- /dev/null +++ b/packages/support/ui-stdio/README.md @@ -0,0 +1,44 @@ +# @deepseek-ai/dsh-ui-stdio + +A minimal stdio (readline) UI, as a plugin. It reads lines from stdin and feeds them to an agent (`send` when idle, `steer` while a turn is running), and renders that agent's streamed output and tool activity to stdout. A UI is "just a plugin" here — it only consumes the `agent/*` event taxonomy plus the `agents` service (`inject: ['agents']`), so the same plugin drives any example or product surface. + +This package consolidates what were two near-identical copies under `examples/echo-agent` and `examples/coding-agent`. The coding copy was a superset; this package IS that superset — dimmed chain-of-thought rendering plus robust piped-stdin EOF handling — with the per-consumer differences moved into `Config`. + +## Config + +| Key | Type | Default | Notes | +|---|---|---|---| +| `welcome` | string | `'ready.'` | Banner printed once on start, before the first `> ` prompt. | +| `agent` | string | `'main'` | Id of the agent that stdin **drives** (`send`/`steer`) and whose `agent/status` gates the EOF exit. Rendering is **not** scoped by it — see below. | + +```yaml +- id: ui-stdio + name: '@deepseek-ai/dsh-ui-stdio' + config: + welcome: 'coding-agent ready. Give it a coding task.' +``` + +## Rendering + +Rendering is **global** — every agent's events are written to stdout, not just `config.agent`'s. `config.agent` scopes only *input* (which agent stdin drives) and the EOF-exit gate; the single-agent demos this serves have just one agent, so the distinction is moot for them. (A multi-agent UI that needs per-agent panes would filter these handlers by the agent argument — deliberately out of scope here.) + +- `agent/stream-chunk` — `text-delta` is written verbatim; `reasoning-delta` is wrapped in the dim SGR (`\x1B[2m … \x1B[0m`) so the chain-of-thought is visually subordinate to the answer. Reasoning rendering is inert when no `reasoning-delta` chunks arrive (e.g. a mock model), so it is always on. +- `agent/turn-start` / `agent/turn-end` — a `[ turn N]` header and a trailing `> ` prompt. +- `session/event` — `tool/call` renders `[tool call] name(args)`; `tool/result` renders the joined text blocks as `[tool result] …`. + +## The I/O seam + +The production entry point `apply(ctx, config)` binds the real `process` streams. The testable core is `createStdioChat(ctx, config, runtime)`, where `runtime: StdioRuntime` supplies `input` / `output` / `exit`. This seam is deliberately **not** part of the serializable `Config` (streams and functions do not belong in YAML config); it exists so the render, EOF, and disposal branches can be exercised with fakes instead of hijacking globals. + +## Piped-stdin exit + +On stdin EOF the plugin exits the process, but carefully: + +- **No work submitted** (empty stdin, blank-only lines): exit immediately — no turn will ever start, so there is nothing to wait for. Gating on an observed `running` here would hang forever. +- **Work submitted**: exit the next time the agent settles to `idle` *after* having been observed `running`. `agent.send()` does not synchronously flip status to `running`, so requiring an observed `running` first (`sawRunning`) avoids exiting in the gap before the turn starts and dropping work; and the loop batches several queued messages into one turn, so the exit keys off the idle transition rather than counting sends. + +Disposal (HMR or fiber teardown) closes the readline interface, which also fires `close` — a `disposed` guard ensures teardown never calls `process.exit`. + +## Plugin export shape + +Named `name` / `inject` / `Config` / `apply`, with **no default export**: the cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray default would collapse the module to the bare function and drop the `inject` namespace (see [docs/postmortem/0001](../../../docs/postmortem/0001-acp-default-export-drops-inject.md)). The keyless Loader-path e2e smokes in `examples/{echo,coding}-agent` guard this end-to-end. diff --git a/packages/support/ui-stdio/package.json b/packages/support/ui-stdio/package.json new file mode 100644 index 0000000000..5d65356e79 --- /dev/null +++ b/packages/support/ui-stdio/package.json @@ -0,0 +1,39 @@ +{ + "name": "@deepseek-ai/dsh-ui-stdio", + "description": "Minimal stdio (readline) UI plugin: renders agent/* events to stdout and feeds stdin lines to the agent", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/support/ui-stdio/src/index.ts b/packages/support/ui-stdio/src/index.ts new file mode 100644 index 0000000000..070e842094 --- /dev/null +++ b/packages/support/ui-stdio/src/index.ts @@ -0,0 +1,205 @@ +/** + * Minimal stdio UI plugin: reads lines from stdin → `agent.send()`/`steer()`, + * and renders the agent's stream chunks and tool activity to stdout. A UI is + * "just a plugin" — it only consumes the `agent/*` event taxonomy and the + * `agents` service, so the same plugin drives any example or product surface. + * + * Consolidates what were two near-identical copies under `examples/echo-agent` + * and `examples/coding-agent` (the latter a superset). This package IS that + * superset: dimmed chain-of-thought rendering plus the robust piped-stdin + * EOF→idle exit handling, configured per consumer via {@link Config}. + * + * Plugin export shape: named `name`/`inject`/`Config`/`apply`, NO default + * export — the cordis Loader's `unwrapExports` does `exports.default ?? exports`, + * so a stray default would collapse the module to the bare function and drop + * the `inject` namespace (see docs/postmortem/0001). The keyless Loader-path + * e2e smokes in `examples/{echo,coding}-agent` guard this end-to-end. + * + * @module @deepseek-ai/dsh-ui-stdio + */ + +import { createInterface } from 'node:readline' +import type { Readable, Writable } from 'node:stream' +import type { Context } from 'cordis' +import z from 'schemastery' +import { AgentId } from '@deepseek-ai/dsh-agent' + +export const name = 'ui-stdio' +export const inject = ['agents'] + +/** Serializable plugin configuration (cordis-native, schemastery). */ +export interface Config { + /** Banner printed once on start, before the first `> ` prompt. */ + welcome?: string + /** Id of the agent stdin drives (`send`/`steer`) and whose status gates the EOF exit; rendering is global. Defaults to `'main'`. */ + agent?: string +} + +export const Config: z = z.object({ + welcome: z.string().default('ready.'), + agent: z.string().default('main'), +}) + +/** + * Process-I/O seam — the side-effecting handles the plugin would otherwise + * reach for as globals. Defaulted to the real `process` streams in + * {@link apply}; injected by tests so the EOF, render, and disposal branches + * are exercised without hijacking globals. Deliberately NOT part of the + * serializable {@link Config} (streams/functions don't belong in YAML config). + */ +export interface StdioRuntime { + /** Line source (default `process.stdin`). */ + input: Readable + /** Render sink (default `process.stdout`). */ + output: Writable + /** Process-exit hook (default `process.exit`); called once on stdin EOF. */ + exit: (code: number) => void +} + +function isTTYPair(input: Readable, output: Writable): boolean { + return Boolean((input as { isTTY?: boolean }).isTTY && (output as { isTTY?: boolean }).isTTY) +} + +/** + * The plugin body, parameterized over its I/O runtime. `apply` is the thin + * production wrapper that binds the real `process` streams; tests call this + * directly with fakes. Returns nothing — all registration is via `ctx.on`/ + * `ctx.effect`, so fiber disposal tears every listener and the readline + * interface down. + */ +export function createStdioChat(ctx: Context, config: Config, runtime: StdioRuntime): void { + // Default here too (not just via schemastery's `.default()`): this helper is + // exported and called directly by tests / programmatic consumers that bypass + // Loader validation, so it must be self-contained rather than trusting the + // cast — `config.welcome as string` would otherwise be `undefined` on `{}`. + const welcome = config.welcome ?? 'ready.' + const agentId = AgentId(config.agent ?? 'main') + const { input, output, exit } = runtime + + let inReasoning = false + ctx.on('agent/stream-chunk', (_agent, _turn, _step, chunk) => { + if (chunk.type === 'reasoning-delta') { + // Dim the chain-of-thought so the final answer stands out. + if (!inReasoning) output.write('\x1B[2m') + inReasoning = true + output.write(chunk.text) + } else if (chunk.type === 'text-delta') { + if (inReasoning) output.write('\x1B[0m\n') + inReasoning = false + output.write(chunk.text) + } + }) + + ctx.on('agent/turn-start', (agent, turn) => { + output.write(`\n[${agent.id} turn ${turn}] `) + }) + + ctx.on('agent/turn-end', () => { + if (inReasoning) output.write('\x1B[0m') + inReasoning = false + output.write('\n> ') + }) + + ctx.on('session/event', (_session, event) => { + if (event.type === 'tool/call') { + const { name: toolName, arguments: args } = event.data + if (inReasoning) output.write('\x1B[0m') + inReasoning = false + output.write(`\n [tool call] ${toolName}(${args})`) + } else if (event.type === 'tool/result') { + const { content } = event.data + const text = content.filter(block => block.type === 'text').map(block => block.text).join('') + output.write(`\n [tool result] ${text}\n `) + } + }) + + ctx.effect(() => { + const reader = createInterface({ input, output, terminal: isTTYPair(input, output) }) + // Piped-input exit, once stdin reaches EOF: + // - If no line ever submitted work (empty stdin, blank-only lines), exit + // immediately — no turn will ever start, so there is nothing to wait + // for. (Gating on an observed 'running' here would hang forever.) + // - If work WAS submitted, exit the next time the agent settles to idle + // AFTER having run. Two subtleties this handles: the loop batches + // several queued messages into ONE turn (one idle), so we don't count + // sends; and agent.send() does NOT synchronously flip status to + // 'running', so requiring an observed 'running' first (`sawRunning`) + // avoids exiting in the gap before the turn starts and dropping work. + let stdinClosed = false + let disposed = false + let submittedWork = false + let sawRunning = false + let exitTimer: ReturnType | undefined + + const maybeExit = (): void => { + if (disposed || !stdinClosed) return + // No work submitted: nothing will ever run, exit straight away. + // Work submitted: wait until a turn has run and the agent is idle. + if (submittedWork) { + if (!sawRunning) return + const agent = ctx.agents.get(agentId) + if (agent && agent.status !== 'idle') return // a turn is still running + } + // Let any final output flush, then exit. The handle is tracked so the + // disposer can cancel it — a dispose within the flush window must not let + // the process exit out from under HMR. Re-entrant `maybeExit` calls (e.g. + // repeated idle signals) coalesce onto the one pending timer. + if (exitTimer !== undefined) { + return // exit already scheduled — coalesce re-entrant calls + } + exitTimer = setTimeout(() => { exit(0) }, 200) + } + + const disposeStatusListener = ctx.on('agent/status', (subject, status) => { + if (subject.id !== agentId) return + if (status === 'running') sawRunning = true + if (status === 'idle') maybeExit() + }) + + reader.on('line', (line) => { + const text = line.trim() + if (!text) return + const agent = ctx.agents.get(agentId) + if (!agent) { + ctx.logger.error('ui-stdio: agent "%s" is not running', agentId) + return + } + submittedWork = true + if (agent.status === 'running') { + agent.steer([{ type: 'text', text }]) + } else { + agent.send([{ type: 'text', text }]) + } + }) + reader.on('close', () => { + // Fires for BOTH stdin EOF and plugin disposal (reader.close() below); + // `disposed` guards teardown so HMR/dispose never exits the process. + stdinClosed = true + maybeExit() + }) + output.write(`${welcome}\n> `) + return () => { + disposed = true + if (exitTimer !== undefined) clearTimeout(exitTimer) + disposeStatusListener() + reader.close() + } + }, 'ui-stdio') +} + +/** + * Cordis entry point. Binds the real `process` streams and delegates to + * {@link createStdioChat}; the indirection keeps the side-effecting handles out + * of the testable core, which is why the unit suite drives `createStdioChat` + * directly. This thin wrapper is exercised end-to-end by the keyless + * Loader-path e2e smoke in `examples/echo-agent` (the real product entry). + */ +/* v8 ignore start -- production stdio wiring; testable core is createStdioChat() (covered), exercised e2e by echo-agent keyless smoke */ +export function apply(ctx: Context, config: Config): void { + createStdioChat(ctx, config, { + input: process.stdin, + output: process.stdout, + exit: code => process.exit(code), + }) +} +/* v8 ignore stop */ diff --git a/packages/support/ui-stdio/tests/readline.spec.ts b/packages/support/ui-stdio/tests/readline.spec.ts new file mode 100644 index 0000000000..c8b147ddab --- /dev/null +++ b/packages/support/ui-stdio/tests/readline.spec.ts @@ -0,0 +1,50 @@ +import { EventEmitter } from 'node:events' +import type { Readable, Writable } from 'node:stream' +import { describe, expect, it, vi } from 'vitest' +import type { Context } from 'cordis' +import type { StdioRuntime } from '../src/index.ts' + +const createInterface = vi.hoisted(() => vi.fn(() => { + const reader = new EventEmitter() as EventEmitter & { close(): void } + reader.close = vi.fn() + return reader +})) + +vi.mock('node:readline', () => ({ createInterface })) + +function fakeContext(): Context { + return { + on: vi.fn(() => vi.fn()), + effect: vi.fn((callback: () => () => void) => callback()), + } as unknown as Context +} + +function fakeRuntime(inputIsTTY: boolean, outputIsTTY: boolean): StdioRuntime { + return { + input: { isTTY: inputIsTTY } as Readable & { isTTY: boolean }, + output: { isTTY: outputIsTTY, write: vi.fn(() => true) } as unknown as Writable & { isTTY: boolean }, + exit: vi.fn(), + } +} + +describe('createStdioChat readline mode', () => { + it('enables terminal editing only when both stdio streams are TTYs', async () => { + const { createStdioChat } = await import('../src/index.ts') + + const tty = fakeRuntime(true, true) + createStdioChat(fakeContext(), {}, tty) + expect(createInterface).toHaveBeenLastCalledWith({ + input: tty.input, + output: tty.output, + terminal: true, + }) + + const piped = fakeRuntime(true, false) + createStdioChat(fakeContext(), {}, piped) + expect(createInterface).toHaveBeenLastCalledWith({ + input: piped.input, + output: piped.output, + terminal: false, + }) + }) +}) diff --git a/packages/support/ui-stdio/tests/ui-stdio.spec.ts b/packages/support/ui-stdio/tests/ui-stdio.spec.ts new file mode 100644 index 0000000000..bd5e0f7f91 --- /dev/null +++ b/packages/support/ui-stdio/tests/ui-stdio.spec.ts @@ -0,0 +1,347 @@ +import { Readable } from 'node:stream' +import { describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' +import { createStdioChat, type Config, type StdioRuntime } from '../src/index.ts' + +/** + * Unit tests for the stdio UI plugin. They drive the REAL plugin body + * (`createStdioChat`) with an injected {@link StdioRuntime} so every render, + * input, EOF, and disposal branch runs without touching the real `process` + * streams — the I/O seam is what makes the per-file gate reachable. The + * `agents` service is real (`@deepseek-ai/dsh-agent`); a minimal fake `Agent` + * stands in for the loop, since the loop is the genuinely expensive collaborator + * and we only need its `status` + `send`/`steer` surface here. + */ + +/** A controllable stdin: a Readable we push lines into and can end on demand. */ +function makeInput(): Readable & { feed(line: string): void; finish(): void } { + const stream = new Readable({ read() {} }) as Readable & { feed(line: string): void; finish(): void } + stream.feed = (line: string) => stream.push(`${line}\n`) + stream.finish = () => stream.push(null) + return stream +} + +/** A stdout sink that accumulates everything written, for assertions. */ +function makeOutput(): { write: (s: string) => boolean; text: () => string } { + let buf = '' + return { write: (s: string) => { buf += s; return true }, text: () => buf } +} + +function makeRuntime(over: Partial = {}): { + runtime: StdioRuntime + input: ReturnType + out: ReturnType + exit: ReturnType +} { + const input = makeInput() + const out = makeOutput() + const exit = vi.fn() + return { runtime: { input, output: { write: out.write } as never, exit, ...over }, input, out, exit } +} + +/** A minimal Agent fake exposing the surface the UI touches. */ +function makeAgent(id: string, status: AgentStatus = 'idle'): Agent & { + status: AgentStatus + sent: ContentBlock[][] + steered: ContentBlock[][] +} { + const sent: ContentBlock[][] = [] + const steered: ContentBlock[][] = [] + return { + id: id as Agent['id'], + status, + sent, + steered, + send: (content: ContentBlock[]) => void sent.push(content), + steer: (content: ContentBlock[]) => void steered.push(content), + } as never +} + +const CONFIG: Config = { welcome: 'hi there', agent: 'main' } + +async function setup(config: Config = CONFIG, runtimeOver: Partial = {}) { + const ctx = new Context() + await ctx.plugin(AgentRegistry) + const { runtime, input, out, exit } = makeRuntime(runtimeOver) + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + createStdioChat(inner, config, runtime) + }, { inject: ['agents'] })) + return { ctx, fiber, input, out, exit } +} + +/** Drive a fake idle timer past the 200ms flush delay. */ +function flushExit(): Promise { + return new Promise(resolve => setTimeout(resolve, 250)) +} + +describe('createStdioChat rendering', () => { + it('writes the welcome banner and prompt on start', async () => { + const { out } = await setup() + expect(out.text()).toBe('hi there\n> ') + }) + + it('falls back to default welcome/agent when called with empty config', async () => { + // createStdioChat is exported and may be driven directly (bypassing the + // Loader's schemastery validation), so it must default welcome/agent itself. + const { out } = await setup({}) + expect(out.text()).toBe('ready.\n> ') + // And it drives the default agent id 'main'. + }) + + it('renders text-delta chunks verbatim', async () => { + const { ctx, out } = await setup() + const agent = makeAgent('main') + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'text-delta', index: 0, text: 'hello' }) + expect(out.text()).toContain('hello') + }) + + it('wraps reasoning-delta in the dim SGR and resets on the following text-delta', async () => { + const { ctx, out } = await setup() + const agent = makeAgent('main') + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'think' }) + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'more' }) + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'text-delta', index: 0, text: 'answer' }) + expect(out.text()).toContain('\x1B[2mthinkmore\x1B[0m\nanswer') + }) + + it('ignores stream-chunk types it does not render', async () => { + const { ctx, out } = await setup() + const before = out.text() + const agent = makeAgent('main') + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'block-start', index: 0, blockType: 'text' }) + expect(out.text()).toBe(before) + }) + + it('renders turn-start and turn-end markers', async () => { + const { ctx, out } = await setup() + const agent = makeAgent('main') + ctx.emit('agent/turn-start', agent, 3) + expect(out.text()).toContain('[main turn 3] ') + ctx.emit('agent/turn-end', agent, 3, { kind: 'completed' }) + expect(out.text()).toContain('\n> ') + }) + + it('resets dim styling at turn-end if a turn ends mid-reasoning', async () => { + const { ctx, out } = await setup() + const agent = makeAgent('main') + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'mid' }) + ctx.emit('agent/turn-end', agent, 1, { kind: 'completed' }) + expect(out.text()).toContain('\x1B[2mmid\x1B[0m') + }) + + it('renders tool/call and tool/result session events', async () => { + const { ctx, out } = await setup() + const session = {} as Session + const callEvent = { + type: 'tool/call', seq: 1, time: 0, + data: { turn: 1, step: 0, callId: 'c1', name: 'bash', arguments: '{"command":"ls"}' }, + } as SessionEvent + ctx.emit('session/event', session, callEvent) + expect(out.text()).toContain('[tool call] bash({"command":"ls"})') + + const resultEvent = { + type: 'tool/result', seq: 2, time: 0, + data: { turn: 1, step: 0, callId: 'c1', content: [{ type: 'text', text: 'file.txt' }], isError: false }, + } as SessionEvent + ctx.emit('session/event', session, resultEvent) + expect(out.text()).toContain('[tool result] file.txt') + }) + + it('resets dim styling when a tool/call interrupts reasoning', async () => { + const { ctx, out } = await setup() + const agent = makeAgent('main') + ctx.emit('agent/stream-chunk', agent, 1, 0, { type: 'reasoning-delta', index: 0, text: 'r' }) + const session = {} as Session + ctx.emit('session/event', session, { + type: 'tool/call', seq: 1, time: 0, + data: { turn: 1, step: 0, callId: 'c1', name: 'bash', arguments: '{}' }, + } as SessionEvent) + expect(out.text()).toContain('\x1B[2mr\x1B[0m') + }) + + it('ignores session events it does not render', async () => { + const { ctx, out } = await setup() + const before = out.text() + ctx.emit('session/event', {} as Session, { + type: 'turn/start', seq: 1, time: 0, data: { turn: 1, trigger: { kind: 'continuation' } }, + } as SessionEvent) + expect(out.text()).toBe(before) + }) +}) + +describe('createStdioChat input', () => { + it('sends a typed line to an idle agent', async () => { + const { ctx, input } = await setup() + const agent = makeAgent('main', 'idle') + ctx.agents.register(agent) + input.feed('do a thing') + await new Promise(r => setImmediate(r)) + expect(agent.sent).toEqual([[{ type: 'text', text: 'do a thing' }]]) + expect(agent.steered).toEqual([]) + }) + + it('steers a typed line into a running agent', async () => { + const { ctx, input } = await setup() + const agent = makeAgent('main', 'running') + ctx.agents.register(agent) + input.feed('steer me') + await new Promise(r => setImmediate(r)) + expect(agent.steered).toEqual([[{ type: 'text', text: 'steer me' }]]) + expect(agent.sent).toEqual([]) + }) + + it('ignores blank lines', async () => { + const { ctx, input } = await setup() + const agent = makeAgent('main') + ctx.agents.register(agent) + input.feed(' ') + await new Promise(r => setImmediate(r)) + expect(agent.sent).toEqual([]) + }) + + it('logs and drops a line when the target agent is not running', async () => { + const { ctx, input } = await setup() + const spy = vi.spyOn(ctx.logger, 'error').mockImplementation(() => {}) + input.feed('nobody home') + await new Promise(r => setImmediate(r)) + expect(spy).toHaveBeenCalledWith('ui-stdio: agent "%s" is not running', 'main') + }) + + it('drives the agent named in config, not a hardcoded id', async () => { + const { ctx, input } = await setup({ welcome: 'w', agent: 'worker' }) + const agent = makeAgent('worker') + ctx.agents.register(agent) + input.feed('hi') + await new Promise(r => setImmediate(r)) + expect(agent.sent).toHaveLength(1) + }) +}) + +describe('createStdioChat EOF exit', () => { + it('exits immediately on EOF when no work was submitted', async () => { + const { input, exit } = await setup() + input.finish() + await flushExit() + expect(exit).toHaveBeenCalledWith(0) + }) + + it('waits for the agent to settle idle after running before exiting', async () => { + const { ctx, input, exit } = await setup() + const agent = makeAgent('main', 'idle') + ctx.agents.register(agent) + input.feed('work') + await new Promise(r => setImmediate(r)) + input.finish() + await new Promise(r => setImmediate(r)) + // Work submitted but no 'running' observed yet — must NOT exit. + expect(exit).not.toHaveBeenCalled() + // The turn starts, then settles. + ctx.emit('agent/status', agent, 'running') + ;(agent as { status: AgentStatus }).status = 'idle' + ctx.emit('agent/status', agent, 'idle') + await flushExit() + expect(exit).toHaveBeenCalledWith(0) + }) + + it('schedules the exit only once when idle fires repeatedly', async () => { + const { ctx, input, exit } = await setup() + const agent = makeAgent('main', 'running') + ctx.agents.register(agent) + input.feed('work') + await new Promise(r => setImmediate(r)) + ctx.emit('agent/status', agent, 'running') // sawRunning = true + input.finish() + await new Promise(r => setImmediate(r)) // let readline 'close' set stdinClosed + ;(agent as { status: AgentStatus }).status = 'idle' + // Two idle signals while stdin is already closed: the first arms the timer, + // the second must hit the already-scheduled guard, not arm a second. + ctx.emit('agent/status', agent, 'idle') + ctx.emit('agent/status', agent, 'idle') + await flushExit() + expect(exit).toHaveBeenCalledTimes(1) + }) + + it('does not exit on an idle transition for a different agent', async () => { + const { ctx, input, exit } = await setup() + const agent = makeAgent('main', 'idle') + ctx.agents.register(agent) + input.feed('work') + await new Promise(r => setImmediate(r)) + input.finish() + const other = makeAgent('other') + ctx.emit('agent/status', other, 'running') + ctx.emit('agent/status', other, 'idle') + await flushExit() + expect(exit).not.toHaveBeenCalled() + }) + + it('does not exit while a turn is still running at EOF', async () => { + const { ctx, input, exit } = await setup() + const agent = makeAgent('main', 'idle') + ctx.agents.register(agent) + input.feed('work') + await new Promise(r => setImmediate(r)) + ctx.emit('agent/status', agent, 'running') + ;(agent as { status: AgentStatus }).status = 'running' + input.finish() + // sawRunning is true, but the agent is still running — the idle gate holds. + ctx.emit('agent/status', agent, 'idle') // a stale/duplicate signal while status stays 'running' + await flushExit() + expect(exit).not.toHaveBeenCalled() + }) +}) + +describe('createStdioChat disposal (HMR safety)', () => { + it('never exits the process when EOF arrives after fiber dispose', async () => { + const { fiber, input, exit } = await setup() + await fiber.dispose() + // A late EOF after disposal (reader.close() also fires 'close') must not exit. + input.finish() + await flushExit() + expect(exit).not.toHaveBeenCalled() + }) + + it('cancels a scheduled exit if disposed within the flush window', async () => { + const { fiber, input, exit } = await setup() + // EOF with no work submitted schedules the 200ms flush-then-exit timer. + input.finish() + await new Promise(r => setImmediate(r)) + expect(exit).not.toHaveBeenCalled() // not yet — still inside the window + // Dispose BEFORE the timer fires: the tracked handle must be cleared. + await fiber.dispose() + await flushExit() + expect(exit).not.toHaveBeenCalled() + }) + + it('stops handling input after dispose', async () => { + const { ctx, fiber, input } = await setup() + const agent = makeAgent('main') + ctx.agents.register(agent) + await fiber.dispose() + // The readline interface is closed on dispose; a late line reaches no handler. + input.feed('too late') + await new Promise(r => setImmediate(r)) + expect(agent.sent).toEqual([]) + }) + + it('removes the agent/status listener on dispose', async () => { + const { ctx, fiber, input, exit } = await setup() + const agent = makeAgent('main', 'idle') + ctx.agents.register(agent) + input.feed('work') + await new Promise(r => setImmediate(r)) + await fiber.dispose() + // After dispose, status transitions must neither throw nor schedule an exit + // (the listener and the EOF-exit path are both torn down). + expect(() => { + ctx.emit('agent/status', agent, 'running') + ctx.emit('agent/status', agent, 'idle') + }).not.toThrow() + await flushExit() + expect(exit).not.toHaveBeenCalled() + }) +}) diff --git a/packages/support/ui-stdio/tsconfig.json b/packages/support/ui-stdio/tsconfig.json new file mode 100644 index 0000000000..b333de0302 --- /dev/null +++ b/packages/support/ui-stdio/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + } + ] +} diff --git a/packages/system-prompt/tsconfig.json b/packages/system-prompt/tsconfig.json deleted file mode 100644 index e226412a53..0000000000 --- a/packages/system-prompt/tsconfig.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../llm" } - ] -} diff --git a/packages/tool-bash/tsconfig.json b/packages/tool-bash/tsconfig.json deleted file mode 100644 index 4741cb67f3..0000000000 --- a/packages/tool-bash/tsconfig.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../llm" }, - { "path": "../tools" }, - { "path": "../agent" }, - { "path": "../bash" } - ] -} diff --git a/packages/tools/tsconfig.json b/packages/tools/tsconfig.json deleted file mode 100644 index 8e29228fc8..0000000000 --- a/packages/tools/tsconfig.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "lib" - }, - "include": ["src"], - "references": [ - { "path": "../../vendor/cosmokit" }, - { "path": "../../vendor/cordis" }, - { "path": "../llm" }, - { "path": "../system-prompt" }, - { "path": "../agent" } - ] -} diff --git a/packages/ui/README.md b/packages/ui/README.md new file mode 100644 index 0000000000..075dfd524d --- /dev/null +++ b/packages/ui/README.md @@ -0,0 +1,13 @@ +# ui/ — editor/client integration surfaces + +Integrations that expose the agent to an external editor or client. These are **product** packages: a real surface a user drives the harness through. + +| Package | Role | ctx key | +|---|---|---| +| `acp/` | Agent Client Protocol bridge: serves the agent to an ACP editor (Zed) over JSON-RPC stdio | (drives `ctx.agents`/`ctx.sessions`) | +| `stdio-agent/` | Terminal stdio chat APP: the agent-core spine + console logger + readline UI + a pre-created `main` agent, with a `bin` | (composition + `bin`) | +| `acp-agent/` | ACP server APP: the agent-core spine + JSONL persistence + the `acp` bridge (no stdout logger), with a `bin` | (composition + `bin`) | + +A UI integration is a client-driver plugin, not a loop change and not a capability seam: it consumes the existing `agent/*` event taxonomy and the `dsh-agent` factory. The readline `ui-stdio` plugin is the unstructured analogue but lives in `support/` because it exists chiefly for the examples and the coverage gate — `ui/` is reserved for surfaces shipped as product. + +`stdio-agent` and `acp-agent` are the two **app packages**: each composes the [`core/agent-core`](../core/agent-core/README.md) spine with its coupled front-door cluster (and owns the boot `bin`), so a leaf `cordis.yml` is just the swappable backends plus one app entry. They live in `ui/` because each IS a user-facing front door; the stdout-purity coupling (logger vs. no logger) becomes a property of the artifact rather than a leaf convention. diff --git a/packages/ui/acp-agent/README.md b/packages/ui/acp-agent/README.md new file mode 100644 index 0000000000..3403ef2126 --- /dev/null +++ b/packages/ui/acp-agent/README.md @@ -0,0 +1,41 @@ +# @deepseek-ai/dsh-acp-agent + +The **ACP server app**: a Cordis app plugin that composes the providerless agent spine ([`@deepseek-ai/dsh-agent-core`](../../core/agent-core/README.md)) with the front-door cluster an [Agent Client Protocol](../acp/README.md) server needs, and a `bin` that boots a leaf `cordis.yml` speaking ACP JSON-RPC on stdio. + +It is the structured counterpart to [`@deepseek-ai/dsh-stdio-agent`](../stdio-agent/README.md): both consume the same spine, but this one bakes in the OPPOSITE front-door cluster. + +## What it bakes in — and what it deliberately omits + +stdout is the ACP JSON-RPC channel, so the cluster is defined as much by what it LEAVES OUT as what it includes: + +| Plugin | Why | +|---|---| +| `@deepseek-ai/dsh-agent-core` | the spine, pre-creating **no** agents (ACP `session/new` creates them on demand) | +| `@deepseek-ai/dsh-session-persistence-jsonl` | durable JSONL session log (the bridge advertises `loadSession`) | +| `@deepseek-ai/dsh-acp` | the bridge that owns stdout for JSON-RPC | +| ~~console logger~~ | **omitted** — it writes to stdout and would corrupt the protocol frames ([the stdout-purity footgun](../acp/README.md)) | +| ~~`hmr`~~ | **omitted** — the editor owns the subprocess | + +Because the package wires no logger entry, an ACP leaf has **nothing to get wrong by default**: it only picks backends, so the common mistake — copying a console-logger entry from the stdio config — has no place here. (A leaf author technically *can* still add `@cordisjs/plugin-logger-console` as a sibling entry; the package can't forbid that. So the rule stands: never add a stdout logger to an ACP leaf — stdout is the JSON-RPC channel. Use a stderr exporter if you need logs.) + +## Config + +| Key | Default | Routed to | +|---|---|---| +| `model` | (required) | the per-session agent template the bridge creates agents from | +| `systemPrompt` | (required) | the per-session agent's system prompt | +| `persistenceRoot` | `./.sessions` | the JSONL backend's root directory | + +The leaf supplies the swappable backends: an LLM adapter (`llm-deepseek` for the real model, `llm-replay` for keyless snapshot replay) and a bash executor (`bash-local`). + +## The bin + +`dsh-acp-agent [path-to-cordis.yml]` (default `./cordis.yml`): + +- loads a gitignored `.env` from the cwd — **skipped** in snapshot REPLAY so a stray key can never trigger a live call; +- honors `DSH_SNAPSHOT=replay` by booting the sibling `cordis.snapshot.yml` (the keyless replay tree, `llm-replay` in place of `llm-deepseek`); +- in a snapshot run, disposes the context on stdin EOF so the session log is fully flushed before exit. + +Run it under `node --expose-internals`: the cordis Loader resolves the config's bare plugin specifiers through its internal module loader, active only under that flag. (`demo:acp` runs under tsx, whose tsconfig `paths` map resolves them instead.) + +All diagnostics go to **stderr** — stdout is the protocol. diff --git a/packages/ui/acp-agent/package.json b/packages/ui/acp-agent/package.json new file mode 100644 index 0000000000..72eb95b2f7 --- /dev/null +++ b/packages/ui/acp-agent/package.json @@ -0,0 +1,50 @@ +{ + "name": "@deepseek-ai/dsh-acp-agent", + "description": "ACP server app: the agent-core spine + JSONL persistence + the ACP bridge (no stdout logger, no hmr, no pre-created agents), with a bin to boot a leaf cordis.yml over JSON-RPC stdio", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "bin": { + "dsh-acp-agent": "lib/bin.js" + }, + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./bin": { + "types": "./lib/types/bin.d.ts", + "default": "./lib/bin.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/bin.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@cordisjs/plugin-include": "^1.0.4", + "@cordisjs/plugin-loader": "^1.0.0-rc.4", + "@deepseek-ai/dsh-acp": "^0.0.1", + "@deepseek-ai/dsh-agent-core": "^0.0.1", + "@deepseek-ai/dsh-session-persistence-jsonl": "^0.0.1", + "cordis": "^4.0.0-rc.6", + "schemastery": "^3.17.0" + }, + "devDependencies": { + "@cordisjs/plugin-include": "workspace:^", + "@cordisjs/plugin-loader": "workspace:^", + "@deepseek-ai/dsh-acp": "workspace:^", + "@deepseek-ai/dsh-agent-core": "workspace:^", + "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", + "cordis": "^4.0.0-rc.6", + "schemastery": "^3.17.0" + } +} diff --git a/packages/ui/acp-agent/src/bin.ts b/packages/ui/acp-agent/src/bin.ts new file mode 100644 index 0000000000..24e31f3251 --- /dev/null +++ b/packages/ui/acp-agent/src/bin.ts @@ -0,0 +1,168 @@ +#!/usr/bin/env node +/** + * The `dsh-acp-agent` bin: boot the ACP server from a leaf `cordis.yml` that + * loads the {@link @deepseek-ai/dsh-acp-agent} app plugin (plus an LLM adapter + * and a bash executor), speaking ACP JSON-RPC on stdio. + * + * Owns the ACP-specific boot glue the example's `start.ts` once held: + * - `.env` loading (`DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL`) — SKIPPED in + * snapshot REPLAY so a stray key can never trigger a live model call. + * - snapshot-mode config selection: `DSH_SNAPSHOT=replay` swaps the given + * `cordis.yml` for its sibling `cordis.snapshot.yml` (the keyless replay + * tree: `llm-replay` in place of `llm-deepseek`). + * - the stdin-dispose lifecycle: in a snapshot run the harness closes stdin + * when done, so dispose the context (flushing persistence) and exit cleanly. + * + * IMPORTANT: stdout is the ACP JSON-RPC channel. This bin writes diagnostics to + * STDERR only; the app plugin loads no stdout logger. A stray stdout write + * corrupts the protocol frames. + * + * Usage: `dsh-acp-agent [path-to-cordis.yml]` (default `./cordis.yml`). + * + * @module @deepseek-ai/dsh-acp-agent/bin + */ + +import { pathToFileURL } from 'node:url' +import { basename, dirname, resolve } from 'node:path' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' + +/** + * Resolve the config to boot, honoring snapshot REPLAY. Given the requested + * path, replay mode swaps a `cordis.yml` basename for `cordis.snapshot.yml` in + * the SAME directory (the keyless replay tree). Other modes use the path as-is. + * Returns an absolute path resolved from the cwd. + */ +export function resolveConfigPath(configPath: string, snapshotMode: string | undefined): string { + const absolute = resolve(process.cwd(), configPath) + if (snapshotMode !== 'replay') return absolute + const dir = dirname(absolute) + const replayName = basename(absolute).replace(/cordis\.ya?ml$/, 'cordis.snapshot.yml') + return resolve(dir, replayName) +} + +/** + * Load `DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL` from a gitignored `.env` in the + * cwd (Node native). Diagnostics go to STDERR (stdout is the protocol). In + * REPLAY mode the caller skips this entirely — replay must never reach the + * network, so a present `.env` must not enable a live call. + */ +function loadEnv(): void { + try { + process.loadEnvFile(resolve(process.cwd(), '.env')) + } catch (error) { + if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { + process.stderr.write(`dsh-acp-agent: failed to load .env: ${String(error)}\n`) + } + // ENOENT (no .env) is fine — rely on the ambient environment. + } +} + +/** + * Make a load failure fail loud with a clear message on stderr. Covers the + * failure path the entry-tree check below cannot: when the include's + * `[Service.init]` throws (e.g. a config FILE missing in a real directory), the + * cordis Loader surfaces it as an unhandled promise rejection AFTER `boot()` + * resolves — `loader.await()` does NOT rethrow it (`EntryTree.await()` uses + * `Promise.allSettled`, which swallows rejections). Node's default handler + * already exits non-zero on an unhandled rejection, so this does not change the + * exit code; it replaces the noisy stack dump with a single labelled line (on + * STDERR — stdout is the ACP JSON-RPC channel) and guarantees `process.exit(1)`. + * Install before `boot()`. + */ +export function installFailLoud(): void { + process.on('unhandledRejection', (err: unknown) => { + process.stderr.write(`dsh-acp-agent: fatal load failure: ${err instanceof Error ? err.stack ?? err.message : String(err)}\n`) + process.exit(1) + }) +} + +/** + * After the tree settles, assert every loader entry actually started. This is + * the load-bearing guard against the SILENT-exit-0 bug: a plugin module that + * fails to IMPORT (e.g. a config path in a non-existent directory) is caught and + * only LOGGED by the cordis Loader (`entry._init`), leaving the entry with no + * `fiber` and producing no rejection — so the process would otherwise exit 0. A + * started entry has a `fiber`; throw on any entry still missing one so `boot()` + * rejects. + * + * A `disabled` entry is the one legitimate fiber-less state: `Entry.refresh()` + * deliberately skips `init()` for it, so it settles without a fiber by design — + * a valid "plugin turned off" config, not a failed import. Exclude it. + */ +function assertEntriesLoaded(ctx: Context): void { + const failed = [...ctx.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled) + if (failed.length > 0) { + const names = failed.map(entry => entry.options.name).join(', ') + throw new Error(`dsh-acp-agent: plugin(s) failed to load: ${names} (see the error(s) logged above)`) + } +} + +/** + * Boot the Loader against `absoluteConfigPath`. The include is handed the + * config's ABSOLUTE `file://` URL as its `path`, so resolution never depends on + * `ctx.baseUrl` (an absolute URL ignores the base) and can never fall back to + * the cwd. `baseUrl` is still pinned to the config's directory so the config's + * OWN relative plugin/include paths resolve against it. Returns the root context + * once the whole tree has settled. + * + * The `await ctx.loader.await()` is load-bearing: `loader.create()` returns once + * the include ENTRY is registered, but the include then loads its child plugins + * asynchronously. Without awaiting the tree, `boot()` would resolve while the ACP + * bridge is still mounting — the process would have no stdin handle attached yet + * and could exit 0 silently. Awaiting keeps the process alive until the bridge + * is up. + * + * `loader.await()` does NOT rethrow load errors (`EntryTree.await()` uses + * `Promise.allSettled`), so failures are surfaced two ways: a plugin that fails + * to IMPORT leaves an entry with no fiber, caught here by + * {@link assertEntriesLoaded} (this `boot()` rejects); a plugin whose init THROWS + * surfaces as an unhandled rejection caught by {@link installFailLoud} (installed + * by `main()` before this runs). Together any load failure exits non-zero. + * + * Bare plugin specifiers in the config (`@deepseek-ai/dsh-*`, npm packages) are + * resolved by the cordis Loader's internal module loader, which is only active + * under `node --expose-internals`. The `demo:acp` script runs under tsx (whose + * tsconfig `paths` map resolves the workspace plugins instead), but a consumer + * running the built bin under plain node must pass `--expose-internals` so the + * Loader resolves the config's plugins from the config directory rather than + * relative to its own module. + */ +export async function boot(absoluteConfigPath: string): Promise { + const ctx = new Context() + ctx.baseUrl = pathToFileURL(dirname(absoluteConfigPath)).href + '/' + await ctx.plugin(Loader) + await ctx.loader.create({ + name: '@cordisjs/plugin-include', + config: { path: pathToFileURL(absoluteConfigPath).href }, + }) + await ctx.loader.await() + assertEntriesLoaded(ctx) + return ctx +} + +/** + * Entry point. Installs the fail-loud guard, selects the config (snapshot-aware), + * loads `.env` outside replay, boots, and — in a snapshot run — disposes the + * context on stdin EOF so the session log is fully flushed before exit and the + * harness's `waitForExit` resolves. In a normal editor session stdin stays open + * for the connection's lifetime (the editor kills the process), so the EOF + * handler never fires. + */ +export async function main(argv: string[] = process.argv.slice(2)): Promise { + installFailLoud() + const snapshotMode = process.env.DSH_SNAPSHOT + const configPath = resolveConfigPath(argv[0] ?? './cordis.yml', snapshotMode) + if (snapshotMode !== 'replay') loadEnv() + const ctx = await boot(configPath) + if (snapshotMode !== undefined) { + process.stdin.on('end', () => { + void ctx.fiber.dispose().then(() => { process.exit(0) }) + }) + } +} + +/* v8 ignore start -- top-level CLI invocation; the testable core is + resolveConfigPath()/boot()/main(), driven by the keyless snapshot + Loader-path tests */ +await main() +/* v8 ignore stop */ diff --git a/packages/ui/acp-agent/src/index.ts b/packages/ui/acp-agent/src/index.ts new file mode 100644 index 0000000000..625467cac2 --- /dev/null +++ b/packages/ui/acp-agent/src/index.ts @@ -0,0 +1,72 @@ +/** + * The ACP server app: the providerless agent spine ({@link + * @deepseek-ai/dsh-agent-core}) plus the coupled front-door cluster an ACP + * server needs — JSONL session persistence and the {@link @deepseek-ai/dsh-acp} + * bridge, and DELIBERATELY NOTHING that writes to stdout. + * + * The cluster is the OPPOSITE of {@link @deepseek-ai/dsh-stdio-agent}'s, and + * baking it in is the whole point: an ACP server speaks JSON-RPC on stdout, so + * a stray console logger would corrupt the protocol frames (the [stdout-purity + * footgun]). This package contains NO console-logger entry, NO `hmr` (the editor + * owns the subprocess), and pre-creates NO agents (ACP `session/new` creates + * them on demand) — so the default front door has no logger entry to get wrong. + * (A leaf `cordis.yml` could still add a sibling `@cordisjs/plugin-logger-console`, + * which this app does not prevent — so the rule "never add a stdout logger to an + * ACP leaf" still stands; the app just gives the leaf nothing to misconfigure.) + * + * The leaf supplies only the swappable backends: the LLM adapter (`llm-deepseek` + * for the real model, `llm-replay` for keyless snapshot replay) and the bash + * executor (`bash-local`). This app's {@link Config} (model, system prompt, + * persistence root) routes each value to where it is wired — model/prompt onto + * the bridge's per-session agent template, the root onto the JSONL backend. + * + * Plugin export shape: named `name`/`Config`/`apply`, NO default export — the + * cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray + * default would collapse the module to the bare `apply` and drop the `Config` + * namespace (see docs/postmortem/0001 — the exact bug that shipped here once). + * The keyless ACP snapshot/Loader-path tests guard this end-to-end. + * + * @module @deepseek-ai/dsh-acp-agent + */ + +import type { Context } from 'cordis' +import z from 'schemastery' +import * as acp from '@deepseek-ai/dsh-acp' +import * as agentCore from '@deepseek-ai/dsh-agent-core' +import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' + +export const name = 'acp-agent' + +/** + * App config: the swappable per-deployment values. `model`/`systemPrompt` + * configure the agent template the ACP bridge creates each session's agent from + * (NOT a pre-created agent — ACP creates agents at `session/new`); + * `persistenceRoot` is the JSONL backend's directory. + */ +export interface Config { + /** Model name for ACP-created agents (must have a registered adapter). */ + model: string + /** Per-agent system prompt for ACP-created agents. */ + systemPrompt: string + /** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */ + persistenceRoot?: string +} + +export const Config: z = z.object({ + model: z.string().required(), + systemPrompt: z.string().required(), + persistenceRoot: z.string().default('./.sessions'), +}) + +/** + * Compose the spine with the ACP front door. The agent-core bundle pre-creates + * NO agents (its `agents` list defaults to `[]`); the JSONL backend persists + * under `persistenceRoot`; the ACP bridge owns stdout for JSON-RPC and creates + * one agent per `session/new` from `model`/`systemPrompt`. No logger, no `hmr` — + * stdout stays pure. + */ +export function apply(ctx: Context, config: Config): void { + ctx.plugin(agentCore) + ctx.plugin(SessionPersistenceJsonl, { root: config.persistenceRoot ?? './.sessions' }) + ctx.plugin(acp, { model: config.model, systemPrompt: config.systemPrompt }) +} diff --git a/packages/ui/acp-agent/tests/acp-agent.spec.ts b/packages/ui/acp-agent/tests/acp-agent.spec.ts new file mode 100644 index 0000000000..7a02837fca --- /dev/null +++ b/packages/ui/acp-agent/tests/acp-agent.spec.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' +import * as acpAgent from '../src/index.ts' + +/** + * In-process unit coverage for the @deepseek-ai/dsh-acp-agent composition: + * mounting it brings up the agent-core spine + JSONL persistence + the ACP + * bridge in one `ctx.plugin`. Unlike the stdio app, this one loads NO + * Loader-only plugin (no hmr), so it mounts in a plain Context. + * + * The REAL Loader-path guard (export shape via `unwrapExports`, the headline + * ACP operations end-to-end) is the keyless bin smoke in `load-path.e2e.ts`; + * this spec asserts the composition and the persistenceRoot default branch. + */ +async function mount(config: acpAgent.Config): Promise { + const ctx = new Context() + await ctx.plugin(acpAgent, config) + // The bundle mounts its children inside apply() (not awaited there); let their + // fibers settle so the spine services are ready. + await new Promise(resolve => setTimeout(resolve, 50)) + return ctx +} + +describe('dsh-acp-agent composition', () => { + it('brings up the spine + persistence + the ACP bridge', async () => { + const ctx = await mount({ model: 'mock', systemPrompt: 'hi', persistenceRoot: '/tmp/dsh-acp-agent-test' }) + expect(ctx.get('agents')).toBeDefined() + expect(ctx.get('sessions')).toBeDefined() + expect(ctx.get('sessionPersistence')).toBeDefined() + expect(ctx.get('agentLoop')).toBeDefined() + // No pre-created agents — ACP session/new creates them on demand. + expect(ctx.get('agents')!.list()).toHaveLength(0) + await ctx.fiber.dispose() + }) + + it('defaults the persistence root when omitted', async () => { + // Exercises the `?? './.sessions'` fallback for a direct-apply caller that + // bypasses the schema's `.default(...)`: call `apply` directly (not via + // `ctx.plugin`, which validates+defaults the config first) with no + // persistenceRoot, so the runtime fallback is the one that fires. + const ctx = new Context() + acpAgent.apply(ctx, { model: 'mock', systemPrompt: 'hi' }) + await new Promise(resolve => setTimeout(resolve, 50)) + expect(ctx.get('sessionPersistence')).toBeDefined() + await ctx.fiber.dispose() + }) + + it('exposes its plugin shape', () => { + expect(acpAgent.name).toBe('acp-agent') + expect(acpAgent.Config).toBeDefined() + }) + + it('has the namespace-plugin export shape (no stray default) so the Loader keeps name/Config/apply', () => { + // Postmortem 0001 guard: a stray `export default apply` makes the Loader's + // `unwrapExports` (`exports.default ?? exports`) collapse the module to the + // bare `apply` function, DROPPING the named `name`/`Config`. This package has + // no `inject` export, so that collapse would NOT crash at load (the keyless + // bin smoke would still answer `initialize`) — it would silently lose its + // config schema. So guard the shape directly here: assert no `default` + // export, and that the real `unwrapExports` leaves `name`/`Config`/`apply` + // intact. Adding `export default` to src/index.ts fails this test. + expect('default' in acpAgent).toBe(false) + expect(typeof acpAgent.apply).toBe('function') + + const loader = Object.create(Loader.prototype) as Loader + const unwrapped = loader.unwrapExports(acpAgent) as Record + expect(unwrapped).toBe(acpAgent) + expect(unwrapped.name).toBe('acp-agent') + expect(unwrapped.Config).toBeDefined() + expect(typeof unwrapped.apply).toBe('function') + }) +}) diff --git a/packages/ui/acp-agent/tests/built-bin.e2e.ts b/packages/ui/acp-agent/tests/built-bin.e2e.ts new file mode 100644 index 0000000000..51c5d53c0b --- /dev/null +++ b/packages/ui/acp-agent/tests/built-bin.e2e.ts @@ -0,0 +1,195 @@ +import { spawn } from 'node:child_process' +import { mkdtemp, mkdir, rm, symlink, writeFile, readFile } from 'node:fs/promises' +import { existsSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { fileURLToPath, pathToFileURL } from 'node:url' +import { + ClientSideConnection, + ndJsonStream, + PROTOCOL_VERSION, + type Agent as AcpAgent, + type Client, + type RequestPermissionRequest, + type RequestPermissionResponse, + type SessionNotification, +} from '@agentclientprotocol/sdk' +import { Readable, Writable } from 'node:stream' +import { afterEach, describe, expect, it } from 'vitest' + +/** + * BUILT-ARTIFACT smoke for the published `dsh-acp-agent` bin. `load-path.e2e.ts` + * boots `src/bin.ts` under tsx — but the package's `bin` field points at + * `lib/bin.js`, run under plain `node` by a real consumer. This runs the REAL + * `lib/bin.js` under `node` (NOT tsx) and asserts it answers an `initialize` + * JSON-RPC frame, so a regression in the published entry (a settle race that + * exits before the bridge attaches, a stdout logger leaking onto the protocol) + * fails here. + * + * It build-gates: SKIPS if `lib/bin.js` is absent (suite run without + * `pnpm run build`); CI runs it after the build step. Setup mirrors a real + * install (a temp dir whose `node_modules` symlinks the built packages) and runs + * `node --expose-internals` (the cordis Loader resolves bare plugin specifiers + * via its internal module loader, active only under that flag). KEYLESS: + * `initialize` never reaches the model; a dummy key lets `llm-deepseek` boot. + */ + +const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url)) +const acpBin = join(repoRoot, 'packages/ui/acp-agent/lib/bin.js') + +const dshPackages = [ + 'core/agent-core', 'core/agent', 'core/session', 'core/system-prompt', + 'core/tools', 'core/agent-loop', 'llm/llm', 'llm/llm-deepseek', 'bash/bash', + 'bash/bash-local', 'bash/tool-bash', 'support/invariants', + 'session-persistence/session-persistence', + 'session-persistence/session-persistence-jsonl', 'ui/acp', 'ui/acp-agent', +] +const vendorPackages = [ + 'cordis', 'loader', 'include', 'timer', 'hmr', 'logger-console', + 'schemastery', 'cosmokit', +] +// Third-party deps the ACP bridge needs at runtime. They are declared by +// `dsh-acp` (NOT by `acp-agent`), so they live under `packages/ui/acp/node_modules` +// and are NOT necessarily hoisted where THIS test file can resolve them — pnpm's +// strict layout only exposes a package's deps under that package. Resolve each +// from the `ui/acp` package directory (the one that declares it) so the lookup +// works regardless of hoisting, then symlink it into the consumer for plain node. +const npmDeps = ['@agentclientprotocol/sdk', 'zod'] +const acpPkgDir = join(repoRoot, 'packages/ui/acp') + +async function pkgName(absDir: string): Promise { + const json = JSON.parse(await readFile(join(absDir, 'package.json'), 'utf8')) as { name: string } + return json.name +} + +async function link(target: string, name: string, nm: string): Promise { + const dest = join(nm, name) + await mkdir(dirname(dest), { recursive: true }) + await symlink(target, dest) +} + +/** Build a temp consumer dir + a minimal acp `cordis.yml`. Returns the dir. */ +async function makeConsumer(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'acp-built-bin-')) + const nm = join(dir, 'node_modules') + for (const rel of dshPackages) { + const abs = join(repoRoot, 'packages', rel) + await link(abs, await pkgName(abs), nm) + } + for (const v of vendorPackages) { + const abs = join(repoRoot, 'vendor', v) + await link(abs, await pkgName(abs), nm) + } + for (const dep of npmDeps) { + // Resolve from `ui/acp`'s package.json URL (the package that declares the + // dep), not this test file's location — `acp-agent` does not depend on these. + const fromAcp = pathToFileURL(join(acpPkgDir, 'package.json')).href + const resolved = fileURLToPath(import.meta.resolve(`${dep}/package.json`, fromAcp)) + await link(dirname(resolved), dep, nm) + } + await writeFile(join(dir, 'cordis.yml'), [ + '- id: llm-deepseek', + ' name: \'@deepseek-ai/dsh-llm-deepseek\'', + ' config:', + ' apiKey: !!js process.env.DEEPSEEK_API_KEY', + ' models: [deepseek-v4-flash]', + '- id: bash', + ' name: \'@deepseek-ai/dsh-bash-local\'', + '- id: acp-agent', + ' name: \'@deepseek-ai/dsh-acp-agent\'', + ' config:', + ' model: deepseek-v4-flash', + ' systemPrompt: \'test agent\'', + '', + ].join('\n')) + return dir +} + +let consumer: string | undefined +let child: ReturnType | undefined + +afterEach(async () => { + if (child !== undefined) { child.kill('SIGKILL'); child = undefined } + if (consumer !== undefined) await rm(consumer, { recursive: true, force: true }) + consumer = undefined +}) + +describe.skipIf(!existsSync(acpBin))('dsh-acp-agent BUILT bin (node lib/bin.js, no tsx)', () => { + it('boots the published bin and answers an initialize JSON-RPC frame on stdout', async () => { + consumer = await makeConsumer() + child = spawn(process.execPath, ['--expose-internals', acpBin, './cordis.yml'], { + cwd: consumer, + // Dummy key: initialize never reaches the model, so it is never used. + env: { ...process.env, DEEPSEEK_API_KEY: process.env.DEEPSEEK_API_KEY ?? 'sk-dummy-for-boot' }, + stdio: ['pipe', 'pipe', 'pipe'], + }) + const stderr: string[] = [] + child.stderr!.setEncoding('utf8') + child.stderr!.on('data', (c: string) => stderr.push(c)) + // Tee raw stdout for a protocol-purity check, and feed it to the SDK client. + const rawOut: string[] = [] + const passthrough = new Readable({ read() {} }) + child.stdout!.on('data', (buf: Buffer) => { rawOut.push(buf.toString('utf8')); passthrough.push(buf) }) + child.stdout!.on('end', () => passthrough.push(null)) + const stream = ndJsonStream( + Writable.toWeb(child.stdin!) as WritableStream, + Readable.toWeb(passthrough) as ReadableStream, + ) + const makeClient = (_a: AcpAgent): Client => ({ + sessionUpdate(_p: SessionNotification): Promise { return Promise.resolve() }, + requestPermission(_p: RequestPermissionRequest): Promise { + return Promise.resolve({ outcome: { outcome: 'cancelled' } }) + }, + }) + const client = new ClientSideConnection(makeClient, stream) + + const init = await client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + // A response at all proves the built bin booted the bridge (the settle-race + // regression would exit before answering); loadSession proves the real app + // mounted, not a collapsed export shape. + expect(init.agentCapabilities?.loadSession).toBe(true) + expect(stderr.join('')).not.toContain('without inject') + // stdout purity: every emitted line is a JSON-RPC frame, no logger leak. + for (const line of rawOut.join('').split('\n').filter(l => l.trim().length > 0)) { + expect(() => JSON.parse(line) as unknown).not.toThrow() + } + }, 30_000) + + it('fails LOUD (non-zero exit + stderr) on a config whose directory does not exist', async () => { + // A typo'd config path must fail clearly, not exit 0. The include plugin + // itself cannot be imported from a non-existent dir; the Loader logs that and + // leaves the entry with no fiber, which boot()'s entry-load check throws on. + const { code, stderr } = await runBinExpectingExit('/nonexistent/dir/cordis.yml') + expect(code).not.toBe(0) + expect(stderr).toContain('failed to load') + }, 30_000) + + it('fails LOUD (non-zero exit + stderr) on a missing config file in a real directory', async () => { + // The directory exists (the include imports), but the file does not — the + // include's init throws "config file not found", which surfaces as an + // unhandled rejection the fail-loud guard turns into a non-zero exit. + consumer = await makeConsumer() + const { code, stderr } = await runBinExpectingExit('./does-not-exist.yml', consumer) + expect(code).not.toBe(0) + expect(stderr).toContain('config file not found') + }, 30_000) +}) + +/** Spawn the built acp bin against `configArg` and resolve with its exit code + stderr. */ +function runBinExpectingExit(configArg: string, cwd: string = tmpdir()): Promise<{ code: number; stderr: string }> { + return new Promise((resolve, reject) => { + const proc = spawn(process.execPath, ['--expose-internals', acpBin, configArg], { + cwd, + env: { ...process.env, DEEPSEEK_API_KEY: process.env.DEEPSEEK_API_KEY ?? 'sk-dummy-for-boot' }, + stdio: ['pipe', 'pipe', 'pipe'], + }) + child = proc + let stderr = '' + proc.stderr.setEncoding('utf8') + proc.stderr.on('data', (c: string) => { stderr += c }) + const timer = setTimeout(() => { proc.kill('SIGKILL'); reject(new Error(`bin did not exit within 25s. stderr:\n${stderr}`)) }, 25_000) + proc.on('exit', (code) => { clearTimeout(timer); resolve({ code: code ?? -1, stderr }) }) + proc.on('error', (err) => { clearTimeout(timer); reject(err) }) + proc.stdin.end() + }) +} diff --git a/packages/ui/acp-agent/tests/load-path.e2e.ts b/packages/ui/acp-agent/tests/load-path.e2e.ts new file mode 100644 index 0000000000..fd559d99bf --- /dev/null +++ b/packages/ui/acp-agent/tests/load-path.e2e.ts @@ -0,0 +1,147 @@ +import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process' +import { Readable, Writable } from 'node:stream' +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' +import { + ClientSideConnection, + ndJsonStream, + PROTOCOL_VERSION, + type Agent as AcpAgent, + type Client, + type RequestPermissionRequest, + type RequestPermissionResponse, + type SessionNotification, +} from '@agentclientprotocol/sdk' + +/** + * REAL-load-path smoke for @deepseek-ai/dsh-acp-agent: boot the app through its + * own `bin` (the demo:acp entry) as a subprocess, driving the cordis Loader and + * `unwrapExports` over a minimal `cordis.yml` that loads THIS package. This is + * the guard a hand-built `ctx.plugin({...})` mount structurally cannot be — that + * bypasses `unwrapExports`, the exact path that once dropped the bridge's + * `inject` and shipped (docs/postmortem/0001). It exercises the headline ACP + * operations end-to-end: `initialize` → `session/new` → `session/load`. + * + * KEYLESS: `session/new` and `session/load` reach the agent FACTORY but never + * the model (no prompt is sent), so no DEEPSEEK_API_KEY is needed. A dummy key + * lets `llm-deepseek`'s `apply()` (key-PRESENT check only) boot the tree. + * + * The config is written into a temp dir whose cwd IS the session workspace, so + * the bash workdir validation passes. We point tsx at the repo-root tsconfig + * (TSX_TSCONFIG_PATH) because the child's cwd is outside the repo and the + * unbuilt `paths` map is found by searching UP from cwd. + */ + +const binScript = fileURLToPath(new URL('../src/bin.ts', import.meta.url)) +const tsxLoader = fileURLToPath(import.meta.resolve('tsx')) +// Repo root is four levels up from packages/ui/acp-agent/tests. +const repoTsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)) + +// A minimal leaf that loads this app + the two backends — the same shape as +// examples/acp-agent/cordis.yml, inlined so the package test owns its fixture. +const CORDIS_YML = ` +- id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' + config: + apiKey: !!js process.env.DEEPSEEK_API_KEY + models: [deepseek-v4-flash] +- id: bash + name: '@deepseek-ai/dsh-bash-local' +- id: acp-agent + name: '@deepseek-ai/dsh-acp-agent' + config: + model: deepseek-v4-flash + systemPrompt: 'You are a test agent.' +` + +interface Spawned { + child: ChildProcessWithoutNullStreams + client: ClientSideConnection + stderr: string[] +} + +let spawned: Spawned | undefined +let workdir: string | undefined + +afterEach(async () => { + if (spawned !== undefined) { + spawned.child.kill('SIGKILL') + spawned = undefined + } + if (workdir !== undefined) await rm(workdir, { recursive: true, force: true }) + workdir = undefined +}) + +async function boot(): Promise { + workdir = await mkdtemp(join(tmpdir(), 'acp-agent-pkg-')) + const cwd = workdir + const configPath = join(cwd, 'cordis.yml') + await writeFile(configPath, CORDIS_YML) + const child = spawn( + process.execPath, + ['--import', tsxLoader, binScript, configPath], + { + cwd, + env: { + ...process.env, + TSX_TSCONFIG_PATH: repoTsconfig, + // Key-present check only; no prompt is sent, so the model is never called. + DEEPSEEK_API_KEY: process.env.DEEPSEEK_API_KEY ?? 'keyless-acp-agent-smoke', + }, + stdio: ['pipe', 'pipe', 'pipe'], + }, + ) + const stderr: string[] = [] + child.stderr.setEncoding('utf8') + child.stderr.on('data', (chunk: string) => stderr.push(chunk)) + const stream = ndJsonStream( + Writable.toWeb(child.stdin) as WritableStream, + Readable.toWeb(child.stdout) as ReadableStream, + ) + const makeClient = (_agent: AcpAgent): Client => ({ + sessionUpdate(_params: SessionNotification): Promise { + return Promise.resolve() + }, + requestPermission(_params: RequestPermissionRequest): Promise { + return Promise.resolve({ outcome: { outcome: 'cancelled' } }) + }, + }) + const client = new ClientSideConnection(makeClient, stream) + spawned = { child, client, stderr } + return { ...spawned, cwd } +} + +describe('dsh-acp-agent real-load-path smoke (bin + Loader, keyless)', () => { + it('boots via its bin and answers initialize → session/new → session/load', async () => { + const { client, cwd, stderr } = await boot() + // initialize: a broken export shape (collapsed bridge plugin, dropped inject) + // crashes the tree on the first service read here — see postmortem 0001. + const init = await client.initialize({ + protocolVersion: PROTOCOL_VERSION, + clientCapabilities: {}, + }) + expect(init.agentCapabilities?.loadSession).toBe(true) + + // session/new reaches the agent FACTORY (create) without the model. + const { sessionId } = await client.newSession({ cwd, mcpServers: [] }) + expect(sessionId).toBeTruthy() + + // session/load reaches the resume FACTORY + persistence without the model: + // load an UNKNOWN id (loading the live `sessionId` would correctly reject as + // "already loaded"). The bridge consults `sessionPersistence.list()` then + // `agents.resume()`, both of which run from the JSON-RPC read loop OUTSIDE + // the bridge's inject scope — the exact path postmortem 0001 crashed. A + // healthy tree rejects with a not-found error; a broken export shape would + // instead throw "cannot get property … without inject" before reaching it. + const unknownId = '00000000-0000-4000-8000-000000000000' + await client.loadSession({ sessionId: unknownId, cwd, mcpServers: [] }).then( + () => { throw new Error('expected session/load of an unknown id to reject') }, + (error: unknown) => { expect(String(error)).not.toContain('without inject') }, + ) + + expect(stderr.join('')).not.toContain('without inject') + }, 30_000) +}) diff --git a/packages/ui/acp-agent/tsconfig.json b/packages/ui/acp-agent/tsconfig.json new file mode 100644 index 0000000000..ffea8ec6f6 --- /dev/null +++ b/packages/ui/acp-agent/tsconfig.json @@ -0,0 +1,30 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../../vendor/loader" + }, + { + "path": "../acp" + }, + { + "path": "../../core/agent-core" + }, + { + "path": "../../session-persistence/session-persistence-jsonl" + } + ] +} diff --git a/packages/ui/acp-agent/tsdown.config.ts b/packages/ui/acp-agent/tsdown.config.ts new file mode 100644 index 0000000000..9dd130b30d --- /dev/null +++ b/packages/ui/acp-agent/tsdown.config.ts @@ -0,0 +1,19 @@ +import { defineConfig } from 'tsdown' + +/** + * acp-agent ships TWO entries: the plugin (`index`) and the CLI `bin` (`bin`), + * the latter referenced by package.json `bin`/`exports["./bin"]`. The root + * tsdown builds only `lib/types/index.js`, so this override adds + * `lib/types/bin.js`. Declarations come from `tsc -b` (dts: false), + * matching every package. + */ +export default defineConfig({ + entry: ['lib/types/index.js', 'lib/types/bin.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, +}) diff --git a/packages/acp/README.md b/packages/ui/acp/README.md similarity index 79% rename from packages/acp/README.md rename to packages/ui/acp/README.md index c514361b15..ad50383542 100644 --- a/packages/acp/README.md +++ b/packages/ui/acp/README.md @@ -1,8 +1,8 @@ # @deepseek-ai/dsh-acp -The **Agent Client Protocol (ACP)** bridge: exposes the DeepSeek Harness coding agent as an ACP server over JSON-RPC stdio, so editors (Zed and other ACP clients) can drive it — streaming render, tool-call display, and resumable sessions. **N concurrent sessions per connection** (see [ACP multi-session](../../docs/rfc/proposed/2026-06-14-acp-multi-session.md)): each maps to its own `LoopAgent`, and every event is demuxed strictly by session id so two sessions streaming at once never interleave. +The **Agent Client Protocol (ACP)** bridge: exposes the DeepSeek Harness coding agent as an ACP server over JSON-RPC stdio, so editors (Zed and other ACP clients) can drive it — streaming render, tool-call display, and resumable sessions. Zed is the current target client: baseline ACP behavior should remain reasonable for other clients, but bridge capabilities and compatibility decisions are evaluated against Zed first. **N concurrent sessions per connection** (see [ACP multi-session](../../../docs/rfc/proposed/feature/2026-06-14-acp-multi-session.md)): each maps to its own `ReactLoopAgent`, and every event is demuxed strictly by session id so two sessions streaming at once never interleave. -It is a **client-driver / UI plugin**, the structured analogue of the readline `stdio-chat` plugin — NOT a loop change and NOT a [capability seam](../../docs/rfc/implemented/2026-06-13-capability-seams.md). It consumes the existing `agent/*` event taxonomy, the `dsh-agent` create/resume factory, and `dsh-session-persistence`. +It is a **client-driver / UI plugin**, the structured analogue of the readline `stdio-chat` plugin — NOT a loop change and NOT a [capability seam](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md). It consumes the existing `agent/*` event taxonomy, the `dsh-agent` create/resume factory, and `dsh-session-persistence`. ## Service / plugin @@ -27,14 +27,14 @@ It is a **client-driver / UI plugin**, the structured analogue of the readline ` | `session/new` | `ctx.agents.create({ sessionId, meta:{cwd} })` | creates a new session/agent; N concurrent sessions are allowed, keyed by id; `cwd` must be absolute (it becomes the session's workspace — see Per-session cwd); non-empty `additionalDirectories` and `mcpServers` rejected | | `session/load` | `ctx.agents.resume(...)` | replays the persisted event log to the client as `session/update` — the USER side (`user/message` → `user_message_chunk`), assistant text/reasoning (`assistant/chunk`), and tool calls/results (`tool/call` + `tool/result`). Re-loading an already-live id is rejected; the id's load slot is reserved (`loadingIds`) BEFORE the async resume so a pipelined load of the SAME id can't leak a second agent (distinct ids load concurrently). The resumed session keeps its PERSISTED header `cwd`, so its bash tools run in the original workspace; the requested `cwd` must be absolute and match the persisted `cwd`. After the async resume a `closed` re-check refuses to install a record if the bridge tore down mid-load | | `session/prompt` | `agent.send()` | supports ACP `text` and `resource_link` blocks; rejects image/audio/embedded resource and empty prompts; one in-flight prompt PER session (independent); settles on the OWNING turn's end (a turn that ends in `error` rejects the RPC) | -| `session/cancel` | `agent.abort()` | aborts a running step + settles the prompt `cancelled` for ONLY that session — a cancel never touches another session's stream or prompt (see limitation below) | +| `session/cancel` | `agent.cancel()` | the queue-aware cancel: aborts a running step, clears queued + steering work, and drops a turn about to start, then settles the prompt `cancelled` — for ONLY that session (a cancel never touches another session's stream or prompt) | | `session/update` | `session/event` | `agent_message_chunk` (text-delta), `agent_thought_chunk` (reasoning-delta), `user_message_chunk` (load replay), `tool_call`/`tool_call_update` (title/kind/rawInput/content owned by the TOOL via `presentCall`/`presentResult` — see Tool-call presentation) | ## Multi-session The bridge multiplexes N sessions over one connection. Live sessions are held in a `Map` (forward) with a `WeakMap` reverse map so `agent/*` events — which carry only the `Agent` — demux in O(1). Every `session/event` and `agent/status` is routed strictly to its owning record, so concurrent sessions never cross-settle or interleave their `session/update` notifications. State is per session: one in-flight prompt each, `session/cancel` aborts and settles only its own agent/prompt, and disposal drains every live session in parallel to quiescence. (Per-session *permission* ownership is reserved for the deferred permission gate — `TODO(rfc010-permission-gate)`.) -Background-task isolation rides on `dsh-tool-bash`: bash task ids are global and predictable, so the tool layer records each background task's owning agent and `bash_output`/`bash_kill` reject a task owned by a different agent — one session's agent can't read or kill another's task. +Background-task isolation rides on `dsh-tool-bash`: bash task ids are global and predictable, so each task carries an opaque owner token — the owning agent's `session.header.id` — stored on the task inside the executor (`dsh-bash`'s `ownerOf(id)` seam). `bash_output`/`bash_kill` reject a task whose token differs from the caller's session token, so one session's agent can't read or kill another's task. Ownership is by session TOKEN, not `Agent` object identity — a different `Agent` object on the same session may access the task — and because the token lives on the executor's task it survives a `tool-bash` HMR reload. ## Per-session cwd @@ -53,7 +53,7 @@ A tool whose call IS a shell command (`bash`) can render as a real **terminal ca - `tool_call`: `content:[…, {type:'terminal', terminalId}]` + `_meta.terminal_info.{terminal_id, cwd}` — the terminal id is the harness `callId`; the cwd is the tool's explicit absolute `terminal.cwd`, else a relative `terminal.cwd` resolved against the session cwd, else the session's workspace cwd (the bridge fills the default, since the pure tool presenter can't see it). Any pending `content` the tool supplied (e.g. bash's `description`) renders BEFORE the terminal block, so the description sits above the card. - `tool_call_update`: `_meta.terminal_output.{terminal_id, data}` (the captured output) plus `_meta.terminal_exit.{terminal_id, exit_code | signal}` when the tool reported a structured exit. In terminal mode the update's `content` is OMITTED — an ACP `tool_call_update.content` REPLACES the call's content, so sending the fenced text block would clobber the terminal content block from the call. -When the client does NOT advertise the capability, none of the `_meta`/terminal content is emitted: the `tool_call` shows the `description` content block and the `tool_call_update` carries the ` ```console ` text block (above) as the rendering — so a non-Zed client is never worse off. The `_meta` object is ACP's spec-blessed extensibility point; the specific `terminal_info`/`terminal_output`/`terminal_exit` keys are a Zed convention, not the ACP `terminal/create` sub-protocol (which would make the editor execute the command, bypassing `dsh-bash`'s sandbox/env-scrub/ownership/cwd). Live incremental streaming and command classification are follow-ups. See [the terminal-rendering RFC](../../docs/rfc/implemented/2026-06-18-acp-terminal-and-tool-rendering.md). +When the client does NOT advertise the capability, none of the `_meta`/terminal content is emitted: the `tool_call` shows the `description` content block and the `tool_call_update` carries the ` ```console ` text block (above) as the rendering — so a non-Zed client is never worse off. The `_meta` object is ACP's spec-blessed extensibility point; the specific `terminal_info`/`terminal_output`/`terminal_exit` keys are a Zed convention, not the ACP `terminal/create` sub-protocol (which would make the editor execute the command, bypassing `dsh-bash`'s sandbox/env-scrub/ownership/cwd). Live incremental streaming and command classification are follow-ups. See [the terminal-rendering RFC](../../../docs/rfc/implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md). ## Settle-exactly-once @@ -61,18 +61,16 @@ A `session/prompt` resolves (or rejects) exactly once, keyed off the canonical s ## Disposal & disconnect -Teardown reaches quiescence: for EVERY live session settle any pending prompt as `cancelled`, `agent.abort()`, then `await agent.whenIdle()` — the interface-level quiescence signal (NOT `agent/status('disposed')`, which fires before the driver exits). The agents drain in parallel. The same teardown runs on a **client disconnect** (`conn.closed` resolves when the editor quits / the transport EOFs), so a vanished client never leaves an orphaned running agent whose `session/update` writes are silently swallowed. The two paths are idempotent and memoized (the first clears the `sessions` map; a second caller awaits the same teardown promise). +Teardown reaches quiescence: for EVERY live session settle any pending prompt as `cancelled`, then run that session's [`AgentHandle`](../../core/agent/README.md) `dispose()` — which stops the loop (sets `disposed` + aborts the in-flight step), `await`s the loop's exit (the final `turn/end` + `session/flush` are captured while the session is still attached), unregisters the agent, and removes its session from the store. A turn cut off mid-flight by teardown ends with reason `disposed` (not `aborted` — `dispose()` uses the disposed path, not `session/cancel`'s queue-aware `cancel()`). The per-session disposes run in parallel. The same teardown runs on a **client disconnect** (`conn.closed` resolves when the editor quits / the transport EOFs), so a vanished client never leaves an orphaned running — or idled-but-still-registered — agent whose `session/update` writes are silently swallowed. The two paths are idempotent and memoized (the first clears the `sessions` map; a second caller awaits the same teardown promise). ## Known limitations (tracked TODOs) -- **`TODO(rfc010-permission-gate)`** — the `tools/execute` permission gate (`session/request_permission`) is NOT implemented; tools run with the executor's full authority. The `agent→sessionId` reverse map is in place so the gate can route a permission request (which receives only `exec.agent`) back to its originating session. [ACP support](../../docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md) and [ACP multi-session](../../docs/rfc/proposed/2026-06-14-acp-multi-session.md) stay `proposed` until the gate (and per-session permission ownership) land. -- **`TODO(rfc010-cancel-prestep)`** — `session/cancel` (and teardown/disconnect) is honest RPC/UI cancellation plus best-effort abort: a *running* step is aborted, but a turn that is queued-but-not-yet-started (the gap before `agent.abort()` has an `AbortController` to signal) may still run to completion. This same window means disposal/disconnect can return while one short queued turn per session still runs, and a prompt accepted right after a pre-step cancel can be batched into the cancelled turn (the loop merges queued messages into one turn). A loop-level queue-aware cancel will close this; the single-in-flight-per-session rule bounds the worst case to one extra prompt per session. -- **`TODO(rfc010-agent-disposal)`** — the factory (`ctx.agents.create`/`resume`) returns no per-agent disposer, so teardown aborts+drains each agent but cannot individually unregister it; on a bare client disconnect (no host dispose) the idled agents linger in `ctx.agents` until the host context disposes. A reconnect spins up a fresh context, so this strands no work; a per-agent disposal seam is the follow-up. +- **`TODO(rfc010-permission-gate)`** — the `tools/execute` permission gate (`session/request_permission`) is NOT implemented; tools run with the executor's full authority. The `agent→sessionId` reverse map is in place so the gate can route a permission request (which receives only `exec.agent`) back to its originating session. [ACP support](../../../docs/rfc/proposed/feature/2026-06-14-acp-agent-client-protocol.md) and [ACP multi-session](../../../docs/rfc/proposed/feature/2026-06-14-acp-multi-session.md) stay `proposed` until the gate (and per-session permission ownership) land. - **`additionalDirectories`** — rejected. A session operates in its single `cwd` (see Per-session cwd); widening the tool/filesystem scope to extra roots is a separate sandbox concern, not yet implemented. ## stdout is the protocol -The JSON-RPC frames go on stdout, so this plugin MUST run in an example that loads **no stdout logger** (the console logger writes to stdout and would corrupt the frames). The guarantee is config-only — see `examples/acp-agent` (no console logger) and [ACP support risks](../../docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md#risks). A stderr exporter is fine for logging. +The JSON-RPC frames go on stdout, so this plugin MUST run in an example that loads **no stdout logger** (the console logger writes to stdout and would corrupt the frames). The guarantee is config-only — see `examples/acp-agent` (no console logger) and [ACP support risks](../../../docs/rfc/proposed/feature/2026-06-14-acp-agent-client-protocol.md#risks). A stderr exporter is fine for logging. ## Running diff --git a/packages/ui/acp/acp-feature-support.md b/packages/ui/acp/acp-feature-support.md new file mode 100644 index 0000000000..429c862b27 --- /dev/null +++ b/packages/ui/acp/acp-feature-support.md @@ -0,0 +1,162 @@ +# ACP feature support checklist + +A structured inventory of [Agent Client Protocol](https://agentclientprotocol.com) (ACP) features and where the harness's ACP bridge ([`@deepseek-ai/dsh-acp`](README.md)) stands on each. The bridge exposes the harness agent as an ACP **server** (the agent side of an editor↔agent connection), so "supported" below means *the bridge implements the agent's half* — answering an agent method, advertising a capability, or calling a client method. + +## Scope + +This tracks the **stable** ACP v1 surface (schema `1.14.0`, `schema/v1/schema.json`) PLUS the **unstable/draft** features that the two reference adapters — [`claude-agent-acp`](https://github.com/zed-industries/claude-code-acp) (Claude Code) and [`codex-acp`](https://github.com/zed-industries/codex-acp) (OpenAI Codex) — actually ship. A purely-unstable feature that neither reference adapter uses is omitted (see [Out of scope](#out-of-scope)). + +Legend: ✅ supported · ⚠️ partial / fallback · ❌ not yet · — n/a. The **Stable** column marks whether the feature is in the released v1 schema (S) or only the unstable schema (U). The **Claude** / **Codex** columns record whether each reference adapter ships it, as a maturity signal. + +## At a glance + +The bridge implements the **core prompt-turn loop** for N concurrent sessions: initialize, session new/load, prompt, cancel, streamed assistant/thought chunks, tool-call rendering (including Zed terminal cards), and resumable session replay. The largest **unbuilt** areas are the **permission gate** (`session/request_permission`), **MCP passthrough**, **session modes / config options / model selection**, **slash commands**, and **agent plans** — all of which both reference adapters ship — plus the client **filesystem** and **terminal** method families (which the adapters mostly do NOT drive either — see rows 43-49). See [Gap summary](#gap-summary). + +## 1. Agent methods (client → agent) + +| Method | Stable | Bridge | Claude | Codex | Notes | +|---|---|---|---|---|---| +| `initialize` | S | ✅ | ✅ | ✅ | Negotiates `PROTOCOL_VERSION`; advertises `loadSession` + baseline prompt caps. Snapshots the Zed `_meta.terminal_output` client cap. | +| `authenticate` | S | ⚠️ | ✅ | ✅ | No-op stub; the bridge advertises no `authMethods`, so there is nothing to authenticate. | +| `logout` | S | ❌ | ✅ | ✅ | Gated by `agentCapabilities.auth.logout`; not advertised. | +| `session/new` | S | ✅ | ✅ | ✅ | Maps to `agents.create`; requires an absolute `cwd` (becomes the session workspace); rejects non-empty `additionalDirectories` / `mcpServers`. | +| `session/load` | S | ✅ | ✅ | ✅ | Maps to `agents.resume` + full event-log replay; validates persisted `cwd` before constructing the agent. | +| `session/resume` | S | ❌ | ✅ | ✅ | Reconnect WITHOUT replay; gated by `sessionCapabilities.resume`. Not advertised. | +| `session/close` | S | ❌ | ✅ | ✅ | No `session/close` handler — the SDK dispatch returns `method_not_found`. The bridge tears sessions down on client disconnect / Cordis disposal (cross-cutting, see [§8](#8-cross-cutting)), but that is not the on-demand per-session method. | +| `session/prompt` | S | ✅ | ✅ | ✅ | Maps to `agent.send`; one in-flight prompt per session; settles on the owning turn's end. | +| `session/cancel` | S | ✅ | ✅ | ✅ | Queue-aware `agent.cancel`; settles the in-flight prompt `cancelled`, scoped to the one session. | +| `session/set_mode` | S | ❌ | ✅ | ✅ | Session modes not modeled (see [§6 Modes](#6-session-modes--config-options--models)). | +| `session/set_config_option` | S | ❌ | ✅ | ✅ | Config options not modeled. | +| model selection | S | ❌ | ✅ | ✅ | No distinct stable `session/set_model` — model is the `model`-category `session/set_config_option`. The bridge fixes the model per-bridge via config; no runtime switch. Codex still uses the legacy `unstable_setSessionModel` ext method. | +| `session/list` | S | ❌ | ✅ | ✅ | Gated by `sessionCapabilities.list`. The harness HAS `sessionPersistence.list()` (used internally for load-cwd validation) but does not expose it over ACP. | +| `session/delete` | S | ❌ | ✅ | ✅ | Gated by `sessionCapabilities.delete`. | +| `session/fork` | U | ❌ | ✅ | ❌ | Claude ships `unstable_forkSession`; Codex does not. | + +## 2. Client methods the agent CALLS (agent → client) + +These are capabilities the bridge would *drive* on the editor. The harness runs tools in-process (its own `dsh-bash` executor, direct file I/O), so it does not yet delegate to the editor for any of these. + +| Method | Stable | Bridge | Claude | Codex | Notes | +|---|---|---|---|---|---| +| `session/update` | S | ✅ | ✅ | ✅ | The bridge's primary output channel (see [§4](#4-sessionupdate-variants)). | +| `session/request_permission` | S | ❌ | ✅ | ✅ | **The biggest gap.** Tools run with the executor's full authority; no user authorization round-trip. The `agent→sessionId` reverse map is already in place to route a future permission request. Tracked `TODO(rfc010-permission-gate)`. | +| `fs/read_text_file` | S | ❌ | ✅ | ❌ | The harness reads files directly (it does not see the editor's unsaved buffer state). Claude delegates; Codex does not. | +| `fs/write_text_file` | S | ❌ | ✅ | ❌ | Same — direct writes, no editor delegation. | +| `terminal/create` | S | ❌ | ❌ | ❌ | Neither reference adapter drives the client terminal API either — both, like the bridge, render shell output as tool-call content + a `_meta` channel (see [§5 Terminal](#terminal-rendering)). | +| `terminal/output` | S | ❌ | ❌ | ❌ | As above. | +| `terminal/wait_for_exit` | S | ❌ | ❌ | ❌ | As above. | +| `terminal/kill` | S | ❌ | ❌ | ❌ | As above. | +| `terminal/release` | S | ❌ | ❌ | ❌ | As above. | +| `elicitation/create` · `elicitation/complete` | U | ❌ | ✅ | ⚠️ | Structured user-input forms. Claude calls the `unstable_*` elicitation methods (to surface MCP server elicitations); Codex does NOT — its `CodexElicitationHandler` maps elicitations onto `session/request_permission` instead. | + +## 3. Capabilities + +### 3a. `agentCapabilities` (advertised by the bridge) + +| Capability | Stable | Bridge | Claude | Codex | Notes | +|---|---|---|---|---|---| +| `loadSession` | S | ✅ | ✅ | ✅ | Advertised `true`; backs `session/load`. | +| `promptCapabilities.image` | S | ❌ | ✅ | ✅ | Bridge advertises `image: false`; image prompt blocks are rejected. | +| `promptCapabilities.audio` | S | ❌ | ❌ | ❌ | `audio: false`; neither adapter accepts audio either. | +| `promptCapabilities.embeddedContext` | S | ❌ | ✅ | ✅ | `embeddedContext: false`; embedded `resource` blocks rejected. | +| `mcpCapabilities.{http,sse}` | S | ❌ | ✅ | ⚠️ | No MCP passthrough; `mcpServers` is rejected. Claude advertises http+sse, Codex http only. | +| `sessionCapabilities.*` | S | ❌ | ✅ | ✅ | None advertised (list/delete/resume/close/additionalDirectories/fork all off). | +| `auth.logout` | S | ❌ | ✅ | ✅ | Not advertised. | +| `authMethods[]` | S | ⚠️ | ✅ | ✅ | Advertised as empty (no auth required to reach the model). | +| `agentInfo` (name/version) | S | ✅ | ✅ | ✅ | From `agentName` / `agentVersion` config. | +| `_meta` custom caps | S | ❌ | ✅ | — | E.g. Claude's `claudeCode.promptQueueing`. The bridge advertises no custom `_meta`. | + +### 3b. `clientCapabilities` (consumed by the bridge) + +| Capability | Stable | Bridge | Notes | +|---|---|---|---| +| `fs.{readTextFile,writeTextFile}` | S | ❌ | Not consulted (the bridge never calls `fs/*`). | +| `terminal` | S | ❌ | Not consulted; the bridge keys terminal rendering off the Zed `_meta.terminal_output` cap instead. | +| `_meta.terminal_output` (Zed) | S (`_meta`) | ✅ | Snapshotted per session at create/load; gates terminal-card rendering. | + +## 4. `session/update` variants + +| `sessionUpdate` | Stable | Bridge | Claude | Codex | Notes | +|---|---|---|---|---|---| +| `agent_message_chunk` | S | ✅ | ✅ | ✅ | From `assistant/chunk` text-delta. | +| `agent_thought_chunk` | S | ✅ | ✅ | ✅ | From `assistant/chunk` reasoning-delta. | +| `user_message_chunk` | S | ✅ | ✅ | ✅ | Emitted during `session/load` replay to reconstruct the user side. | +| `tool_call` | S | ✅ | ✅ | ✅ | Tool-owned presentation (`presentCall`); see [§5](#5-tool-call-rendering). | +| `tool_call_update` | S | ✅ | ✅ | ✅ | From `tool/result` via `presentResult`. | +| `plan` | S | ❌ | ✅ | ✅ | No agent plan emitted. Both adapters emit real plan entries (Codex's `CodexEventHandler.updatePlan` maps `turn/plan/updated` → `{ sessionUpdate: 'plan', entries }`). | +| `available_commands_update` | S | ❌ | ✅ | ✅ | No slash commands advertised. | +| `current_mode_update` | S | ❌ | ✅ | ✅ | No session modes. | +| `config_option_update` | S | ❌ | ✅ | ✅ | No config options. | +| `usage_update` | S | ❌ | ✅ | ✅ | Token/cost reporting not surfaced (the harness records token usage internally on `assistant/message`). | +| `session_info_update` | S | ❌ | ⚠️ | ⚠️ | Session title/metadata not pushed. | + +## 5. Tool-call rendering + +Tool-call presentation is **owned by each tool** (`presentCall` / `presentResult` on the `dsh-tools` definition), not special-cased in the bridge — see the [terminal-and-tool-rendering RFC](../../../docs/rfc/implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md). + +| Feature | Stable | Bridge | Claude | Codex | Notes | +|---|---|---|---|---|---| +| `ToolCallKind` mapping | S | ✅ | ✅ | ✅ | `execute`/`read`/`edit`/`other` inferred from the tool; richer mapping possible. | +| `ToolCallStatus` | S | ✅ | ✅ | ✅ | `in_progress` → `completed`/`failed`. | +| `content` blocks | S | ✅ | ✅ | ✅ | Text content; the description renders above the card. | +| `diff` content | S | ❌ | ✅ | ✅ | No structured diff rendering for edits (would need a diffing edit tool + presenter). | +| `terminal` content | S | ✅ | ✅ | ✅ | Via the Zed `_meta` terminal convention (see below), not the spec `terminal/*` sub-protocol. | +| `locations` (follow-along) | S | ❌ | ✅ | ✅ | No file-location hints emitted. | +| `rawInput` | S | ✅ | ⚠️ | ✅ | Parsed tool args surfaced as `rawInput`. | +| `rawOutput` | S | ❌ | ⚠️ | ✅ | Not emitted. | + +### Terminal rendering + +⚠️ Implemented via the **Zed `_meta` convention** (`terminal_info` / `terminal_output` / `terminal_exit`), gated on the client advertising `_meta.terminal_output` — NOT the spec's `terminal/create` sub-protocol (which would make the editor execute the command, bypassing `dsh-bash`'s sandbox / env-scrub / ownership / cwd). Both reference adapters take the same `_meta` approach. Live incremental streaming (`terminal_output_delta`, which Codex negotiates) is a follow-up — the bridge currently sends the full captured output once on the result. + +## 6. Session modes / config options / models + +❌ None modeled. Both reference adapters ship modes (Claude: a "plan" auto-mode; Codex: read-only / agent / agent-full-access mapping to its approval+sandbox policy), the newer config-option surface, and runtime model selection. The harness fixes the model per-bridge via `AcpConfig.model`. These are coupled to the unbuilt **permission gate** (a mode often selects an approval policy), so they are natural follow-ups to it. + +## 7. Content blocks + +| Block | Stable | In prompts | In updates | Notes | +|---|---|---|---|---| +| `text` | S | ✅ | ✅ | Baseline. | +| `resource_link` | S | ✅ | ⚠️ | Accepted in prompts and rendered into text (`acpPromptToText`); not emitted as a structured update block. | +| `image` | S | ❌ | ❌ | Rejected in prompts (`promptCapabilities.image: false`). | +| `audio` | S | ❌ | ❌ | Rejected. | +| `resource` (embedded) | S | ❌ | ❌ | Rejected (`embeddedContext: false`). | + +The bridge rejects unsupported prompt blocks rather than silently dropping them (`promptHasUnsupportedContent`), per the "explicit over implicit" convention. + +## 8. Cross-cutting + +| Feature | Stable | Bridge | Notes | +|---|---|---|---| +| `StopReason` mapping | S | ✅ | `turnEndToStopReason` is total over harness turn-end reasons → `end_turn`/`max_tokens`/`cancelled`. | +| Multi-session (N per connection) | S | ✅ | Strict per-session demux; concurrent streams never interleave. See the [multi-session RFC](../../../docs/rfc/proposed/feature/2026-06-14-acp-multi-session.md). | +| Disconnect / disposal teardown | S | ✅ | Quiesces every live session on client disconnect or Cordis disposal. | +| `_meta` extensibility | S | ⚠️ | Consumed (Zed terminal cap) and emitted (terminal `_meta`); no other custom extensions. | +| Background-task ownership isolation | — | ✅ | `bash_output`/`bash_kill` reject another session's task via an opaque owner token. | +| stdout-is-the-protocol guarantee | S | ✅ | The bridge runs in an example with no stdout logger. | + +## Gap summary + +Ranked by how commonly the reference adapters ship them and how much UX they unlock: + +1. **Permission gate** — `session/request_permission` + permission options. Tracked `TODO(rfc010-permission-gate)`; the reverse map is already wired. Foundational, and a prerequisite for modes. +2. **Session lifecycle** — `session/list` + `session/delete` (the persistence layer already lists), then `session/resume` / `session/close`. +3. **Modes / config options / model selection** — coupled to the permission gate. +4. **Agent plan** (`sessionUpdate: 'plan'`) — surface the loop's plan as structured entries. +5. **Slash commands** (`available_commands_update`). +6. **MCP passthrough** (`mcpServers` on `session/new` + `mcpCapabilities`). +7. **Richer prompt content** — image / embedded `resource` blocks (needs a multimodal model path). +8. **Diff + location tool rendering** — `diff` content and `locations` for edit tools. +9. **Usage reporting** (`usage_update`) — the harness already records token usage internally (on `assistant/message`). +10. **Editor filesystem delegation** (`fs/read_text_file` / `fs/write_text_file`) — lets the agent see unsaved buffers; lower priority since the harness has direct disk access. + +## Out of scope + +Unstable/draft ACP features that **neither** reference adapter ships are not tracked above: `providers/*` (LLM provider selection), `mcp/connect`·`mcp/message`·`mcp/disconnect` (client-side MCP passthrough), `nes/*` (Next Edit Suggestion), `document/did*` (LSP-style document sync), the v2 plan model (`plan_update` / `plan_removed`), boolean config options, `$/cancel_request`, and the draft Streamable-HTTP transport. They can be added if a target editor adopts them. + +## Sources + +- Stable spec: `schema/v1/schema.json` (schema `1.14.0`) and `docs/protocol/v1/*.mdx` in the [agent-client-protocol](https://github.com/agentclientprotocol/agent-client-protocol) repo. +- Reference adapters: [`claude-agent-acp`](https://github.com/zed-industries/claude-code-acp) and [`codex-acp`](https://github.com/zed-industries/codex-acp). +- Bridge: [`README.md`](README.md), [`src/index.ts`](src/index.ts), and the ACP RFCs under [`docs/rfc/`](../../../docs/rfc/README.md). diff --git a/packages/acp/package.json b/packages/ui/acp/package.json similarity index 90% rename from packages/acp/package.json rename to packages/ui/acp/package.json index 0a4a890658..6973dc5e20 100644 --- a/packages/acp/package.json +++ b/packages/ui/acp/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "license": "BSD-3-Clause", diff --git a/packages/acp/src/codec.ts b/packages/ui/acp/src/codec.ts similarity index 97% rename from packages/acp/src/codec.ts rename to packages/ui/acp/src/codec.ts index dd28bc5ae6..5f5a53f529 100644 --- a/packages/acp/src/codec.ts +++ b/packages/ui/acp/src/codec.ts @@ -26,7 +26,7 @@ import type { ContentBlock as AcpContentBlock, StopReason } from '@agentclientpr * * - `completed` → `end_turn` (the model chose to stop) * - `max-tokens` → `max_tokens` (cut off at the output-token ceiling) - * - `aborted` → `cancelled` (an `agent.abort()`, e.g. from `session/cancel`) + * - `aborted` → `cancelled` (a step abort or a queue-aware `agent.cancel()`, e.g. from `session/cancel`) * - `error` → `end_turn` (defensive fallback only: the bridge REJECTS the * `session/prompt` RPC on an error turn BEFORE calling this, so * a client sees a JSON-RPC error, not a stop reason — see diff --git a/packages/acp/src/index.ts b/packages/ui/acp/src/index.ts similarity index 87% rename from packages/acp/src/index.ts rename to packages/ui/acp/src/index.ts index 66be7458ec..ec79e97443 100644 --- a/packages/acp/src/index.ts +++ b/packages/ui/acp/src/index.ts @@ -13,10 +13,12 @@ * - `session/load` → `ctx.agents.resume(...)` then replay the event log * - `session/prompt` → `agent.send()`, settle on the owning turn's end (a turn * that ends in `error` rejects the RPC) - * - `session/cancel` → `agent.abort()` + settle the in-flight prompt + * - `session/cancel` → `agent.cancel()` (the queue-aware cancel: aborts a + * running step, clears queued + steering work, and drops a + * turn about to start) + settle the in-flight prompt * * Multi-session (RFC 011): N concurrent sessions per connection, each mapped to - * its own `LoopAgent`. Sessions are keyed by id in `sessions` (forward) with an + * its own `ReactLoopAgent`. Sessions are keyed by id in `sessions` (forward) with an * `agent→sessionId` reverse map for O(1) demux of `agent/*` events; every * `session/event` and `agent/*` event is routed strictly to its owning session * record, so two sessions streaming at once never interleave their @@ -58,7 +60,10 @@ import { type StopReason, } from '@agentclientprotocol/sdk' import type { ContentBlock } from '@deepseek-ai/dsh-llm' +import { CallId } from '@deepseek-ai/dsh-llm' import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent' +import { AgentId } from '@deepseek-ai/dsh-agent' +import { SessionId } from '@deepseek-ai/dsh-session' import type { SessionEvent, TurnEndReason } from '@deepseek-ai/dsh-session' import type { ToolCallKind, ToolCallPresentation, ToolRegistry, ToolResultPresentation, ToolTerminal } from '@deepseek-ai/dsh-tools' // Side-effect type import: declaration-merges `ctx.sessionPersistence` onto @@ -136,8 +141,15 @@ export const Config: Schema = Schema.object({ * map keyed by id (RFC 011 multi-session). */ interface SessionRecord { - sessionId: string + sessionId: SessionId agent: Agent + /** + * The owned-agent disposer (from the {@link AgentHandle} the factory returned). + * Teardown calls it to unregister this ONE agent, stop its loop, await + * quiescence, and remove its session — instead of leaving it for the bridge + * fiber to reclaim. + */ + dispose: () => Promise /** * Resolves tool-owned presentation for THIS session's tool calls and remembers * each in-flight call's `(name, args)` so the matching `tool/result` can find @@ -194,6 +206,10 @@ interface SessionRecord { * others are no-ops (settle-exactly-once). */ export function apply(ctx: Context, config: AcpConfig): void { + // TODO(double-default): these literals duplicate the Config schema defaults + // (`agentName`/`agentVersion` `.default(...)` above). The Loader applies the + // schema before apply() runs, so the `??` only fires for direct-apply unit + // tests. Pick one home for the default to avoid drift. const agentName = config.agentName ?? 'deepseek-harness-acp' const agentVersion = config.agentVersion ?? '0.0.1' @@ -216,12 +232,12 @@ export function apply(ctx: Context, config: AcpConfig): void { // reverse map so `agent/*` events (which carry only the Agent) demux in O(1). // The two stay in lockstep: a record is added to `sessions` and the agent to // `bySession` together, and removed together. - const sessions = new Map() - const bySession = new WeakMap() + const sessions = new Map() + const bySession = new WeakMap() // Session ids whose `session/load` is mid-`resume()` (the slot is reserved // before the async resume so a pipelined load/new for the SAME id can't create // two agents). Distinct ids load concurrently; a given id loads once at a time. - const loadingIds = new Set() + const loadingIds = new Set() // Set once the bridge has torn down (disposal or client disconnect). An async // `session/load` mid-`resume()` when teardown ran must observe this after its // await and NOT install a record (which would resurrect a live agent/listeners @@ -252,7 +268,7 @@ export function apply(ctx: Context, config: AcpConfig): void { } /** Resolve the live record for a sessionId, or throw an ACP error. */ - const requireSession = (sessionId: string): SessionRecord => { + const requireSession = (sessionId: SessionId): SessionRecord => { const rec = sessions.get(sessionId) if (rec === undefined) { throw invalidParams(`unknown session: ${sessionId}`) @@ -433,22 +449,32 @@ export function apply(ctx: Context, config: AcpConfig): void { assertOpen() validateWorkspaceParams(params) validateMcpServers(params) - const sessionId = randomUUID() - const agent = agents.create({ - agentId: sessionId, + const sessionId = SessionId(randomUUID()) + const handle = agents.create({ + agentId: AgentId(sessionId), sessionId, meta: { cwd: params.cwd }, agentOptions: agentOptions(config), }) - bySession.set(agent, sessionId) - sessions.set(sessionId, { sessionId, agent, presenter: makePresenter(), terminalEnabled: terminalOutputCap, inflight: undefined }) + bySession.set(handle.agent, sessionId) + sessions.set(sessionId, { + sessionId, + agent: handle.agent, + dispose: () => handle.dispose(), + presenter: makePresenter(), + terminalEnabled: terminalOutputCap, + inflight: undefined, + }) return Promise.resolve({ sessionId }) }, async loadSession(params: LoadSessionRequest): Promise { assertOpen() - if (sessions.has(params.sessionId) || loadingIds.has(params.sessionId)) { - throw invalidParams(`session ${params.sessionId} is already loaded`) + // The wire `params.sessionId` is a raw protocol string; brand it once at + // this entry so the session collections and the resume factory see a SessionId. + const sessionId = SessionId(params.sessionId) + if (sessions.has(sessionId) || loadingIds.has(sessionId)) { + throw invalidParams(`session ${sessionId} is already loaded`) } validateWorkspaceParams(params) validateMcpServers(params) @@ -457,12 +483,12 @@ export function apply(ctx: Context, config: AcpConfig): void { // resume() is pending, then both install a record and leak a second // agent. (Distinct ids load concurrently — the set is keyed by id.) The // slot is released in `finally` so a rejected load never wedges the id. - loadingIds.add(params.sessionId) + loadingIds.add(sessionId) try { // Validate the PERSISTED cwd BEFORE resuming — `list()` is a // metadata-only read (no full-log parse), so this rejects a session we // can't honor WITHOUT ever constructing/registering an agent (a - // post-resume reject would leak the registered agent — abort() does not + // post-resume reject would leak the registered agent — cancel() does not // unregister it — and wedge the id against re-load). The session's bash // workdir is derived from its persisted `header.cwd` and the request // `cwd` does NOT override it (resume takes no cwd), so a session with no @@ -471,44 +497,52 @@ export function apply(ctx: Context, config: AcpConfig): void { // always has a cwd (session/new requires it); reject the rest loudly. // (An id unknown to `list()` falls through to resume, which rejects with // the backend's not-found error.) - const meta = (await sessionPersistence.list()).find(m => m.id === params.sessionId) + const meta = (await sessionPersistence.list()).find(m => m.id === sessionId) if (meta !== undefined) { const persistedCwd = meta.cwd if (persistedCwd === undefined || !isAbsolute(persistedCwd)) { throw invalidParams( - `session ${params.sessionId} has no absolute persisted cwd; cannot determine its workspace (it predates per-session cwd, or was created without one)`, + `session ${sessionId} has no absolute persisted cwd; cannot determine its workspace (it predates per-session cwd, or was created without one)`, ) } if (!sameWorkspaceCwd(persistedCwd, params.cwd)) { - throw invalidParams(`session ${params.sessionId} cwd mismatch: persisted ${persistedCwd}, requested ${params.cwd}`) + throw invalidParams(`session ${sessionId} cwd mismatch: persisted ${persistedCwd}, requested ${params.cwd}`) } } - const agent = await agents.resume({ - agentId: params.sessionId, - resumeSessionId: params.sessionId, + const handle = await agents.resume({ + agentId: AgentId(sessionId), + resumeSessionId: sessionId, agentOptions: agentOptions(config), }) // The bridge may have torn down (disposal / client disconnect) while // resume() was pending. Its listeners are gone, so installing a record - // now would resurrect a live agent the bridge can no longer drive or - // tear down. Bail: the just-resumed agent is reclaimed with the host - // context (no per-agent disposer — TODO(rfc010-agent-disposal)). - /* v8 ignore next 3 -- the in-memory test transport rejects the in-flight + // now would resurrect a live agent the bridge can no longer drive. Bail — + // and tear down the just-resumed agent (unregister + stop + remove its + // session) before throwing, so it does not leak: it has no SessionRecord, + // so quiesce() would never see it. + /* v8 ignore next 4 -- the in-memory test transport rejects the in-flight session/load request the instant it closes (before this post-await code runs), so the guard can't be hit in tests; it protects the real stdio path, where a closed pipe need not reject a mid-flight handler. */ if (closed) { + await handle.dispose() throw invalidParams('connection closed during session/load') } - bySession.set(agent, params.sessionId) + const agent = handle.agent + bySession.set(agent, sessionId) // Snapshot the terminal capability ONCE for this session (used by both // the replay below and the post-load live stream) so a later // `initialize` can't desync the call/result of a tool card. const terminalEnabled = terminalOutputCap const record: SessionRecord = { - sessionId: params.sessionId, agent, presenter: makePresenter(), terminalEnabled, inflight: undefined, + sessionId, + agent, + dispose: () => handle.dispose(), + presenter: makePresenter(), + terminalEnabled, + inflight: undefined, } - sessions.set(params.sessionId, record) + sessions.set(sessionId, record) // Replay the persisted event log to the client as session/update. Use // the raw event log (NOT deriveMessages, which drops assistant/chunk // and trace events): RFC 010's load contract reconstructs the streamed @@ -528,17 +562,17 @@ export function apply(ctx: Context, config: AcpConfig): void { cwd: agent.session.header.cwd, } for (const event of agent.session.events) { - streamSessionEventUpdate(params.sessionId, event, notify, replayPresenter, replayTerminal) + streamSessionEventUpdate(sessionId, event, notify, replayPresenter, replayTerminal) } return {} } finally { - loadingIds.delete(params.sessionId) + loadingIds.delete(sessionId) } }, async prompt(params: PromptRequest): Promise { assertOpen() - const rec = requireSession(params.sessionId) + const rec = requireSession(SessionId(params.sessionId)) if (rec.inflight !== undefined) { throw invalidParams('a prompt is already in flight for this session') } @@ -567,25 +601,21 @@ export function apply(ctx: Context, config: AcpConfig): void { }, cancel(params: CancelNotification): Promise { - const rec = sessions.get(params.sessionId) + const rec = sessions.get(SessionId(params.sessionId)) if (rec === undefined) return Promise.resolve() - // RFC 010: session/cancel maps to agent.abort(reason). This aborts a - // RUNNING step (the turn ends 'aborted' → 'cancelled' via turn-end). - // It aborts and settles ONLY this session's agent/prompt — a cancel in - // one session never touches another's stream or pending prompt (RFC 011 - // isolation). It also settles the in-flight prompt as cancelled directly, - // in case the abort lands in the pre-step window (queued-but-not-started) - // where abort() has no AbortController to signal — see the README - // TODO(rfc010-cancel-prestep): a not-yet-started queued turn may still - // run to completion until a loop-level cancel lands. Best-effort abort - // plus honest RPC/UI cancellation. A secondary consequence of that same - // gap: because the loop batches all queued messages into one turn, a - // prompt accepted right after a pre-step cancel can be merged into the - // same turn as the cancelled one — that turn then carries both prompts' - // text and the new prompt settles for it. Both are closed by the same - // queue-aware loop cancel; the single-in-flight rule bounds the blast - // radius to one extra prompt. - rec.agent.abort('session/cancel') + // session/cancel maps to the queue-aware agent.cancel(reason): it aborts + // a RUNNING step, clears the queued + steering FIFOs, and drops a + // turn that is about to start (the pre-step window) — so a queued-but- + // not-yet-started prompt never runs, and a prompt accepted right after + // cannot be batched into the cancelled turn. Scoped to THIS session's + // agent — a cancel in one session never touches another's stream or + // pending prompt (RFC 011 isolation). We ALSO settle the in-flight prompt + // as cancelled directly here: do NOT rely on the resulting turn/end to + // settle it, because cancel() may drop the turn before any turn/end is + // emitted, and removing this direct settle would move the RPC's + // resolution onto the settleFromLog/agent-status path, changing its + // timing. + rec.agent.cancel('session/cancel') settlePrompt(rec, 'cancelled') return Promise.resolve() }, @@ -608,35 +638,29 @@ export function apply(ctx: Context, config: AcpConfig): void { /** * Tear ALL live sessions down to quiescence (AGENTS.md "dispose must reach - * quiescence"): for each session settle any pending prompt `cancelled`, abort - * the agent, and AWAIT it draining via the interface-level `whenIdle()` signal - * (NOT `agent/status('disposed')`, which fires before the driver exits). The - * agents drain in parallel. Idempotent — clears the `sessions` map first and - * memoizes, so a second call (close racing dispose) is a no-op. + * quiescence"): for each session settle any pending prompt `cancelled`, then + * run that session's {@link AgentHandle} `dispose()` — which stops the loop + * (sets `disposed`, aborts the in-flight step), AWAITS the loop's exit (the + * final `turn/end` + `session/flush` are captured while `onAppend` is still + * attached), unregisters the agent, and removes its session from the store. + * The per-session disposes run in parallel. Idempotent — clears the `sessions` + * map first and memoizes, so a second call (close racing dispose) is a no-op. * Shared by Cordis disposal AND client disconnect (`conn.closed`). * - * Caveat (same window as TODO(rfc010-cancel-prestep)): if teardown lands in - * the pre-step window — `agent.send()` queued a turn but the loop has not yet - * flipped to `running` — `abort()` has no live `AbortController` to signal and - * `whenIdle()` returns immediately (status is still `idle`), so that queued - * turn may still start and run after teardown returns. Reaching true - * quiescence in that window needs a queue-aware loop cancel primitive (a - * loop-level change); the single-in-flight-per-session rule bounds the worst - * case to one short queued turn per session. - * - * The agents are NOT individually disposed/unregistered here. The factory - * (`ctx.agents.create`/`resume`) registers each via `AgentLoop.start`'s - * `this.ctx.effect(...)`; because the factory is reached through this bridge's - * traceable service proxy, that effect's `this.ctx` is the CALLER context (the - * bridge fiber), so every registry entry is bound to the bridge fiber and is - * reclaimed when the bridge fiber disposes (whole-context dispose, or an - * ACP-only HMR `acpFiber.dispose()` — both unregister all the bridge's - * agents). What this teardown path handles is a bare client disconnect, which - * resolves `conn.closed` WITHOUT disposing the fiber: each live agent is - * idled+aborted here but stays in `ctx.agents` until the fiber is disposed. - * Since a reconnect spins up a fresh context, the lingering idle agents strand - * no work. A per-agent disposal seam (unregister on disconnect) is a follow-up - * (TODO(rfc010-agent-disposal)). + * Per-agent disposal closes the former pre-step best-effort window — but via + * the DISPOSED path, not `cancel()`: the start-disposer resolves `handle.disposed`, + * which wakes the parked loop, and `isDisposed()` breaks the loop before a + * queued-but-not-yet-running turn can start (a turn cut off mid-flight ends + * with reason `disposed`, not `aborted`). A bare client disconnect (resolves + * `conn.closed` WITHOUT disposing the fiber) thus leaves NO registered agent + * and NO session-store entry — not an idled-but-still-registered one. When the + * fiber IS disposed (whole-context or an ACP-only HMR + * `acpFiber.dispose()`), this same memoized teardown runs first; the factory's + * register+start+session effects are ALSO bound to the bridge fiber (the + * factory is reached through this bridge's traceable service proxy, so + * `AgentLoop.start`'s `this.ctx.effect(...)` binds to the CALLER context — the + * bridge fiber), so any agent this path did not reach is still reclaimed by + * fiber disposal. */ let quiescing: Promise | undefined const quiesce = (): Promise => { @@ -654,8 +678,12 @@ export function apply(ctx: Context, config: AcpConfig): void { quiescing = (async () => { await Promise.all(recs.map(async (rec) => { settlePrompt(rec, 'cancelled') - rec.agent.abort('disposed') - await rec.agent.whenIdle() + // Per-agent dispose (the AgentHandle disposer): unregister this agent, + // stop its loop (sets disposed + aborts the in-flight step), await + // quiescence (the loop exit + final flush), and remove its session — so + // a bare client disconnect leaves NO registered agent and NO + // session-store entry, not just an idled-but-still-registered one. + await rec.dispose() })) })() return quiescing @@ -747,11 +775,11 @@ function validateMcpServers(params: { mcpServers?: unknown[] }): void { * generic fallback (title = tool name, raw args as input) when no registry is * available (e.g. pure translator tests). * - * Other event types (turn/step boundaries, context/message, usage, …) produce + * Other event types (turn/step boundaries, context/message, …) produce * no client update. */ export function streamSessionEventUpdate( - sessionId: string, + sessionId: SessionId, event: SessionEvent, notify: (notification: SessionNotification) => void, presenter: Pick = nullToolPresenter, @@ -845,7 +873,7 @@ export function streamSessionEventUpdate( }) return } - // turn/step boundaries, context/message, steering, usage, error, + // turn/step boundaries, context/message, steering, // assistant/message — no direct ACP client update. default: return @@ -916,7 +944,7 @@ interface ResolvedResultPresentation { * stale entry's only cost is one map slot until the session ends. */ export class ToolPresenter { - private readonly pending = new Map() + private readonly pending = new Map() /** * @param tools the registry to resolve tool definitions by name. @@ -932,7 +960,7 @@ export class ToolPresenter { ) {} /** Pending-state presentation for a `tool/call`; remembers `(name, args)` for the matching result. */ - call(callId: string, name: string, argsJson: string): ResolvedCallPresentation { + call(callId: CallId, name: string, argsJson: string): ResolvedCallPresentation { const args = parseToolArguments(argsJson) let present: ToolCallPresentation | undefined try { @@ -964,7 +992,7 @@ export class ToolPresenter { } /** Completed-state presentation for a `tool/result`; consumes the remembered `(name, args)`. */ - result(callId: string, content: ContentBlock[], isError: boolean): ResolvedResultPresentation { + result(callId: CallId, content: ContentBlock[], isError: boolean): ResolvedResultPresentation { const call = this.pending.get(callId) this.pending.delete(callId) // No remembered call (unknown/late callId) → nothing to present from; raw content. diff --git a/packages/acp/tests/bridge.spec.ts b/packages/ui/acp/tests/bridge.spec.ts similarity index 95% rename from packages/acp/tests/bridge.spec.ts rename to packages/ui/acp/tests/bridge.spec.ts index dd10a88bcb..e9ebca8d62 100644 --- a/packages/acp/tests/bridge.spec.ts +++ b/packages/ui/acp/tests/bridge.spec.ts @@ -3,6 +3,7 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' +import { AgentId } from '@deepseek-ai/dsh-agent' import { makeBridgeHarness, textResponse, type BridgeHarness } from './harness.ts' /** @@ -61,8 +62,8 @@ describe('acp bridge', () => { expect(b.sessionId).toBeTruthy() expect(a.sessionId).not.toBe(b.sessionId) // Both agents are live and independently registered. - expect(harness.ctx.agents.get(a.sessionId)).toBeDefined() - expect(harness.ctx.agents.get(b.sessionId)).toBeDefined() + expect(harness.ctx.agents.get(AgentId(a.sessionId))).toBeDefined() + expect(harness.ctx.agents.get(AgentId(b.sessionId))).toBeDefined() }) it('rejects a non-absolute cwd but accepts any absolute cwd (per-session workspace)', async () => { @@ -77,7 +78,7 @@ describe('acp bridge', () => { const res = await harness.client.newSession({ cwd: '/tmp', mcpServers: [] }) expect(res.sessionId).toBeTruthy() // The session header records that cwd, so its bash tools run there. - expect(harness.ctx.agents.get(res.sessionId)!.session.header.cwd).toBe('/tmp') + expect(harness.ctx.agents.get(AgentId(res.sessionId))!.session.header.cwd).toBe('/tmp') }) it('rejects non-empty additionalDirectories', async () => { @@ -117,7 +118,7 @@ describe('acp bridge', () => { ], }) expect(result.stopReason).toBe('end_turn') - const user = harness.ctx.agents.get(sessionId)!.session.events.find(event => event.type === 'user/message') + const user = harness.ctx.agents.get(AgentId(sessionId))!.session.events.find(event => event.type === 'user/message') expect(JSON.stringify(user)).toContain('resource_link') }) diff --git a/packages/acp/tests/codec.spec.ts b/packages/ui/acp/tests/codec.spec.ts similarity index 97% rename from packages/acp/tests/codec.spec.ts rename to packages/ui/acp/tests/codec.spec.ts index 38a7a6cb41..9d82fe7533 100644 --- a/packages/acp/tests/codec.spec.ts +++ b/packages/ui/acp/tests/codec.spec.ts @@ -16,7 +16,7 @@ describe('turnEndToStopReason', () => { expect(turnEndToStopReason({ kind: 'max-tokens' })).toBe('max_tokens') expect(turnEndToStopReason({ kind: 'aborted', reason: 'x' })).toBe('cancelled') expect(turnEndToStopReason({ kind: 'disposed' })).toBe('cancelled') - expect(turnEndToStopReason({ kind: 'error', message: 'boom' })).toBe('end_turn') + expect(turnEndToStopReason({ kind: 'error', step: 1, message: 'boom' })).toBe('end_turn') }) it('falls back to end_turn for an unknown (merge-extensible) future kind', () => { diff --git a/packages/ui/acp/tests/dispose.spec.ts b/packages/ui/acp/tests/dispose.spec.ts new file mode 100644 index 0000000000..ac092d9d16 --- /dev/null +++ b/packages/ui/acp/tests/dispose.spec.ts @@ -0,0 +1,320 @@ +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' +import { SessionId } from '@deepseek-ai/dsh-session' +import { AgentId } from '@deepseek-ai/dsh-agent' +import { makeBridgeHarness, textResponse } from './harness.ts' + +describe('acp bridge — disposal & HMR safety', () => { + let storageDir: string + + beforeEach(async () => { storageDir = await mkdtemp(join(tmpdir(), 'acp-dispose-')) }) + afterEach(async () => { await rm(storageDir, { recursive: true, force: true }) }) + + it('disposal reaches quiescence: a running turn is aborted and awaited before dispose returns', async () => { + const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const agent = harness.ctx.agents.get(AgentId(sessionId))! + + // Start a prompt that hangs in the model stream. + const promptDone = harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }) + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('running') + + // Dispose the whole context. The bridge's teardown must abort the agent and + // AWAIT whenIdle() — so right after dispose resolves, the agent is settled + // (not still running). Proves disposal waited, not just requested. + await harness.ctx.fiber.dispose() + expect(agent.status).not.toBe('running') + + // The in-flight prompt settled (cancelled) rather than hanging forever. + const res = await promptDone + expect(res.stopReason).toBe('cancelled') + }) + + it('after an ACP-only HMR dispose, a late session/new creates no orphan agent (closed guard)', async () => { + // Dispose JUST the bridge's fiber (an HMR reload) while agents/agent-loop + // stay up and the transport is still live. A late session/new must hit the + // `closed` guard and reject — NOT create an agent the disposed bridge can no + // longer stream or settle. Verify the world: no agent appeared. + const harness = await makeBridgeHarness({ storageDir, script: [] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const before = harness.ctx.agents.list().length + await harness.acpFiber.dispose() // tear down ONLY the bridge + await expect(harness.client.newSession({ cwd: process.cwd(), mcpServers: [] })) + .rejects.toThrow(/disposed/) + expect(harness.ctx.agents.list().length).toBe(before) + await harness.dispose() + }) + + it('an agent created through the bridge is unregistered when ONLY the bridge fiber is disposed', async () => { + // The factory (`ctx.agents.create`) is reached through the bridge's + // traceable service proxy, so `AgentLoop.start`'s `this.ctx.effect(...)` + // registration binds to the CALLER context — the bridge fiber — not the + // AgentLoop fiber. Disposing JUST the bridge fiber (an ACP-only HMR reload) + // must therefore reclaim the agent's registry entry, even though agents/ + // agent-loop stay up. This pins the fiber-ownership the bridge's teardown + // doc comment relies on; if a refactor rebinds the registration to the + // AgentLoop fiber, the agent would survive bridge dispose and this fails. + const harness = await makeBridgeHarness({ storageDir, script: [] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + expect(harness.ctx.agents.get(AgentId(sessionId))).toBeDefined() + + await harness.acpFiber.dispose() // tear down ONLY the bridge + expect(harness.ctx.agents.get(AgentId(sessionId))).toBeUndefined() + await harness.dispose() + }) + + it('no agent is created by a session/new after the bridge has closed (closed guard)', async () => { + // After teardown (here a client disconnect sets `closed`), a late + // `session/new` must NOT create an orphan agent the bridge can no longer + // drive/settle. The transport is gone so the RPC rejects; assert the world: + // no new agent appeared in the registry. + const harness = await makeBridgeHarness({ storageDir, script: [] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const before = harness.ctx.agents.list().length + await harness.closeClientTransport() // teardown → closed = true + await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }).catch(() => {}) + await new Promise(r => setTimeout(r, 10)) + expect(harness.ctx.agents.list().length).toBe(before) + await harness.dispose() + }) + + it('a client disconnect mid-prompt disposes the session (no registered agent left)', async () => { + // The ACP transport closes (editor quits) while a turn runs. The bridge must + // settle the in-flight prompt cancelled and DISPOSE the agent (the session's + // per-agent AgentHandle teardown) rather than leaving an orphaned running — + // or even idled-but-still-registered — agent whose updates are swallowed. + const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const agent = harness.ctx.agents.get(AgentId(sessionId))! + // Start a prompt that hangs in the model stream. The prompt RPC will never + // return (its transport is severed), so do not await it. + void harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }).catch(() => {}) + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('running') + + // Sever the transport — the bridge's conn.closed teardown runs and drives the + // agent's AgentHandle dispose to quiescence on its OWN (before any dispose()). + await harness.closeClientTransport() + await agent.whenIdle() + // The agent's loop has stopped: status `disposed`. + expect(agent.status).toBe('disposed') + + // Await the bridge teardown to completion WITHOUT tearing down the root + // agents/sessions services (so we can still query them). acpFiber.dispose() + // invokes the SAME memoized quiesce() the disconnect started and awaits its + // promise — which resolves only after every rec.dispose() (loop exit + + // session removal) has finished, closing the whenIdle()/owned.dispose() + // microtask race. The AgentHandle dispose has run: the agent is unregistered + // and its session removed from the store, not merely idled (the old + // behavior). The services live on the root ctx, so they survive this. + await harness.acpFiber.dispose() + expect(harness.ctx.agents.get(AgentId(sessionId))).toBeUndefined() + expect(harness.ctx.sessions.get(SessionId(sessionId))).toBeUndefined() + await harness.dispose() + }) + + it('a client disconnect racing fiber dispose both reach quiescence (shared teardown)', async () => { + // conn.closed teardown and ctx.fiber.dispose() can fire near-simultaneously. + // They must share one teardown promise: dispose() must NOT return before the + // disconnect teardown's whenIdle() has settled (a `record === undefined`-only + // guard would let the second caller return early mid-teardown). + const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const agent = harness.ctx.agents.get(AgentId(sessionId))! + void harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }).catch(() => {}) + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('running') + + // Fire both teardown paths without awaiting the first, then await both. + const close = harness.closeClientTransport() + const dispose = harness.ctx.fiber.dispose() + await Promise.all([close, dispose]) + // After BOTH settle, the agent has fully drained (not still running). + expect(agent.status).not.toBe('running') + }) + + it('after dispose, session/update listeners are gone (no further updates emitted)', async () => { + const harness = await makeBridgeHarness({ storageDir, script: [] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const session = harness.ctx.agents.get(AgentId(sessionId))!.session + + await harness.ctx.fiber.dispose() + const before = harness.updates.length + // Append an event directly to the (now-detached) session: the bridge's + // session/event listener should have been disposed, so no update fires. + session.append('turn/start', { turn: 99, trigger: { kind: 'message', source: { kind: 'user' } } }) + await new Promise(r => setTimeout(r, 10)) + expect(harness.updates.length).toBe(before) + }) + + it('the final turn closing events are persisted across an AgentHandle dispose (durability)', async () => { + // The teardown-ORDER guarantee: a per-agent dispose must stop the loop, + // AWAIT its exit (so the loop's final `turn/end` + `session/flush` fire + // through the still-attached `session.onAppend` → `session/event`), and only + // THEN detach onAppend + remove the session. If the order were inverted + // (detach first), the closing events would never reach persistence. Drive a + // CLEAN turn to completion, dispose JUST the bridge, then re-load the + // persisted log from disk and assert the closing turn/end is on disk — the + // world, not the agent's self-report. + const harness = await makeBridgeHarness({ storageDir, script: [textResponse('done')] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + await harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }) + const liveEvents = harness.ctx.agents.get(AgentId(sessionId))!.session.events.length + expect(liveEvents).toBeGreaterThan(0) + + // Tear down JUST the bridge (the AgentHandle dispose runs to quiescence). + await harness.acpFiber.dispose() + expect(harness.ctx.agents.get(AgentId(sessionId))).toBeUndefined() + + // Re-load the session from disk: every live event (incl. the closing + // turn/end) was flushed before the session was detached. + const reloaded = await harness.ctx.sessionPersistence.load(SessionId(sessionId)) + expect(reloaded.events.length).toBe(liveEvents) + const last = reloaded.events.at(-1)! + expect(last.type).toBe('turn/end') + await harness.dispose() + }) + + it('a turn aborted BY the dispose still flushes its closing turn/end to disk (durability, mid-turn)', async () => { + // The teardown-order contract only earns its keep when the closing events are + // produced BY the dispose itself. Here the model stream HANGS, so the turn is + // still open when teardown runs: the composite agent effect stops the loop, + // the loop unwinds and appends `turn/end {disposed}` + runs its final + // `session/flush` — all while `onAppend` is still attached (the session + // detach is the LAST disposer in the same effect's LIFO chain) — and only + // THEN is the session detached. If the order were inverted (or the session + // were a racing SIBLING effect), the abort-produced `turn/end` would never + // reach disk and a re-load would instead show crash-recovery's synthetic + // `interrupted` closer. Re-load from disk and assert the REAL `disposed` + // reason landed — proving the loop's own closing event was captured, not a + // recovered substitute. + const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const agent = harness.ctx.agents.get(AgentId(sessionId))! + void harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }).catch(() => {}) + await new Promise(r => setTimeout(r, 30)) + expect(agent.status).toBe('running') + // The turn is OPEN in the log (turn/start appended, no turn/end yet). + const openTurnEnds = agent.session.events.filter(e => e.type === 'turn/end').length + + // Dispose JUST the bridge: a fiber unload that must STILL honor the ordered + // teardown (the composite effect runs its disposer chain as a unit). + await harness.acpFiber.dispose() + expect(harness.ctx.agents.get(AgentId(sessionId))).toBeUndefined() + + // The loop's own `turn/end {disposed}` is on disk (re-load: the world, not + // self-report) — NOT a crash-recovery `interrupted` substitute. + const reloaded = await harness.ctx.sessionPersistence.load(SessionId(sessionId)) + const persistedTurnEnds = reloaded.events.filter(e => e.type === 'turn/end') + expect(persistedTurnEnds.length).toBe(openTurnEnds + 1) + expect(persistedTurnEnds.at(-1)!.data.reason).toMatchObject({ kind: 'disposed' }) + await harness.dispose() + }) + + it('per-session AgentHandle dispose leaves sibling agents untouched', async () => { + // The factory returns a per-agent AgentHandle whose dispose() tears down + // EXACTLY that agent + its session — RFC 011 isolation. Create two agents + // directly through the registry factory (the same path the ACP bridge uses), + // dispose one handle, and assert the other survives, registered and + // queryable, with its session still in the store. + const harness = await makeBridgeHarness({ storageDir, script: [] }) + const handleA = harness.ctx.agents.create({ + agentId: AgentId('sib-a'), sessionId: SessionId('sib-a'), agentOptions: { model: 'mock' }, + }) + const handleB = harness.ctx.agents.create({ + agentId: AgentId('sib-b'), sessionId: SessionId('sib-b'), agentOptions: { model: 'mock' }, + }) + expect(harness.ctx.agents.get(AgentId('sib-a'))).toBe(handleA.agent) + expect(harness.ctx.agents.get(AgentId('sib-b'))).toBe(handleB.agent) + + await handleA.dispose() + // A is gone — unregistered AND its session removed from the store. + expect(harness.ctx.agents.get(AgentId('sib-a'))).toBeUndefined() + expect(harness.ctx.sessions.get(SessionId('sib-a'))).toBeUndefined() + expect(handleA.agent.status).toBe('disposed') + // B is wholly unaffected. + expect(harness.ctx.agents.get(AgentId('sib-b'))).toBe(handleB.agent) + expect(harness.ctx.sessions.get(SessionId('sib-b'))).toBeDefined() + expect(handleB.agent.status).not.toBe('disposed') + await harness.dispose() + }) + + it('a throwing agent/disposed listener does not prevent session removal (composite-effect containment)', async () => { + // The AgentHandle teardown folds session-detach, register, and loop-stop + // into ONE composite effect whose disposers run as a `.then()` chain. The + // register disposer emits `agent/disposed`; if a listener throws and the + // emit is UNCONTAINED, the rejected chain skips the LATER session-detach + // disposer — stranding the session in the store with `onAppend` attached (a + // leak AND a durability hole, since the new design relies on detach + // running). The emit must be contained. Register a throwing listener, drive + // a clean turn, dispose, and assert the session was STILL removed. + const harness = await makeBridgeHarness({ storageDir, script: [textResponse('ok')] }) + harness.ctx.on('agent/disposed', () => { throw new Error('boom disposed listener') }) + const handle = harness.ctx.agents.create({ + agentId: AgentId('guard-a'), sessionId: SessionId('guard-a'), agentOptions: { model: 'mock' }, + }) + handle.agent.send([{ type: 'text', text: 'go' }]) + await handle.agent.whenIdle() + expect(harness.ctx.sessions.get(SessionId('guard-a'))).toBeDefined() + + // Dispose: the throwing listener must NOT break the chain before detach. + await handle.dispose() + expect(harness.ctx.agents.get(AgentId('guard-a'))).toBeUndefined() + expect(harness.ctx.sessions.get(SessionId('guard-a'))).toBeUndefined() // detach still ran + await harness.dispose() + }) + + it('concurrent AgentHandle dispose() calls all await the SAME teardown (memoized)', async () => { + // The handle's dispose() must memoize: the underlying cordis effect disposer + // is single-shot, so a second dispose() while the first is mid-teardown would + // otherwise resolve IMMEDIATELY (effect epoch already cleared) — before the + // first call's await agent.done + final flush finished. Every caller must + // observe the same quiescence boundary. + const harness = await makeBridgeHarness({ storageDir, script: ['hang'] }) + const handle = harness.ctx.agents.create({ + agentId: AgentId('conc-a'), sessionId: SessionId('conc-a'), agentOptions: { model: 'mock' }, + }) + // Drive a turn that hangs in the model stream, so the loop is mid-turn when + // disposed — its exit runs a final session/flush we can gate to hold the + // teardown observably in-flight. + handle.agent.send([{ type: 'text', text: 'go' }]) + await new Promise(r => setTimeout(r, 30)) + expect(handle.agent.status).toBe('running') + let releaseFlush!: () => void + const flushGate = new Promise((resolve) => { releaseFlush = resolve }) + harness.ctx.on('session/flush', () => flushGate) + + // First dispose enters teardown (aborts the hanging step) and blocks in the + // gated final flush. + const first = handle.dispose() + let firstSettled = false + void first.then(() => { firstSettled = true }) + await new Promise(r => setTimeout(r, 20)) + expect(firstSettled).toBe(false) + + // Second dispose MUST await the same in-flight teardown, not resolve early. + const second = handle.dispose() + let secondSettled = false + void second.then(() => { secondSettled = true }) + await new Promise(r => setTimeout(r, 20)) + expect(secondSettled).toBe(false) // memoized: still pending with the first + + // Release the flush; both resolve together and the session is gone. + releaseFlush() + await Promise.all([first, second]) + expect(harness.ctx.agents.get(AgentId('conc-a'))).toBeUndefined() + expect(harness.ctx.sessions.get(SessionId('conc-a'))).toBeUndefined() + await harness.dispose() + }) +}) diff --git a/packages/acp/tests/edges.spec.ts b/packages/ui/acp/tests/edges.spec.ts similarity index 92% rename from packages/acp/tests/edges.spec.ts rename to packages/ui/acp/tests/edges.spec.ts index 9484368322..69c935139d 100644 --- a/packages/acp/tests/edges.spec.ts +++ b/packages/ui/acp/tests/edges.spec.ts @@ -3,6 +3,8 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' +import { AgentId } from '@deepseek-ai/dsh-agent' +import { SessionId } from '@deepseek-ai/dsh-session' import { makeBridgeHarness, textResponse, type BridgeHarness } from './harness.ts' describe('acp bridge — demux & config edges', () => { @@ -25,7 +27,7 @@ describe('acp bridge — demux & config edges', () => { await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) const before = harness.updates.length - const foreign = harness.ctx.agents.create({ agentId: 'foreign', sessionId: 'foreign-session', agentOptions: { model: 'mock' } }) + const { agent: foreign } = harness.ctx.agents.create({ agentId: AgentId('foreign'), sessionId: SessionId('foreign-session'), agentOptions: { model: 'mock' } }) foreign.send([{ type: 'text', text: 'hi' }]) await foreign.whenIdle() await new Promise(r => setTimeout(r, 10)) diff --git a/packages/acp/tests/harness.ts b/packages/ui/acp/tests/harness.ts similarity index 100% rename from packages/acp/tests/harness.ts rename to packages/ui/acp/tests/harness.ts diff --git a/packages/acp/tests/load.spec.ts b/packages/ui/acp/tests/load.spec.ts similarity index 95% rename from packages/acp/tests/load.spec.ts rename to packages/ui/acp/tests/load.spec.ts index f06c707d85..a87fa49a9e 100644 --- a/packages/acp/tests/load.spec.ts +++ b/packages/ui/acp/tests/load.spec.ts @@ -3,7 +3,8 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' -import { SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' +import { AgentId } from '@deepseek-ai/dsh-agent' import { makeBridgeHarness, textResponse, toolCallResponse, type BridgeHarness, type CapturedUpdate } from './harness.ts' /** Concatenate the text of all agent_message_chunk updates. */ @@ -155,7 +156,7 @@ describe('acp bridge — session/load replay', () => { release() // resume() finishes AFTER teardown expect(await loadResult).toBe('rejected') // No live agent was installed for the closed connection. - expect(loader.ctx.agents.get(sessionId)).toBeUndefined() + expect(loader.ctx.agents.get(AgentId(sessionId))).toBeUndefined() }) it('rejects load when the requested cwd does not match the persisted session cwd', async () => { @@ -166,7 +167,7 @@ describe('acp bridge — session/load replay', () => { loader = await makeBridgeHarness({ storageDir, script: [] }) const otherCwd = '/some/other/workspace' await loader.ctx.sessionPersistence.create({ - version: 1, id: SessionId('elsewhere'), createdAt: 1, cwd: otherCwd, updatedAt: 1, + version: SESSION_FORMAT_VERSION, id: SessionId('elsewhere'), createdAt: 1, cwd: otherCwd, }) await loader.ctx.sessionPersistence.append(SessionId('elsewhere'), [ { type: 'turn/start', seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, @@ -176,11 +177,11 @@ describe('acp bridge — session/load replay', () => { await loader.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) await expect(loader.client.loadSession({ sessionId: 'elsewhere', cwd: process.cwd(), mcpServers: [] })) .rejects.toThrow(/cwd mismatch/) - expect(loader.ctx.agents.get('elsewhere')).toBeUndefined() + expect(loader.ctx.agents.get(AgentId('elsewhere'))).toBeUndefined() const res = await loader.client.loadSession({ sessionId: 'elsewhere', cwd: `${otherCwd}/.`, mcpServers: [] }) expect(res).toBeDefined() - expect(loader.ctx.agents.get('elsewhere')!.session.header.cwd).toBe(otherCwd) + expect(loader.ctx.agents.get(AgentId('elsewhere'))!.session.header.cwd).toBe(otherCwd) }) it('rejects load for a non-absolute cwd (still required to be absolute)', async () => { @@ -203,7 +204,7 @@ describe('acp bridge — session/load replay', () => { // to the server's launch dir (the request cwd does not override the header). loader = await makeBridgeHarness({ storageDir, script: [] }) await loader.ctx.sessionPersistence.create({ - version: 1, id: SessionId('legacy'), createdAt: 1, updatedAt: 1, // no cwd + version: SESSION_FORMAT_VERSION, id: SessionId('legacy'), createdAt: 1, // no cwd }) await loader.ctx.sessionPersistence.append(SessionId('legacy'), [ { type: 'turn/start', seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, @@ -215,7 +216,7 @@ describe('acp bridge — session/load replay', () => { // Rejected BEFORE resume (metadata-only check) — no agent was registered, so // the id is not wedged: a later attempt hits the same clean rejection, not a // duplicate-registration error. - expect(loader.ctx.agents.get('legacy')).toBeUndefined() + expect(loader.ctx.agents.get(AgentId('legacy'))).toBeUndefined() await expect(loader.client.loadSession({ sessionId: 'legacy', cwd: process.cwd(), mcpServers: [] })) .rejects.toThrow(/no absolute persisted cwd/) }) diff --git a/packages/acp/tests/multi-session.spec.ts b/packages/ui/acp/tests/multi-session.spec.ts similarity index 97% rename from packages/acp/tests/multi-session.spec.ts rename to packages/ui/acp/tests/multi-session.spec.ts index 1c20d2ba39..ca11934046 100644 --- a/packages/acp/tests/multi-session.spec.ts +++ b/packages/ui/acp/tests/multi-session.spec.ts @@ -3,6 +3,7 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' +import { AgentId } from '@deepseek-ai/dsh-agent' import { makeBridgeHarness, textResponse, type BridgeHarness, type CapturedUpdate } from './harness.ts' /** Text of the agent_message_chunk updates scoped to one session id. */ @@ -101,8 +102,8 @@ describe('acp bridge — RFC 011 multi-session isolation', () => { await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) const a = (await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] })).sessionId const b = (await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] })).sessionId - const agentA = harness.ctx.agents.get(a)! - const agentB = harness.ctx.agents.get(b)! + const agentA = harness.ctx.agents.get(AgentId(a))! + const agentB = harness.ctx.agents.get(AgentId(b))! // Wait deterministically for BOTH agents to enter `running` (not a fixed // sleep — agent startup latency is unbounded on a loaded worker). diff --git a/packages/acp/tests/properties.spec.ts b/packages/ui/acp/tests/properties.spec.ts similarity index 96% rename from packages/acp/tests/properties.spec.ts rename to packages/ui/acp/tests/properties.spec.ts index 5364c02d7b..3dcb4c760f 100644 --- a/packages/acp/tests/properties.spec.ts +++ b/packages/ui/acp/tests/properties.spec.ts @@ -17,7 +17,7 @@ import { describe, expect, it } from 'vitest' import fc from 'fast-check' import { CallId } from '@deepseek-ai/dsh-llm' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { SessionId, type SessionEvent } from '@deepseek-ai/dsh-session' import type { SessionNotification } from '@agentclientprotocol/sdk' import { streamSessionEventUpdate } from '../src/index.ts' @@ -85,7 +85,7 @@ function actionsToEvents(actions: Action[]): SessionEvent[] { function runStream(events: SessionEvent[]): SessionNotification['update'][] { const out: SessionNotification['update'][] = [] - for (const event of events) streamSessionEventUpdate('s1', event, n => out.push(n.update)) + for (const event of events) streamSessionEventUpdate(SessionId('s1'), event, n => out.push(n.update)) return out } diff --git a/packages/acp/tests/stream-update.spec.ts b/packages/ui/acp/tests/stream-update.spec.ts similarity index 97% rename from packages/acp/tests/stream-update.spec.ts rename to packages/ui/acp/tests/stream-update.spec.ts index cdba5a3bf3..30cbd17c40 100644 --- a/packages/acp/tests/stream-update.spec.ts +++ b/packages/ui/acp/tests/stream-update.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' import { CallId } from '@deepseek-ai/dsh-llm' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { SessionId, type SessionEvent } from '@deepseek-ai/dsh-session' import type { SessionNotification } from '@agentclientprotocol/sdk' import type { ToolDefinition, ToolRegistry } from '@deepseek-ai/dsh-tools' import { streamSessionEventUpdate, agentOptions, ToolPresenter } from '../src/index.ts' @@ -8,14 +8,14 @@ import { streamSessionEventUpdate, agentOptions, ToolPresenter } from '../src/in /** Collect the updates a single event produces (no presenter → generic fallback). */ function updatesFor(event: SessionEvent): SessionNotification['update'][] { const out: SessionNotification['update'][] = [] - streamSessionEventUpdate('s1', event, n => out.push(n.update)) + streamSessionEventUpdate(SessionId('s1'), event, n => out.push(n.update)) return out } /** Collect the updates emitted by the live prompt stream (user echo suppressed). */ function liveUpdatesFor(event: SessionEvent): SessionNotification['update'][] { const out: SessionNotification['update'][] = [] - streamSessionEventUpdate('s1', event, n => out.push(n.update), undefined, undefined, { includeUserMessages: false }) + streamSessionEventUpdate(SessionId('s1'), event, n => out.push(n.update), undefined, undefined, { includeUserMessages: false }) return out } @@ -116,7 +116,7 @@ describe('streamSessionEventUpdate', () => { it('produces no update for boundary/other event types', () => { expect(updatesFor(evt('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }))).toEqual([]) expect(updatesFor(evt('turn/end', { turn: 1, reason: { kind: 'completed' } }))).toEqual([]) - expect(updatesFor(evt('usage', { turn: 1, step: 1, usage: { inputTokens: 1, outputTokens: 1 } }))).toEqual([]) + expect(updatesFor(evt('step/start', { turn: 1, step: 1 }))).toEqual([]) }) }) @@ -138,7 +138,7 @@ describe('ToolPresenter (tool-owned presentation via the tool registry)', () => function updatesWith(presenter: ToolPresenter, ...events: SessionEvent[]): SessionNotification['update'][] { const out: SessionNotification['update'][] = [] - for (const event of events) streamSessionEventUpdate('s1', event, n => out.push(n.update), presenter) + for (const event of events) streamSessionEventUpdate(SessionId('s1'), event, n => out.push(n.update), presenter) return out } @@ -324,7 +324,7 @@ describe('terminal-card mapping (capability-gated)', () => { function termUpdates(tool: ToolDefinition, enabled: boolean, cwd: string | undefined, ...events: SessionEvent[]): SessionNotification['update'][] { const presenter = new ToolPresenter(registryOf(tool)) const out: SessionNotification['update'][] = [] - for (const event of events) streamSessionEventUpdate('s1', event, n => out.push(n.update), presenter, { enabled, cwd }) + for (const event of events) streamSessionEventUpdate(SessionId('s1'), event, n => out.push(n.update), presenter, { enabled, cwd }) return out } diff --git a/packages/acp/tests/turns.spec.ts b/packages/ui/acp/tests/turns.spec.ts similarity index 85% rename from packages/acp/tests/turns.spec.ts rename to packages/ui/acp/tests/turns.spec.ts index 19c3be2556..7634602a35 100644 --- a/packages/acp/tests/turns.spec.ts +++ b/packages/ui/acp/tests/turns.spec.ts @@ -3,6 +3,7 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { defineTool } from '@deepseek-ai/dsh-tools' +import { AgentId } from '@deepseek-ai/dsh-agent' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' import { errorResponse, @@ -283,7 +284,7 @@ describe('acp bridge — turn outcomes', () => { // OWN turn with the real model answer. harness = await makeBridgeHarness({ storageDir, script: [textResponse('real answer')] }) const sessionId = await newSession(harness) - const agent = harness.ctx.agents.get(sessionId)! + const agent = harness.ctx.agents.get(AgentId(sessionId))! // On the queued prompt, synchronously inject a one-shot context turn (idle // inject writes turn/start{injection} → context/message → turn/end). Fire // once so it lands between install and the prompt turn. @@ -327,21 +328,64 @@ describe('acp bridge — turn outcomes', () => { expect(res.stopReason).toBe('cancelled') }) - it('cancel in the pre-step window still settles the prompt cancelled exactly once', async () => { - // No script entry is consumed before cancel: cancel immediately after the - // prompt is sent, before the model step starts. The prompt must still - // settle cancelled (best-effort abort + settle), not hang. - harness = await makeBridgeHarness({ storageDir, script: [textResponse('late')] }) + it('cancel right after prompt settles cancelled and leaves the agent idle, no leaked turn', async () => { + // Over the async JSON-RPC transport the loop usually wakes before cancel + // arrives, so this is a running/mid-step cancel (the synchronous pre-step + // DROP is unit-tested in agent-loop/cancel.spec.ts). The ACP-level guarantee: + // the prompt settles cancelled, the agent reaches idle, and no second/leaked + // turn runs afterward. + harness = await makeBridgeHarness({ storageDir, script: [textResponse('answer'), textResponse('leaked')] }) const sessionId = await newSession(harness) const promptDone = harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }) await harness.client.cancel({ sessionId }) const res = await promptDone expect(res.stopReason).toBe('cancelled') - // The queued turn may still start after the cancel cleared the in-flight - // slot (the documented TODO(rfc010-cancel-prestep) best-effort window): its - // turn-start then fires with no prompt to tag, and the bridge does nothing. - // Let it run to completion and assert nothing re-settles (no throw, no hang). - await harness.ctx.agents.get(sessionId)!.whenIdle() + const agent = harness.ctx.agents.get(AgentId(sessionId))! + await agent.whenIdle() + // At most ONE turn ran (the cancelled one) — the cancel cleared the queue, so + // no second turn was batched or leaked. (A best-effort abort that left queued + // work could have started a second turn.) + const turnStarts = agent.session.events.filter(e => e.type === 'turn/start').length + expect(turnStarts).toBeLessThanOrEqual(1) + }) + + it('idle session/cancel then session/prompt runs the prompt (no intervening whenIdle)', async () => { + // The ACP bridge settles the cancel RPC synchronously and accepts the next + // prompt WITHOUT awaiting quiescence — so this drives cancel→prompt with NO + // whenIdle() between, the production race. An idle cancel must be a no-op that + // does NOT drop the following prompt. + harness = await makeBridgeHarness({ storageDir, script: [textResponse('real answer')] }) + const sessionId = await newSession(harness) + // Cancel while idle (no prompt in flight) — a no-op. + await harness.client.cancel({ sessionId }) + // Immediately prompt, no whenIdle() between. + const res = await harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }) + expect(res.stopReason).toBe('end_turn') + const text = harness.updates + .filter(u => u.sessionUpdate === 'agent_message_chunk') + .map(u => (u.content.type === 'text' ? u.content.text : '')) + .join('') + expect(text).toContain('real answer') + }) + + it('mid-stream cancel then an IMMEDIATE next prompt runs (no intervening whenIdle)', async () => { + // Cancel a running turn, then send the next prompt WITHOUT awaiting quiescence + // (the synchronous-settle path). The new prompt must run — the cancel marker + // must not leak onto it. + harness = await makeBridgeHarness({ storageDir, script: ['hang', textResponse('next answer')] }) + const sessionId = await newSession(harness) + const a = harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'A' }] }) + await new Promise(r => setTimeout(r, 30)) + await harness.client.cancel({ sessionId }) + expect((await a).stopReason).toBe('cancelled') + // Immediately — no whenIdle() — send the next prompt. + const b = await harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'B' }] }) + expect(b.stopReason).toBe('end_turn') + const text = harness.updates + .filter(u => u.sessionUpdate === 'agent_message_chunk') + .map(u => (u.content.type === 'text' ? u.content.text : '')) + .join('') + expect(text).toContain('next answer') }) it('a cancelled turn\'s late turn/end does not settle the NEXT prompt', async () => { diff --git a/packages/ui/acp/tsconfig.json b/packages/ui/acp/tsconfig.json new file mode 100644 index 0000000000..5989363d7f --- /dev/null +++ b/packages/ui/acp/tsconfig.json @@ -0,0 +1,36 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../core/tools" + }, + { + "path": "../../session-persistence/session-persistence" + } + ] +} diff --git a/packages/ui/stdio-agent/README.md b/packages/ui/stdio-agent/README.md new file mode 100644 index 0000000000..a78df0fa72 --- /dev/null +++ b/packages/ui/stdio-agent/README.md @@ -0,0 +1,60 @@ +# @deepseek-ai/dsh-stdio-agent + +The **terminal stdio chat app**: a Cordis app plugin that composes the providerless agent spine ([`@deepseek-ai/dsh-agent-core`](../../core/agent-core/README.md)) with the front-door cluster a terminal chat needs, and a `bin` that boots a leaf `cordis.yml`. + +It is the readline counterpart to [`@deepseek-ai/dsh-acp-agent`](../acp-agent/README.md): both consume the same spine, but each bakes in the OPPOSITE front-door cluster. + +## What it bakes in + +A terminal chat always wants the same cluster, so the package owns it rather than trusting each leaf to re-wire it: + +| Plugin | Why it is here | +|---|---| +| `@cordisjs/plugin-logger-console` | the console logger — stdout is just the terminal here, so logging to it is correct (the ACP app must NOT have this) | +| `@deepseek-ai/dsh-agent-core` | the spine, pre-creating a `main` agent from this app's `model`/`systemPrompt` | +| `@deepseek-ai/dsh-session-persistence-jsonl` | durable JSONL session log under `persistenceRoot` | +| `@deepseek-ai/dsh-ui-stdio` | the readline UI, bound to the `main` agent | + +`@cordisjs/plugin-hmr` (the dev/demo edit-reload loop) is deliberately a **leaf** entry, NOT baked in here: it is a Loader-only, subprocess-only dev plugin — its constructor throws without `node --expose-internals` + a live `loader`, and the in-process test tier cannot even import it (so a package whose `apply` statically pulled it in could never carry the per-file coverage gate). Unlike the console logger, a stray `hmr` is not a stdout-purity footgun, so leaving it at the leaf costs no safety. The `demo:echo` / `demo:coding` leaves load it and pass `--expose-internals`. + +The leaf `cordis.yml` supplies only the **swappable backends** — an LLM adapter (`llm-deepseek` for the real model, or the mock `mock-llm` for a demo) and a bash executor (`bash-local`) — `hmr`, plus this app's [`Config`](#config). The whole plugin tree a run loads is therefore: this app's cluster, the spine inside `agent-core`, `hmr`, and the two leaf backends. + +## Config + +| Key | Default | Routed to | +|---|---|---| +| `model` | (required) | the pre-created `main` agent's model | +| `systemPrompt` | (required) | the `main` agent's system prompt | +| `persistenceRoot` | `./.sessions` | the JSONL backend's root directory | +| `welcome` | `ready.` | the stdin-chat banner | +| `resumeSessionId` | — | resume a persisted session id instead of starting fresh (sourced from an env var in the leaf) | + +## The bin + +`dsh-stdio-agent [path-to-cordis.yml]` (default `./cordis.yml`) loads a gitignored `.env` from the cwd (`DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL`), then drives the cordis Loader against the config and awaits the whole plugin tree before returning. Run it under `node --expose-internals`: the cordis Loader resolves the config's bare plugin specifiers (`@deepseek-ai/dsh-*`, npm packages) through its internal module loader, which is only active under that flag. The `demo:echo` / `demo:coding` scripts invoke it that way. + +## Example leaf `cordis.yml` + +```yaml +# A real coding agent: hmr + the DeepSeek adapter + local bash, then this app. +- id: hmr + name: '@cordisjs/plugin-hmr' + config: + root: ['.'] +- id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' + config: + apiKey: !!js process.env.DEEPSEEK_API_KEY + models: [deepseek-v4-flash] +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + timeoutMs: 60000 +- id: stdio-agent + name: '@deepseek-ai/dsh-stdio-agent' + config: + model: deepseek-v4-flash + systemPrompt: 'You are a CLI coding assistant. Your only tools are bash…' +``` + +Swap `llm-deepseek` for a `mock-llm` leaf plugin and you have the echo demo — "swap the backend, keep the app". diff --git a/packages/ui/stdio-agent/package.json b/packages/ui/stdio-agent/package.json new file mode 100644 index 0000000000..bc9c98a411 --- /dev/null +++ b/packages/ui/stdio-agent/package.json @@ -0,0 +1,56 @@ +{ + "name": "@deepseek-ai/dsh-stdio-agent", + "description": "Terminal stdio chat app: the agent-core spine + console logger + readline UI + a pre-created main agent, with a bin to boot a leaf cordis.yml", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "bin": { + "dsh-stdio-agent": "lib/bin.js" + }, + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./bin": { + "types": "./lib/types/bin.d.ts", + "default": "./lib/bin.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/bin.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@cordisjs/plugin-include": "^1.0.4", + "@cordisjs/plugin-loader": "^1.0.0-rc.4", + "@cordisjs/plugin-logger-console": "^1.0.0", + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-agent-core": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "@deepseek-ai/dsh-session-persistence-jsonl": "^0.0.1", + "@deepseek-ai/dsh-ui-stdio": "^0.0.1", + "cordis": "^4.0.0-rc.6", + "schemastery": "^3.17.0" + }, + "devDependencies": { + "@cordisjs/plugin-include": "workspace:^", + "@cordisjs/plugin-loader": "workspace:^", + "@cordisjs/plugin-logger-console": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-core": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", + "@deepseek-ai/dsh-ui-stdio": "workspace:^", + "cordis": "^4.0.0-rc.6", + "schemastery": "^3.17.0" + } +} diff --git a/packages/ui/stdio-agent/src/bin.ts b/packages/ui/stdio-agent/src/bin.ts new file mode 100644 index 0000000000..92cd9f5e90 --- /dev/null +++ b/packages/ui/stdio-agent/src/bin.ts @@ -0,0 +1,140 @@ +#!/usr/bin/env node +/** + * The `dsh-stdio-agent` bin: boot a Cordis app from a leaf `cordis.yml` that + * loads the {@link @deepseek-ai/dsh-stdio-agent} app plugin (plus a backend LLM + * adapter and a bash executor). Owns the boot glue the three `examples/*` once + * duplicated in their `start.ts`: load the gitignored repo-root `.env`, then + * drive the cordis Loader against the config path (default `./cordis.yml`). + * + * Usage: `dsh-stdio-agent [path-to-cordis.yml]`. The `demo:echo` / `demo:coding` + * scripts invoke it with the example's config. + * + * @module @deepseek-ai/dsh-stdio-agent/bin + */ + +import { pathToFileURL } from 'node:url' +import { dirname, resolve } from 'node:path' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' + +/** + * Load `DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL` from a gitignored `.env` in the + * CURRENT WORKING DIRECTORY (Node native `process.loadEnvFile`). An absent file + * is fine — the environment may already carry the variables; the leaf + * `cordis.yml` reads them via the `!!js` tag. A present-but-unreadable/malformed + * `.env` is a real misconfiguration: surface it on stderr rather than silently + * running with the wrong environment. The mock-model demo (echo) ships no key + * and simply has no `.env`. + */ +function loadEnv(): void { + try { + process.loadEnvFile(resolve(process.cwd(), '.env')) + } catch (error) { + if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { + process.stderr.write(`dsh-stdio-agent: failed to load .env: ${String(error)}\n`) + } + // ENOENT (no .env) is fine — rely on the ambient environment. + } +} + +/** + * Make a load failure fail loud with a clear message on stderr. Covers the + * failure path the entry-tree check below cannot: when the include's + * `[Service.init]` throws (e.g. a config FILE that does not exist in a real + * directory), the cordis Loader surfaces it as an unhandled promise rejection + * AFTER `boot()` has resolved — `loader.await()` does NOT rethrow it, because + * `EntryTree.await()` uses `Promise.allSettled`, which swallows rejections. + * Node's default handler already exits non-zero on an unhandled rejection, so + * this does not change the exit code; it replaces Node's noisy stack dump with a + * single labelled line and guarantees `process.exit(1)`. Install before `boot()`. + */ +export function installFailLoud(): void { + process.on('unhandledRejection', (err: unknown) => { + process.stderr.write(`dsh-stdio-agent: fatal load failure: ${err instanceof Error ? err.stack ?? err.message : String(err)}\n`) + process.exit(1) + }) +} + +/** + * After the tree settles, assert every loader entry actually started. This is + * the load-bearing guard against the SILENT-exit-0 bug: when a plugin module + * fails to IMPORT (e.g. a config path in a non-existent directory, so the include + * plugin itself cannot be resolved), the cordis Loader catches the import error + * and only LOGS it (`entry._init`), leaving the entry with no `fiber` and + * producing no rejection — so the process would otherwise exit 0 with a usable + * config typo reported only as a log line. A started entry has a `fiber`; an + * entry with `fiber === undefined` after the tree settled never loaded. Throw on + * any such entry so `boot()` rejects (and the top-level `await` fails the process + * non-zero) instead of returning a half-empty context. + * + * A `disabled` entry is the one legitimate fiber-less state: `Entry.refresh()` + * deliberately skips `init()` for it, so it settles without a fiber by design. + * That is a valid config (a consumer turning an optional plugin off), not a + * failed import — exclude it so the guard catches only real load failures. + */ +function assertEntriesLoaded(ctx: Context): void { + const failed = [...ctx.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled) + if (failed.length > 0) { + const names = failed.map(entry => entry.options.name).join(', ') + throw new Error(`dsh-stdio-agent: plugin(s) failed to load: ${names} (see the error(s) logged above)`) + } +} + +/** + * Boot the Loader against `configPath` (resolved from the CWD). The include is + * handed the config's ABSOLUTE `file://` URL as its `path`, so resolution never + * depends on `ctx.baseUrl` (an absolute URL ignores the base) and can never fall + * back to the cwd. `baseUrl` is still pinned to the config's directory so the + * config's OWN relative plugin/include paths (e.g. `./src/mock-llm.ts`) resolve + * against it. Returns the root context once the whole tree has settled. + * + * The `await ctx.loader.await()` is load-bearing: `loader.create()` returns once + * the include ENTRY is registered, but the include then loads its child plugins + * asynchronously. Without awaiting the tree, `boot()` (and `main()`) would + * resolve while the app plugins — the stdin reader, the agent loop — are still + * mounting, and a CLI process with no attached handles yet exits 0 silently. + * Awaiting the tree keeps the process alive until the app's handles are attached. + * + * `loader.await()` does NOT, however, rethrow load errors (`EntryTree.await()` + * uses `Promise.allSettled`), so failures are surfaced two ways: a plugin that + * fails to IMPORT leaves an entry with no fiber, caught here by + * {@link assertEntriesLoaded} (this `boot()` rejects); a plugin whose init + * THROWS surfaces as an unhandled rejection caught by {@link installFailLoud} + * (installed by `main()` before this runs). Together they make any load failure + * exit non-zero with a clear message. + * + * Bare plugin specifiers in the config (`@deepseek-ai/dsh-*`, npm packages) are + * resolved by the cordis Loader's internal module loader, which is only active + * under `node --expose-internals` (the flag the `demo:echo`/`demo:coding` scripts + * pass). Without it the Loader falls back to resolving relative to its own module + * and cannot find the config's plugins, so a consumer running the built bin must + * pass `--expose-internals` (or install the plugins where node hoists them). + */ +export async function boot(configPath: string): Promise { + const absolute = resolve(process.cwd(), configPath) + const ctx = new Context() + ctx.baseUrl = pathToFileURL(dirname(absolute)).href + '/' + await ctx.plugin(Loader) + await ctx.loader.create({ + name: '@cordisjs/plugin-include', + config: { path: pathToFileURL(absolute).href }, + }) + await ctx.loader.await() + assertEntriesLoaded(ctx) + return ctx +} + +/** + * Entry point: install the fail-loud guard, load `.env`, then boot the config + * named on argv (default `./cordis.yml`). Awaited at the module top level by the + * published bin (`#!/usr/bin/env node` shebang via the package's `bin` field). + */ +export async function main(argv: string[] = process.argv.slice(2)): Promise { + installFailLoud() + loadEnv() + await boot(argv[0] ?? './cordis.yml') +} + +/* v8 ignore start -- top-level CLI invocation; the testable core is boot()/main(), driven by the keyless Loader-path smoke */ +await main() +/* v8 ignore stop */ diff --git a/packages/ui/stdio-agent/src/index.ts b/packages/ui/stdio-agent/src/index.ts new file mode 100644 index 0000000000..c4b9ed202c --- /dev/null +++ b/packages/ui/stdio-agent/src/index.ts @@ -0,0 +1,98 @@ +/** + * The stdio chat app: the providerless agent spine ({@link + * @deepseek-ai/dsh-agent-core}) plus the coupled front-door cluster a terminal + * chat needs — a console logger, the readline `ui-stdio` UI, JSONL session + * persistence, and a pre-created `main` agent the UI drives. + * + * The cluster is BAKED IN, not left to the leaf: a stdio app always logs to the + * console (stdout is just the terminal) and always pre-creates the `main` agent + * `ui-stdio` sends to. The leaf supplies only the swappable backends (the LLM + * adapter, the bash executor), the optional `hmr` dev-reload plugin, and this + * app's {@link Config} (model, prompt, persistence root, welcome banner). + * + * `hmr` is deliberately a LEAF entry, not baked in here: it is a Loader-only, + * subprocess-only dev plugin (its constructor throws without `--expose-internals` + * + a live `loader`, and the in-process test tier cannot even import it), so a + * package whose `apply` statically pulled it in could never be unit-tested or + * carry the per-file coverage gate. Unlike the console logger, a stray `hmr` is + * not a stdout-purity footgun — so leaving it at the leaf costs no safety, while + * baking the LOGGER in (the real coupling) keeps stdout-vs-no-stdout a property + * of the artifact. + * + * Counterpart to {@link @deepseek-ai/dsh-acp-agent}, which bakes in the OPPOSITE + * cluster (no stdout logger, no pre-created agents — the ACP bridge reserves + * stdout for JSON-RPC and creates agents on demand). Splitting the two front + * doors into two packages makes each cluster a property of the artifact: there + * is no logger entry in the ACP leaf to get wrong. + * + * Plugin export shape: named `name`/`Config`/`apply`, NO default export — the + * cordis Loader's `unwrapExports` does `exports.default ?? exports`, so a stray + * default would collapse the module to the bare `apply` and drop the `Config` + * namespace (see docs/postmortem/0001). The keyless Loader-path smoke in the + * echo example guards this end-to-end. + * + * @module @deepseek-ai/dsh-stdio-agent + */ + +import type { Context } from 'cordis' +import ConsoleExporter from '@cordisjs/plugin-logger-console' +import z from 'schemastery' +import { AgentId } from '@deepseek-ai/dsh-agent' +import { SessionId } from '@deepseek-ai/dsh-session' +import * as agentCore from '@deepseek-ai/dsh-agent-core' +import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' +import * as uiStdio from '@deepseek-ai/dsh-ui-stdio' + +export const name = 'stdio-agent' + +/** + * App config: the swappable per-demo values, each routed to where the app wires + * it. `model`/`systemPrompt`/`resumeSessionId` configure the pre-created `main` + * agent (through {@link @deepseek-ai/dsh-agent-core}'s forwarded `agents` list); + * `persistenceRoot` is the JSONL backend's directory; `welcome` is the UI banner. + */ +export interface Config { + /** Model name for the `main` agent (must have a registered adapter). */ + model: string + /** System prompt for the `main` agent. */ + systemPrompt: string + /** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */ + persistenceRoot?: string + /** stdin-chat banner printed once on start. Defaults to `'ready.'`. */ + welcome?: string + /** + * If set, the `main` agent RESUMES this persisted session id instead of + * starting fresh. Sourced from an env var in the leaf `cordis.yml` + * (`resumeSessionId: !!js process.env.RESUME_SESSION_ID`). + */ + resumeSessionId?: string +} + +export const Config: z = z.object({ + model: z.string().required(), + systemPrompt: z.string().required(), + persistenceRoot: z.string().default('./.sessions'), + welcome: z.string().default('ready.'), + resumeSessionId: z.string(), +}) + +/** + * Compose the spine with the stdio front door. The console logger comes first + * (infra), then the agent-core bundle pre-creating the `main` agent from this + * app's `model`/`systemPrompt`/`resumeSessionId`, then the JSONL backend, then + * the `ui-stdio` UI bound to `main`. The `hmr` dev-reload plugin is a leaf + * concern (see the module doc), so it is not mounted here. + */ +export function apply(ctx: Context, config: Config): void { + ctx.plugin(ConsoleExporter) + ctx.plugin(agentCore, { + agents: [{ + id: AgentId('main'), + model: config.model, + systemPrompt: config.systemPrompt, + ...config.resumeSessionId !== undefined ? { resumeSessionId: SessionId(config.resumeSessionId) } : {}, + }], + }) + ctx.plugin(SessionPersistenceJsonl, { root: config.persistenceRoot ?? './.sessions' }) + ctx.plugin(uiStdio, { welcome: config.welcome ?? 'ready.', agent: 'main' }) +} diff --git a/packages/ui/stdio-agent/tests/built-bin.e2e.ts b/packages/ui/stdio-agent/tests/built-bin.e2e.ts new file mode 100644 index 0000000000..7605b35bb7 --- /dev/null +++ b/packages/ui/stdio-agent/tests/built-bin.e2e.ts @@ -0,0 +1,186 @@ +import { spawn } from 'node:child_process' +import { cp, mkdtemp, mkdir, rm, symlink, writeFile, readFile } from 'node:fs/promises' +import { existsSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' + +/** + * BUILT-ARTIFACT smoke for the published `dsh-stdio-agent` bin. The other smokes + * boot `src/bin.ts` under tsx — but the package's `bin` field points at + * `lib/bin.js`, run under plain `node` by a real consumer. tsx masks two failure + * modes the built bin had: (1) `boot()` returned before the loader tree settled, + * so the process exited 0 with no output and load errors surfaced as unhandled + * rejections AFTER boot; (2) config-path resolution could fall back to the cwd. + * This test runs the REAL `lib/bin.js` under `node` (NOT tsx) and asserts the + * banner + echo round-trip, so a regression in the published entry fails here. + * + * It build-gates: if `lib/bin.js` is absent (suite run without `pnpm run build`) + * the test SKIPS with a note. CI runs it after the build step. Setup mirrors a + * real install: a temp dir whose `node_modules/@deepseek-ai/*` (and the vendored + * `cordis`/`@cordisjs/*`) are symlinked to the built packages, a `cordis.yml` + * that loads the app + the example's mock backend, and `node --expose-internals` + * (the cordis Loader resolves bare plugin specifiers via its internal module + * loader, active only under that flag — the same flag `demo:echo` passes). + */ + +const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url)) +const stdioBin = join(repoRoot, 'packages/ui/stdio-agent/lib/bin.js') + +// Workspace packages the stdio app's tree needs, by repo-relative path. Each is +// symlinked into the temp consumer's node_modules under its package name, so +// plain `node` resolves the bare `@deepseek-ai/dsh-*` specifiers in cordis.yml +// to the built `lib/` (package.json `main`), exactly as an installed dep would. +const dshPackages = [ + 'core/agent-core', 'core/agent', 'core/session', 'core/system-prompt', + 'core/tools', 'core/agent-loop', 'llm/llm', 'bash/bash', 'bash/bash-local', + 'bash/tool-bash', 'support/invariants', 'support/ui-stdio', + 'session-persistence/session-persistence', + 'session-persistence/session-persistence-jsonl', 'ui/stdio-agent', +] +const vendorPackages = [ + 'cordis', 'loader', 'include', 'timer', 'hmr', 'logger-console', + 'schemastery', 'cosmokit', +] + +async function pkgName(absDir: string): Promise { + const json = JSON.parse(await readFile(join(absDir, 'package.json'), 'utf8')) as { name: string } + return json.name +} + +/** + * Build a temp consumer dir: `node_modules` with the workspace + vendor packages + * symlinked in, a `src/` carrying the example mock backend, and a `cordis.yml` + * that wires them onto the stdio app. Returns the dir (caller removes it). + * + * `disabledBrokenEntry` appends an entry that points at a non-existent plugin but + * is marked `disabled: true`. The Loader leaves a disabled entry fiber-less by + * design, so it exercises that the fail-loud entry-load guard does NOT mistake a + * valid disabled entry for a failed import. + */ +async function makeConsumer(welcome: string, disabledBrokenEntry = false): Promise { + const dir = await mkdtemp(join(tmpdir(), 'stdio-built-bin-')) + const nm = join(dir, 'node_modules') + for (const rel of dshPackages) { + const abs = join(repoRoot, 'packages', rel) + const name = await pkgName(abs) + const target = join(nm, name) + await mkdir(dirname(target), { recursive: true }) + await symlink(abs, target) + } + for (const v of vendorPackages) { + const abs = join(repoRoot, 'vendor', v) + const name = await pkgName(abs) + const target = join(nm, name) + await mkdir(dirname(target), { recursive: true }) + await symlink(abs, target) + } + // The example's mock model + echo tool are example-local TS plugins (Node 24+ + // strips types natively, so plain `node` loads them); they import the workspace + // packages the symlinked node_modules now provides. + await cp(join(repoRoot, 'examples/echo-agent/src'), join(dir, 'src'), { recursive: true }) + await writeFile(join(dir, 'cordis.yml'), [ + '- id: mock-llm', + ' name: \'./src/mock-llm.ts\'', + '- id: echo-tool', + ' name: \'./src/echo-tool.ts\'', + '- id: bash', + ' name: \'@deepseek-ai/dsh-bash-local\'', + '- id: stdio-agent', + ' name: \'@deepseek-ai/dsh-stdio-agent\'', + ' config:', + ' model: mock-echo', + ' systemPrompt: \'demo\'', + ` welcome: '${welcome}'`, + ...disabledBrokenEntry + ? ['- id: off', ' name: \'./src/does-not-exist.ts\'', ' disabled: true'] + : [], + '', + ].join('\n')) + return dir +} + +/** Run the built bin in `cwd` against `configArg` with one stdin line; resolve with stdout/stderr + exit code. */ +function runBuiltBin(cwd: string, configArg: string, line: string): Promise<{ stdout: string; code: number; stderr: string }> { + return new Promise((resolve, reject) => { + // --expose-internals: the cordis Loader resolves bare plugin specifiers via + // its internal module loader (active only under this flag); demo:echo passes + // it too. NO tsx — this is the published `node lib/bin.js` path. + const child = spawn(process.execPath, ['--expose-internals', stdioBin, configArg], { + cwd, + // Mock model: never calls the network, so no key needed. + env: { ...process.env }, + stdio: ['pipe', 'pipe', 'pipe'], + }) + let stdout = '' + let stderr = '' + child.stdout.setEncoding('utf8') + child.stdout.on('data', (c: string) => { stdout += c }) + child.stderr.setEncoding('utf8') + child.stderr.on('data', (c: string) => { stderr += c }) + const timer = setTimeout(() => { + child.kill('SIGKILL') + reject(new Error(`built bin did not exit within 25s. stdout:\n${stdout}\nstderr:\n${stderr}`)) + }, 25_000) + child.on('exit', (code) => { clearTimeout(timer); resolve({ stdout, code: code ?? -1, stderr }) }) + child.on('error', (err) => { clearTimeout(timer); reject(err) }) + child.stdin.write(`${line}\n`) + child.stdin.end() + }) +} + +let consumer: string | undefined + +afterEach(async () => { + if (consumer !== undefined) await rm(consumer, { recursive: true, force: true }) + consumer = undefined +}) + +describe.skipIf(!existsSync(stdioBin))('dsh-stdio-agent BUILT bin (node lib/bin.js, no tsx)', () => { + it('boots the published bin, prints its banner, and runs the echo tool round-trip', async () => { + consumer = await makeConsumer('BUILT-BIN-OK ready.') + const { stdout, code, stderr } = await runBuiltBin(consumer, './cordis.yml', 'echo hi') + expect(stderr).not.toContain('UNHANDLED') + expect(stderr).not.toContain('without inject') + // The banner proves boot() awaited the tree (the settle-race regression would + // exit 0 with empty stdout); the round-trip proves the whole app mounted. + expect(stdout).toContain('BUILT-BIN-OK ready.') + expect(stdout).toContain('[tool call] echo') + expect(stdout).toContain('[tool result] ECHO: HI') + expect(code).toBe(0) + }, 30_000) + + it('boots cleanly when the config disables an (otherwise unresolvable) entry', async () => { + // A `disabled: true` entry settles without a fiber by design; the fail-loud + // entry-load guard must NOT mistake it for a failed import. Even though its + // plugin path does not exist, the app boots and the round-trip works. + consumer = await makeConsumer('DISABLED-OK ready.', true) + const { stdout, code, stderr } = await runBuiltBin(consumer, './cordis.yml', 'echo hi') + expect(stderr).not.toContain('failed to load') + expect(stdout).toContain('DISABLED-OK ready.') + expect(stdout).toContain('[tool result] ECHO: HI') + expect(code).toBe(0) + }, 30_000) + + it('fails LOUD (non-zero exit + stderr) on a config whose directory does not exist', async () => { + // A consumer who typos the config path must get a clear failure, not silent + // success. This dir does not exist, so the include PLUGIN itself fails to + // import; the cordis Loader logs that and leaves the entry with no fiber (no + // rejection), which `boot()`'s entry-load check turns into a thrown error. + consumer = await makeConsumer('unused') + const { code, stderr } = await runBuiltBin(consumer, '/nonexistent/dir/cordis.yml', '') + expect(code).not.toBe(0) + expect(stderr).toContain('failed to load') + }, 30_000) + + it('fails LOUD (non-zero exit + stderr) on a missing config file in a real directory', async () => { + // The config DIRECTORY exists (the include plugin imports), but the file does + // not — the include's init throws "config file not found", which surfaces as + // an unhandled rejection the fail-loud guard turns into a non-zero exit. + consumer = await makeConsumer('unused') + const { code, stderr } = await runBuiltBin(consumer, './does-not-exist.yml', '') + expect(code).not.toBe(0) + expect(stderr).toContain('config file not found') + }, 30_000) +}) diff --git a/packages/ui/stdio-agent/tests/stdio-agent.spec.ts b/packages/ui/stdio-agent/tests/stdio-agent.spec.ts new file mode 100644 index 0000000000..f72de0a1da --- /dev/null +++ b/packages/ui/stdio-agent/tests/stdio-agent.spec.ts @@ -0,0 +1,92 @@ +import { describe, it, expect } from 'vitest' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' +import { AgentId } from '@deepseek-ai/dsh-agent' +import * as stdioAgent from '../src/index.ts' + +/** + * Unit coverage for the @deepseek-ai/dsh-stdio-agent app plugin: mounting it + * composes the console logger, the agent-core spine (pre-creating the `main` + * agent from the app config), the JSONL backend, and the readline UI in one + * `ctx.plugin`. The forwarded `model`/`systemPrompt` reach the pre-created + * agent; `persistenceRoot`/`welcome`/`resumeSessionId` route to their backends. + * + * `hmr` is NOT part of this plugin (it is a leaf entry — a Loader-only dev + * plugin the in-process tier cannot import); the REAL Loader-path guard (export + * shape, `unwrapExports`, the whole subprocess tree incl. `hmr`) is the keyless + * echo smoke in `examples/echo-agent`. Here we assert the composition + config + * forwarding the unit tier can reach. + */ +async function mount(config: stdioAgent.Config): Promise { + const ctx = new Context() + await ctx.plugin(stdioAgent, config) + // The app mounts its children inside apply() (not awaited there); let their + // fibers settle so the spine services + the pre-created agent are ready. + await new Promise(resolve => setTimeout(resolve, 80)) + return ctx +} + +describe('dsh-stdio-agent app', () => { + it('composes the spine + front-door cluster and pre-creates the main agent', async () => { + const ctx = await mount({ model: 'mock', systemPrompt: 'hi', persistenceRoot: '/tmp/dsh-stdio-agent-spec' }) + // The spine services (brought up by the agent-core bundle) are all present. + expect(ctx.get('agents')).toBeDefined() + expect(ctx.get('agentLoop')).toBeDefined() + expect(ctx.get('sessionPersistence')).toBeDefined() + // The pre-created `main` agent the UI drives. + expect(ctx.get('agents')?.get(AgentId('main'))).toBeDefined() + await ctx.fiber.dispose() + }) + + it('defaults persistenceRoot and welcome when omitted', async () => { + // Direct apply (NOT via ctx.plugin, which validates+defaults the config + // first) so the runtime `?? './.sessions'` / `?? 'ready.'` fallbacks on + // apply()'s last two lines are the ones that fire — covering a + // schema-bypassing direct-mount caller. + const ctx = new Context() + stdioAgent.apply(ctx, { model: 'mock', systemPrompt: 'hi' }) + await new Promise(resolve => setTimeout(resolve, 80)) + expect(ctx.get('sessionPersistence')).toBeDefined() + expect(ctx.get('agents')?.get(AgentId('main'))).toBeDefined() + await ctx.fiber.dispose() + }) + + it('forwards resumeSessionId onto the pre-created agent when set', async () => { + // A resume id defers agent creation until persistence loads; with no backing + // session the resume is contained + logged, so no `main` agent registers — + // the branch that maps resumeSessionId through is what this covers. + const ctx = await mount({ + model: 'mock', + systemPrompt: 'hi', + persistenceRoot: '/tmp/dsh-stdio-agent-spec-resume', + resumeSessionId: 'no-such-session', + }) + expect(ctx.get('agents')?.get(AgentId('main'))).toBeUndefined() + await ctx.fiber.dispose() + }) + + it('exposes its name and Config schema', () => { + expect(stdioAgent.name).toBe('stdio-agent') + expect(stdioAgent.Config).toBeDefined() + }) + + it('has the namespace-plugin export shape (no stray default) so the Loader keeps name/Config/apply', () => { + // Postmortem 0001 guard: a stray `export default apply` makes the Loader's + // `unwrapExports` (`exports.default ?? exports`) collapse the module to the + // bare `apply` function, DROPPING the named `name`/`Config`. This package has + // no `inject` export, so that collapse would NOT crash at load (the keyless + // echo smoke would still boot the tree) — it would silently lose its config + // schema. So guard the shape directly here: assert no `default` export, and + // that the real `unwrapExports` leaves `name`/`Config`/`apply` intact. Adding + // `export default` to src/index.ts fails this test. + expect('default' in stdioAgent).toBe(false) + expect(typeof stdioAgent.apply).toBe('function') + + const loader = Object.create(Loader.prototype) as Loader + const unwrapped = loader.unwrapExports(stdioAgent) as Record + expect(unwrapped).toBe(stdioAgent) + expect(unwrapped.name).toBe('stdio-agent') + expect(unwrapped.Config).toBeDefined() + expect(typeof unwrapped.apply).toBe('function') + }) +}) diff --git a/packages/ui/stdio-agent/tsconfig.json b/packages/ui/stdio-agent/tsconfig.json new file mode 100644 index 0000000000..58b492a549 --- /dev/null +++ b/packages/ui/stdio-agent/tsconfig.json @@ -0,0 +1,39 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../../vendor/loader" + }, + { + "path": "../../../vendor/logger-console" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../core/session" + }, + { + "path": "../../core/agent-core" + }, + { + "path": "../../session-persistence/session-persistence-jsonl" + }, + { + "path": "../../support/ui-stdio" + } + ] +} diff --git a/packages/ui/stdio-agent/tsdown.config.ts b/packages/ui/stdio-agent/tsdown.config.ts new file mode 100644 index 0000000000..53797cdd79 --- /dev/null +++ b/packages/ui/stdio-agent/tsdown.config.ts @@ -0,0 +1,19 @@ +import { defineConfig } from 'tsdown' + +/** + * stdio-agent ships TWO entries: the plugin (`index`) and the CLI `bin` + * (`bin`), the latter referenced by package.json `bin`/`exports["./bin"]`. + * The root tsdown builds only `lib/types/index.js`, so this override adds + * `lib/types/bin.js`. Declarations come from `tsc -b` (dts: false), + * matching every package. + */ +export default defineConfig({ + entry: ['lib/types/index.js', 'lib/types/bin.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, +}) diff --git a/packages/util/README.md b/packages/util/README.md new file mode 100644 index 0000000000..ae73c8125f --- /dev/null +++ b/packages/util/README.md @@ -0,0 +1,9 @@ +# util/ — low-level shared utilities + +Zero-dependency primitives shared across the other groups. A package lands here when it owns a tiny, foundational type or helper that several capability families need but that belongs to none of them — keeping it out of any one group avoids a capability package depending on an unrelated one just to reach a shared primitive. These are **support** packages: small, stable, and free of harness dependencies. + +| Package | Role | +|---|---| +| `brand/` | The type-only `Branded` nominal-typing primitive (no runtime code, no harness deps) | + +`dsh-brand` is the canonical case: it owns ONLY the `Branded` helper, so a capability package can brand the ids it owns (`dsh-bash`'s `BashTaskId`/`OwnerToken`, `dsh-session`'s `SessionId`, …) by depending on `dsh-brand` alone, without pulling in an unrelated package just to reach `Branded`. diff --git a/packages/util/brand/README.md b/packages/util/brand/README.md new file mode 100644 index 0000000000..8f7943def7 --- /dev/null +++ b/packages/util/brand/README.md @@ -0,0 +1,26 @@ +# dsh-brand + +The `Branded` nominal-typing primitive — a tiny, **type-only** package (no runtime code, no harness-package dependency) shared by every package that owns a cross-boundary id. + +## What `Branded` is + +A brand makes structurally-identical strings non-interchangeable at the type level: an `AgentId` cannot be passed where a `CallId` is expected, even though both are plain `string`s at runtime. + +```ts +import type { Branded } from '@deepseek-ai/dsh-brand' + +export type SessionId = Branded<'SessionId'> + +/** Brand a string as a SessionId (a plain cast — zero runtime cost). */ +export function SessionId(id: string): SessionId { + return id as SessionId +} +``` + +Construction goes through the per-id factory in the OWNING package (a plain cast inside — zero runtime cost). Comparison, logging, JSON serialization, and the wire format all behave exactly as for an ordinary string; the brand is erased at compile time. + +## Policy: brand ids that cross package boundaries + +A package brands the ids it OWNS — `CallId` in `dsh-llm` (tool-call correlation), `SessionId` in `dsh-session`, `AgentId` in `dsh-agent`, `BashTaskId`/`OwnerToken` in `dsh-bash`. Branding is for ids that cross package boundaries and could plausibly be confused; **not every string needs a brand.** + +This package owns ONLY the primitive — no concrete id, no runtime code beyond the (erased) type. Keeping the primitive dependency-free is the point: a capability package can brand its ids without depending on an unrelated package. `dsh-bash`, for example, brands `BashTaskId`/`OwnerToken` by depending on `dsh-brand` alone — it never pulls in `dsh-llm` (or `dsh-session`) just to reach `Branded`. diff --git a/packages/util/brand/package.json b/packages/util/brand/package.json new file mode 100644 index 0000000000..8059952170 --- /dev/null +++ b/packages/util/brand/package.json @@ -0,0 +1,30 @@ +{ + "name": "@deepseek-ai/dsh-brand", + "description": "Type-only Branded nominal-typing primitive for the DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/util/brand/src/index.ts b/packages/util/brand/src/index.ts new file mode 100644 index 0000000000..051ced94b7 --- /dev/null +++ b/packages/util/brand/src/index.ts @@ -0,0 +1,27 @@ +/** + * The `Branded` nominal-typing primitive — a type-only utility (no runtime + * code, no harness-package dependency) shared by every package that owns a + * cross-boundary id. + * + * A brand makes structurally-identical strings non-interchangeable at the type + * level: an `AgentId` cannot be passed where a `CallId` is expected, even + * though both are plain strings at runtime. Construction goes through a per-id + * factory in the OWNING package (a plain cast inside — zero runtime cost); + * comparison, logging, and serialization all behave as ordinary strings. + * + * Policy: a package brands the ids it owns — `CallId` in dsh-llm (tool-call + * correlation), `SessionId` in dsh-session, `AgentId` in dsh-agent, + * `BashTaskId`/`OwnerToken` in dsh-bash. Branding is for ids that cross package + * boundaries and could plausibly be confused; not every string needs a brand. + * This package owns ONLY the primitive — no concrete id, no runtime code beyond + * the (erased) type — so the brand vocabulary stays dependency-free and a + * package can brand its ids without depending on an unrelated capability + * package (e.g. dsh-bash brands its ids without pulling in dsh-llm). + * + * @module @deepseek-ai/dsh-brand + */ + +declare const BRAND: unique symbol + +/** A string carrying a compile-time-only brand `B`. */ +export type Branded = string & { readonly [BRAND]: B } diff --git a/packages/util/brand/tsconfig.json b/packages/util/brand/tsconfig.json new file mode 100644 index 0000000000..749cb0208e --- /dev/null +++ b/packages/util/brand/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9c534153ab..3bca330308 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -66,105 +66,16 @@ importers: specifier: ^4.1.8 version: 4.1.8(@types/node@25.9.3)(@vitest/coverage-v8@4.1.8)(vite@8.0.16(@types/node@25.9.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)) - packages/acp: - dependencies: - '@agentclientprotocol/sdk': - specifier: 0.25.1 - version: 0.25.1(zod@4.4.3) - schemastery: - specifier: ^3.17.0 - version: 3.18.0 - zod: - specifier: ^4.0.0 - version: 4.4.3 + packages/bash/bash: devDependencies: - '@deepseek-ai/dsh-agent': + '@deepseek-ai/dsh-brand': specifier: workspace:^ - version: link:../agent - '@deepseek-ai/dsh-agent-loop': - specifier: workspace:^ - version: link:../agent-loop - '@deepseek-ai/dsh-bash-local': - specifier: workspace:^ - version: link:../bash-local - '@deepseek-ai/dsh-llm': - specifier: workspace:^ - version: link:../llm - '@deepseek-ai/dsh-session': - specifier: workspace:^ - version: link:../session - '@deepseek-ai/dsh-session-persistence': - specifier: workspace:^ - version: link:../session-persistence - '@deepseek-ai/dsh-session-persistence-jsonl': - specifier: workspace:^ - version: link:../session-persistence-jsonl - '@deepseek-ai/dsh-system-prompt': - specifier: workspace:^ - version: link:../system-prompt - '@deepseek-ai/dsh-tool-bash': - specifier: workspace:^ - version: link:../tool-bash - '@deepseek-ai/dsh-tools': - specifier: workspace:^ - version: link:../tools + version: link:../../util/brand cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/agent: - devDependencies: - '@deepseek-ai/dsh-llm': - specifier: workspace:^ - version: link:../llm - '@deepseek-ai/dsh-session': - specifier: workspace:^ - version: link:../session - cordis: - specifier: ^4.0.0-rc.6 - version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - - packages/agent-loop: - dependencies: - schemastery: - specifier: ^3.18.0 - version: 3.18.0 - devDependencies: - '@deepseek-ai/dsh-agent': - specifier: workspace:^ - version: link:../agent - '@deepseek-ai/dsh-invariants': - specifier: workspace:^ - version: link:../invariants - '@deepseek-ai/dsh-llm': - specifier: workspace:^ - version: link:../llm - '@deepseek-ai/dsh-session': - specifier: workspace:^ - version: link:../session - '@deepseek-ai/dsh-session-persistence': - specifier: workspace:^ - version: link:../session-persistence - '@deepseek-ai/dsh-session-persistence-jsonl': - specifier: workspace:^ - version: link:../session-persistence-jsonl - '@deepseek-ai/dsh-system-prompt': - specifier: workspace:^ - version: link:../system-prompt - '@deepseek-ai/dsh-tools': - specifier: workspace:^ - version: link:../tools - cordis: - specifier: ^4.0.0-rc.6 - version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - - packages/bash: - devDependencies: - cordis: - specifier: ^4.0.0-rc.6 - version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - - packages/bash-local: + packages/bash/bash-local: dependencies: schemastery: specifier: ^3.18.0 @@ -177,14 +88,44 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/invariants: + packages/bash/tool-bash: devDependencies: '@deepseek-ai/dsh-agent': specifier: workspace:^ - version: link:../agent + version: link:../../core/agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:^ + version: link:../../core/agent-loop + '@deepseek-ai/dsh-bash': + specifier: workspace:^ + version: link:../bash + '@deepseek-ai/dsh-bash-local': + specifier: workspace:^ + version: link:../bash-local '@deepseek-ai/dsh-llm': specifier: workspace:^ - version: link:../llm + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/core/agent: + devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../session @@ -192,13 +133,119 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/llm: + packages/core/agent-core: devDependencies: + '@cordisjs/plugin-timer': + specifier: workspace:^ + version: link:../../../vendor/timer + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:^ + version: link:../agent-loop + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../session + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../system-prompt + '@deepseek-ai/dsh-tool-bash': + specifier: workspace:^ + version: link:../../bash/tool-bash + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../tools cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/llm-deepseek: + packages/core/agent-loop: + dependencies: + schemastery: + specifier: ^3.18.0 + version: 3.18.0 + devDependencies: + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../agent + '@deepseek-ai/dsh-invariants': + specifier: workspace:^ + version: link:../../support/invariants + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../session + '@deepseek-ai/dsh-session-persistence': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence-jsonl + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../system-prompt + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../tools + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/core/session: + devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/core/system-prompt: + devDependencies: + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/core/tools: + devDependencies: + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../agent + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../system-prompt + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/llm/llm: + devDependencies: + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/llm/llm-deepseek: dependencies: schemastery: specifier: ^3.18.0 @@ -211,7 +258,7 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/llm-pi-ai: + packages/llm/llm-pi-ai: dependencies: '@earendil-works/pi-ai': specifier: ^0.79.1 @@ -230,25 +277,16 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/session: - devDependencies: - '@deepseek-ai/dsh-llm': - specifier: workspace:^ - version: link:../llm - cordis: - specifier: ^4.0.0-rc.6 - version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - - packages/session-persistence: + packages/session-persistence/session-persistence: devDependencies: '@deepseek-ai/dsh-session': specifier: workspace:^ - version: link:../session + version: link:../../core/session cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/session-persistence-jsonl: + packages/session-persistence/session-persistence-jsonl: dependencies: schemastery: specifier: ^3.18.0 @@ -256,7 +294,7 @@ importers: devDependencies: '@deepseek-ai/dsh-session': specifier: workspace:^ - version: link:../session + version: link:../../core/session '@deepseek-ai/dsh-session-persistence': specifier: workspace:^ version: link:../session-persistence @@ -264,7 +302,7 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/session-persistence-sqlite: + packages/session-persistence/session-persistence-sqlite: dependencies: schemastery: specifier: ^3.18.0 @@ -272,7 +310,7 @@ importers: devDependencies: '@deepseek-ai/dsh-session': specifier: workspace:^ - version: link:../session + version: link:../../core/session '@deepseek-ai/dsh-session-persistence': specifier: workspace:^ version: link:../session-persistence @@ -280,56 +318,157 @@ importers: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/system-prompt: - devDependencies: - '@deepseek-ai/dsh-llm': - specifier: workspace:^ - version: link:../llm - cordis: - specifier: ^4.0.0-rc.6 - version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - - packages/tool-bash: + packages/support/invariants: devDependencies: '@deepseek-ai/dsh-agent': specifier: workspace:^ - version: link:../agent + version: link:../../core/agent + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/support/llm-replay: + devDependencies: + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/support/ui-stdio: + dependencies: + schemastery: + specifier: ^3.18.0 + version: 3.18.0 + devDependencies: + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../../llm/llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) + + packages/ui/acp: + dependencies: + '@agentclientprotocol/sdk': + specifier: 0.25.1 + version: 0.25.1(zod@4.4.3) + schemastery: + specifier: ^3.17.0 + version: 3.18.0 + zod: + specifier: ^4.0.0 + version: 4.4.3 + devDependencies: + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent '@deepseek-ai/dsh-agent-loop': specifier: workspace:^ - version: link:../agent-loop - '@deepseek-ai/dsh-bash': - specifier: workspace:^ - version: link:../bash + version: link:../../core/agent-loop '@deepseek-ai/dsh-bash-local': specifier: workspace:^ - version: link:../bash-local + version: link:../../bash/bash-local '@deepseek-ai/dsh-llm': specifier: workspace:^ - version: link:../llm + version: link:../../llm/llm '@deepseek-ai/dsh-session': specifier: workspace:^ - version: link:../session + version: link:../../core/session + '@deepseek-ai/dsh-session-persistence': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence-jsonl '@deepseek-ai/dsh-system-prompt': specifier: workspace:^ - version: link:../system-prompt + version: link:../../core/system-prompt + '@deepseek-ai/dsh-tool-bash': + specifier: workspace:^ + version: link:../../bash/tool-bash '@deepseek-ai/dsh-tools': specifier: workspace:^ - version: link:../tools + version: link:../../core/tools cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) - packages/tools: + packages/ui/acp-agent: devDependencies: + '@cordisjs/plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@cordisjs/plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-acp': + specifier: workspace:^ + version: link:../acp + '@deepseek-ai/dsh-agent-core': + specifier: workspace:^ + version: link:../../core/agent-core + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence-jsonl + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader) + schemastery: + specifier: ^3.17.0 + version: 3.18.0 + + packages/ui/stdio-agent: + devDependencies: + '@cordisjs/plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@cordisjs/plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@cordisjs/plugin-logger-console': + specifier: workspace:^ + version: link:../../../vendor/logger-console '@deepseek-ai/dsh-agent': specifier: workspace:^ - version: link:../agent - '@deepseek-ai/dsh-llm': + version: link:../../core/agent + '@deepseek-ai/dsh-agent-core': specifier: workspace:^ - version: link:../llm - '@deepseek-ai/dsh-system-prompt': + version: link:../../core/agent-core + '@deepseek-ai/dsh-session': specifier: workspace:^ - version: link:../system-prompt + version: link:../../core/session + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence-jsonl + '@deepseek-ai/dsh-ui-stdio': + specifier: workspace:^ + version: link:../../support/ui-stdio + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader) + schemastery: + specifier: ^3.17.0 + version: 3.18.0 + + packages/util/brand: + devDependencies: cordis: specifier: ^4.0.0-rc.6 version: 4.0.0-rc.6(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.4) @@ -2796,7 +2935,7 @@ snapshots: '@aws-sdk/types': 3.973.12 '@smithy/core': 3.24.7 '@smithy/fetch-http-handler': 5.4.7 - '@smithy/node-http-handler': 4.7.3 + '@smithy/node-http-handler': 4.7.8 '@smithy/types': 4.14.4 tslib: 2.8.1 @@ -3844,6 +3983,14 @@ snapshots: '@cordisjs/plugin-include': 1.0.4(@cordisjs/plugin-loader@1.0.0-rc.4)(cordis@4.0.0-rc.6) '@cordisjs/plugin-loader': 1.0.0-rc.4(cordis@4.0.0-rc.6) + cordis@4.0.0-rc.6(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader): + dependencies: + '@standard-schema/spec': 1.1.0 + cosmokit: 1.8.1 + optionalDependencies: + '@cordisjs/plugin-include': link:vendor/include + '@cordisjs/plugin-loader': link:vendor/loader + cosmokit@1.8.1: {} cross-spawn@7.0.6: diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 2f957141fe..b2b731fc58 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -1,6 +1,6 @@ packages: - vendor/* - - packages/* + - packages/*/* peerDependencyRules: allowedVersions: diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index 50b1a80078..a669a8572d 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -5,11 +5,16 @@ * Run: `tsx scripts/check-workspace-constraints.ts`. */ -import { readdirSync, readFileSync } from 'node:fs' +import { existsSync, readdirSync, readFileSync } from 'node:fs' import { join, relative, resolve } from 'node:path' const root = resolve(import.meta.dirname, '..') -const workspaceGlobs = ['vendor', 'packages'] as const +// vendor/* is single-level; packages// nests one level deeper +// (the group dirs — core/llm/bash/… — are pure containers with no manifest). +const workspaceGlobs = [ + { dir: 'vendor', depth: 1 }, + { dir: 'packages', depth: 2 }, +] as const const vendoredPackages = new Set([ 'cordis', 'cosmokit', @@ -28,6 +33,18 @@ interface PackageManifest { version?: string private?: boolean type?: string + main?: string + types?: string + bin?: string | Record + exports?: Record< + string, + | { + types?: string + default?: string + } + | undefined + > + files?: string[] peerDependencies?: Record devDependencies?: Record } @@ -42,15 +59,25 @@ function readJson(path: string): PackageManifest { return JSON.parse(readFileSync(path, 'utf8')) as PackageManifest } +/** Repo-relative dirs holding a package.json, walked to the configured depth. */ +function packageDirs(base: string, depth: number): string[] { + if (depth === 1) { + return readdirSync(join(root, base), { withFileTypes: true }) + .filter(entry => entry.isDirectory()) + .map(entry => join(base, entry.name)) + } + return readdirSync(join(root, base), { withFileTypes: true }) + .filter(entry => entry.isDirectory()) + .flatMap(group => packageDirs(join(base, group.name), depth - 1)) +} + function workspaceManifests(): WorkspaceManifest[] { const manifests: WorkspaceManifest[] = [ { dir: '.', manifest: readJson(join(root, 'package.json')) }, ] - for (const workspaceDir of workspaceGlobs) { - for (const entry of readdirSync(join(root, workspaceDir), { withFileTypes: true })) { - if (!entry.isDirectory()) continue - const dir = join(workspaceDir, entry.name) + for (const { dir: base, depth } of workspaceGlobs) { + for (const dir of packageDirs(base, depth)) { manifests.push({ dir, manifest: readJson(join(root, dir, 'package.json')) }) } } @@ -58,6 +85,29 @@ function workspaceManifests(): WorkspaceManifest[] { return manifests } +const dshPackageFiles = [ + 'lib/index.js', + 'lib/types/**/*.d.ts', + 'lib/types/**/*.d.ts.map', + 'src', +] as const + +const dshBinPackageFiles = [ + 'lib/index.js', + 'lib/bin.js', + 'lib/types/**/*.d.ts', + 'lib/types/**/*.d.ts.map', + 'src', +] as const + +function sameStringList(actual: readonly string[] | undefined, expected: readonly string[]): boolean { + return !!actual && actual.length === expected.length && actual.every((value, index) => value === expected[index]) +} + +function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { + return manifest.bin ? dshBinPackageFiles : dshPackageFiles +} + function checkWorkspace({ dir, manifest }: WorkspaceManifest): string[] { const errors: string[] = [] const label = manifest.name ?? dir @@ -85,12 +135,58 @@ function checkWorkspace({ dir, manifest }: WorkspaceManifest): string[] { if (manifest.type !== 'module') { errors.push(`${label}: package.json must set "type": "module"`) } + if (manifest.main !== 'lib/index.js') { + errors.push(`${label}: package.json must set "main": "lib/index.js"`) + } + if (manifest.types !== 'lib/types/index.d.ts') { + errors.push(`${label}: package.json must set "types": "lib/types/index.d.ts"`) + } + if (manifest.exports?.['.']?.types !== './lib/types/index.d.ts') { + errors.push(`${label}: package.json exports["."].types must be "./lib/types/index.d.ts"`) + } + if (manifest.exports?.['.']?.default !== './lib/index.js') { + errors.push(`${label}: package.json exports["."].default must be "./lib/index.js"`) + } + const expectedFiles = expectedDshPackageFiles(manifest) + if (!sameStringList(manifest.files, expectedFiles)) { + errors.push(`${label}: package.json files must be ${JSON.stringify(expectedFiles)}`) + } } return errors.map(error => `${relative(root, join(root, dir, 'package.json'))}: ${error}`) } -const errors = workspaceManifests().flatMap(checkWorkspace) +/** + * Enforce the packages/ hierarchy SHAPE: every package lives at exactly + * `packages//`. A group dir is a pure container — it holds packages, + * never sources of its own — so it must NOT carry a package.json, and a package + * must NOT sit directly at the `packages/` root (the old flat layout) nor nest a + * level deeper. The group NAMES are open on purpose: a new group may be added + * without touching this gate, but the depth-2 shape is fixed. This is what keeps + * a stray flat package or an over-nested one from regressing the hierarchy. + */ +function checkHierarchyShape(): string[] { + const errors: string[] = [] + const packagesRoot = join(root, 'packages') + for (const group of readdirSync(packagesRoot, { withFileTypes: true })) { + if (!group.isDirectory()) continue + const groupRel = join('packages', group.name) + if (existsSync(join(packagesRoot, group.name, 'package.json'))) { + errors.push(`${groupRel}: a group dir must not contain a package.json — packages live at packages//, not directly under packages/`) + continue + } + for (const pkg of readdirSync(join(packagesRoot, group.name), { withFileTypes: true })) { + if (!pkg.isDirectory()) continue + const pkgRel = join(groupRel, pkg.name) + if (!existsSync(join(packagesRoot, group.name, pkg.name, 'package.json'))) { + errors.push(`${pkgRel}: expected a package here (no package.json found) — the hierarchy is exactly packages//, no deeper nesting`) + } + } + } + return errors +} + +const errors = [...workspaceManifests().flatMap(checkWorkspace), ...checkHierarchyShape()] if (errors.length > 0) { console.error(errors.join('\n')) process.exitCode = 1 diff --git a/scripts/doc-typecheck.ts b/scripts/doc-typecheck.ts index 10068eb792..7422d6ac77 100644 --- a/scripts/doc-typecheck.ts +++ b/scripts/doc-typecheck.ts @@ -3,12 +3,19 @@ * Markdown so documentation can't drift from the API it documents. * * Every ```ts block in README.md, docs/** and packages/* /README.md is - * extracted to a temp file and compiled with `tsc --noEmit` against the - * workspace sources (resolved through the same `paths` map vitest uses, so no - * build is required first). A block that is a deliberate sketch rather than - * compilable code opts out with an explicit ` ```ts ignore-check ` info string - * — the opt-out is visible in the source, and this script reports the ratio so - * the escape hatch can't quietly become the norm. + * extracted to a temp typecheck project and compiled against the workspace + * sources through the same project-reference boundaries used by repo + * typecheck. A block that is a deliberate sketch rather than compilable code + * opts out with an explicit ` ```ts ignore-check ` info string — the opt-out + * is visible in the source, and this script reports the ratio so the escape + * hatch can't quietly become the norm. A third info string, + * doc-typecheck.ts recognizes two more fence variants and skips both (each is a + * separately-checked category, not an unchecked sketch, so neither counts in the + * opt-out ratio): ` ```ts type-equiv ` is a verbatim source-type paste that + * `scripts/verify-type-equiv.ts` drift-checks, and ` ```ts cordis-catalog ` is a + * generated event/service signature fragment in the cordis catalog (a bare + * signature is not standalone-compilable; the catalog is generated and frozen by + * `scripts/gen-cordis-catalog.ts` + its `--check` freshness gate). * * Run: `tsx scripts/doc-typecheck.ts`. */ @@ -17,26 +24,44 @@ import { execFileSync } from 'node:child_process' import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import { join, relative, resolve } from 'node:path' import { glob } from 'node:fs/promises' +import ts from 'typescript' const root = resolve(import.meta.dirname, '..') +/** + * How a fenced block participates in this gate: + * - `check` (` ```ts `) — compiled. + * - `ignore` (` ```ts ignore-check `) — a deliberate sketch; skipped, and + * counted in the opt-out ratio so the escape hatch can't quietly take over. + * - `type-equiv` (` ```ts type-equiv `) — a verbatim paste of a source type + * definition, drift-checked by `scripts/verify-type-equiv.ts` against the + * source symbol. Skipped HERE (it is not standalone-compilable — no imports) + * and EXCLUDED from the opt-out ratio: it is a separate fully-checked + * category, not an unchecked sketch. + * - `cordis-catalog` (` ```ts cordis-catalog `) — a generated event/service + * signature fragment in the cordis catalog. Skipped HERE for the same reason + * (a bare signature fragment has no imports and does not stand alone) and + * EXCLUDED from the opt-out ratio: the catalog is generated and frozen by + * `scripts/gen-cordis-catalog.ts` + its `--check` freshness gate. + */ +type BlockKind = 'check' | 'ignore' | 'type-equiv' | 'cordis-catalog' + /** One extracted code block. */ interface Block { file: string /** 1-based line of the opening fence. */ line: number - /** `true` when the fence is ` ```ts ignore-check ` (skip compilation). */ - ignored: boolean + kind: BlockKind code: string } -/** Extract every ```ts / ```ts ignore-check block from one Markdown file. */ +/** Extract every ts / ts ignore-check / ts type-equiv / ts cordis-catalog block from one Markdown file. */ function extractBlocks(absPath: string): Block[] { const text = readFileSync(absPath, 'utf8') const lines = text.split('\n') const file = relative(root, absPath) const blocks: Block[] = [] - let open: { line: number; ignored: boolean; body: string[] } | null = null + let open: { line: number; kind: BlockKind; body: string[] } | null = null lines.forEach((raw, i) => { const fence = /^```(\s*)(\S.*)?$/.exec(raw) @@ -46,58 +71,56 @@ function extractBlocks(absPath: string): Block[] { } if (open) { // closing fence - blocks.push({ file, line: open.line, ignored: open.ignored, code: open.body.join('\n') }) + blocks.push({ file, line: open.line, kind: open.kind, code: open.body.join('\n') }) open = null return } // opening fence — only care about ts blocks const info = (fence[2] ?? '').trim() - if (info === 'ts' || info === 'ts ignore-check') { - open = { line: i + 1, ignored: info === 'ts ignore-check', body: [] } - } + const kind: BlockKind | null = + info === 'ts' ? 'check' + : info === 'ts ignore-check' ? 'ignore' + : info === 'ts type-equiv' ? 'type-equiv' + : info === 'ts cordis-catalog' ? 'cordis-catalog' + : null + if (kind) open = { line: i + 1, kind, body: [] } }) return blocks } -/** - * Read the workspace `paths` map from tsconfig.typecheck.json (JSONC). This map - * resolves vendored packages to their BUILT declarations (`lib`) and harness - * packages to source (`src`) — the same resolution `pnpm run lint`/`typecheck` use. - * Resolving vendor to `lib` (not `src`) is essential: otherwise tsc type-checks - * raw vendor source and floods the run with unrelated errors. Requires the - * vendor `lib/` to exist (a fresh clone runs `pnpm run build` first; CI does too). - */ -function workspacePaths(): Record { - const raw = readFileSync(join(root, 'tsconfig.typecheck.json'), 'utf8') - // Strip // line comments and /* */ block comments so JSON.parse accepts it. - const stripped = raw - .replace(/\/\*[\s\S]*?\*\//g, '') - .replace(/(^|[^:])\/\/.*$/gm, '$1') - return (JSON.parse(stripped) as { compilerOptions: { paths: Record } }) - .compilerOptions.paths -} - -/** The standalone tsconfig for the temp project (copies base resolution, no - * composite/declaration settings that would fight `--noEmit`). */ -function tempTsconfig(): string { - return JSON.stringify({ - compilerOptions: { - target: 'es2024', - module: 'esnext', - moduleResolution: 'bundler', - allowImportingTsExtensions: true, - strict: true, - noEmit: true, - skipLibCheck: true, - types: ['node'], - baseUrl: root, - ignoreDeprecations: '6.0', - paths: workspacePaths(), - }, +/** Reuse the repo typecheck graph references from a temp project one directory below root. */ +function workspaceReferences(): { path: string }[] { + const file = join(root, 'tsconfig.json') + // Parse with TypeScript's own JSONC reader, not a hand-rolled comment strip: + // a regex strip mistakes the `/*/` in a wildcard path candidate + // (`./packages/core/*/src`) for a block comment and corrupts the map. + const result = ts.readConfigFile(file, p => readFileSync(p, 'utf8')) + if (result.error) { + throw new Error(`doc-typecheck: cannot read ${file}: ${ts.flattenDiagnosticMessageText(result.error.messageText, '\n')}`) + } + // `config` is typed `any` by the TS API; narrow it to the one field we read. + const { references } = result.config as { compilerOptions: { paths: Record }; references: { path: string }[] } + return references.map(({ path }) => { + const relativeToTemp = path.startsWith('./') ? `../${path.slice(2)}` : `../${path}` + return { path: relativeToTemp } }) } -const markdownGlobs = ['README.md', 'docs/**/*.md', 'packages/*/README.md'] +/** The standalone tsconfig for the temp typecheck project. */ +function tempTsconfig(): string { + return JSON.stringify({ + extends: '../tsconfig.json', + compilerOptions: { + noUnusedLocals: false, + noUnusedParameters: false, + tsBuildInfoFile: './tsconfig.tsbuildinfo', + }, + include: ['block-*.ts'], + references: workspaceReferences(), + }) +} + +const markdownGlobs = ['README.md', 'docs/**/*.md', 'packages/*/*.md', 'packages/*/*/*.md'] const files: string[] = [] for (const pattern of markdownGlobs) { @@ -106,8 +129,14 @@ for (const pattern of markdownGlobs) { files.sort() const all = files.flatMap(extractBlocks) -const checked = all.filter(b => !b.ignored) -const ignored = all.filter(b => b.ignored) +const checked = all.filter(b => b.kind === 'check') +const ignored = all.filter(b => b.kind === 'ignore') +// `type-equiv` and `cordis-catalog` blocks are verified elsewhere +// (verify-type-equiv.ts and the gen-cordis-catalog `--check` freshness gate), +// not here: neither compiled nor counted toward the opt-out ratio (each is a +// separate fully-checked category, not an unchecked sketch). The ratio's +// denominator is therefore the compile-eligible blocks only. +const ratioDenominator = checked.length + ignored.length if (checked.length === 0) { console.log('doc-typecheck: no ts code blocks to check.') @@ -125,11 +154,12 @@ try { }) try { - execFileSync('node_modules/.bin/tsc', ['-p', join(tmp, 'tsconfig.json')], { cwd: root, stdio: 'pipe' }) + execFileSync('node_modules/.bin/tsc', ['-b', join(tmp, 'tsconfig.json')], { cwd: root, stdio: 'pipe' }) } catch (error: unknown) { - const out = (error as { stdout?: Buffer }).stdout?.toString() ?? '' + const failed = error as { stdout?: Buffer; stderr?: Buffer } + const out = `${failed.stdout?.toString() ?? ''}${failed.stderr?.toString() ?? ''}` // Rewrite "block-N.ts(line,col)" to the real "file:fenceLine" for triage. - const remapped = out.replace(/block-(\d+)\.ts\((\d+),(\d+)\)/g, (_m, idx: string, ln: string, col: string) => { + const remapped = out.replace(/(?:[^\s:()]*[/\\])?block-(\d+)\.ts\((\d+),(\d+)\)/g, (_m, idx: string, ln: string, col: string) => { const block = fileForBlock.get(`block-${idx}.ts`) if (!block) return `block-${idx}.ts(${ln},${col})` return `${block.file} (block at line ${block.line}, +${ln}:${col})` @@ -139,11 +169,12 @@ try { process.exit(1) } - const ratio = ignored.length / all.length - console.log(`doc-typecheck: ${checked.length} block(s) compiled, ${ignored.length} ignored (${(ratio * 100).toFixed(0)}% opt-out).`) + const ratio = ignored.length / ratioDenominator + const skipped = all.length - ratioDenominator + console.log(`doc-typecheck: ${checked.length} block(s) compiled, ${ignored.length} ignored (${(ratio * 100).toFixed(0)}% opt-out), ${skipped} type-equiv/cordis-catalog (checked elsewhere).`) // Guard against the escape hatch becoming the norm. - if (all.length >= 4 && ratio > 0.5) { - console.error(`doc-typecheck: too many blocks opt out of checking (${ignored.length}/${all.length}). Make them compile or delete them.`) + if (ratioDenominator >= 4 && ratio > 0.5) { + console.error(`doc-typecheck: too many blocks opt out of checking (${ignored.length}/${ratioDenominator}). Make them compile or delete them.`) process.exit(1) } } finally { diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts new file mode 100644 index 0000000000..7479f5306c --- /dev/null +++ b/scripts/gen-cordis-catalog.ts @@ -0,0 +1,463 @@ +/** + * Generate (and verify) the cordis events + services catalog in + * docs/cordis-catalog/events-and-services.md. + * + * The catalog is the WIRING-axis reference: every cordis event a plugin can + * listen to (exact signature + dispatch mode) and every `ctx.` service it + * can call (exact public interface). It complements the core-data-structures + * catalog (the VOCABULARY axis — the types these signatures move around). + * + * The catalog is FULLY GENERATED from source — never hand-edit it. The codebase + * is disciplined enough that a pure-AST pass captures the whole truthful + * surface: every event/service is a string literal that round-trips to a static + * `interface Events` / `interface Context` declaration (no dynamically-named + * events, no runtime-only services). So the committed file is a build artifact + * and a regenerate-and-diff freshness check (`--check`) makes drift structurally + * impossible. Because generation enumerates source rather than checking a + * hand-written subset, a brand-new event cannot be silently undocumented — it + * appears in the next regenerate, and an un-regenerated file fails `--check`. + * + * `tsx scripts/gen-cordis-catalog.ts` → write the catalog + * `tsx scripts/gen-cordis-catalog.ts --check` → exit 1 if the committed file + * is stale (CI / pre-push gate) + * + * The HARNESS tier (the `@deepseek-ai/dsh-*` events + services) is rendered in + * full from source: signature, the `@mode` badge, and the declaration's JSDoc. + * Every harness event MUST carry an `@mode emit|waterfall|parallel` tag — the + * generator hard-errors on a missing tag, and where the signature shape is + * conclusive (a trailing `next: () => …` parameter is structurally a waterfall) + * it asserts the tag agrees and hard-errors on a contradiction. The INHERITED + * tier (cordis core + loader/hmr/timer) is pinned vendor source a plugin author + * also sees; it is rendered tersely (name + one-line + source pointer) from a + * curated table in this script, NOT elevated to the harness tier's prominence. + * + * Signature fences use the ` ```ts cordis-catalog ` info string: doc-typecheck + * recognizes it and skips compilation (the signatures are fragments, not + * standalone-compilable, like the ` ```ts type-equiv ` blocks). + */ + +import { globSync, readFileSync, writeFileSync } from 'node:fs' +import { resolve } from 'node:path' +import ts from 'typescript' + +const root = resolve(import.meta.dirname, '..') +const OUT = 'docs/cordis-catalog/events-and-services.md' + +/** The fenced-block info string for generated signature blocks (skipped by + * doc-typecheck, since a bare signature fragment is not standalone-compilable). */ +const FENCE = 'ts cordis-catalog' + +/** A dispatch mode, rendered as the badge after an event name. */ +type Mode = 'emit' | 'waterfall' | 'parallel' + +/** + * Cross-link map: a type name that appears in a signature → the + * core-data-structures page that documents it (path relative to OUT's folder). + * Hand-curated and catalog-owned, NOT derived from type-equiv.manifest.json — + * that manifest documents the `…Map` symbols (`ContentBlockMap`) while + * signatures reference the derived UNION names (`ContentBlock`), and it lists a + * few symbols on two pages. Here each name resolves to exactly one PRIMARY page. + */ +const LINK_MAP: Record = { + Agent: 'core.md', + ContentBlock: 'core.md', + Message: 'core.md', + MessageSource: 'core.md', + GenerateOptions: 'core.md', + SessionEvent: 'core.md', + StreamChunk: 'llm-streaming.md', + TurnEndReason: 'session.md', + ToolDefinition: 'tools.md', + ToolExecution: 'tools.md', + ToolExecutionResult: 'tools.md', + BashExecRequest: 'bash.md', + BashExecSpec: 'bash.md', + BashRunResult: 'bash.md', + BashTask: 'bash.md', + BashTaskRead: 'bash.md', +} + +/** One harness event, extracted from an `interface Events` block. */ +interface EventEntry { + /** Scoped name, e.g. `agent/request`. */ + name: string + /** The scope prefix, e.g. `agent` (everything before the first `/`). */ + scope: string + /** Full signature text (the method-signature member, JSDoc stripped). */ + signature: string + /** Dispatch mode from the `@mode` tag. */ + mode: Mode + /** Description prose (JSDoc minus the `@mode` tag), one line per paragraph. */ + doc: string + /** Source pointer `packages/…/file.ts:line` of the declaration. */ + source: string +} + +/** One harness service, extracted from an `interface Context` block. */ +interface ServiceEntry { + /** The `ctx.` name, e.g. `llm`. */ + key: string + /** The service class/interface name, e.g. `LlmService`. */ + type: string + /** Whether the service class is abstract (a seam interface). */ + abstract: boolean + /** Class-level JSDoc prose, one line per paragraph. */ + doc: string + /** Public method signatures (bodies stripped), in source order. */ + methods: string[] + /** Source pointer of the class declaration. */ + source: string +} + +/** A terse inherited-tier entry (pinned vendor surface). */ +interface InheritedEntry { + name: string + summary: string + /** Source pointer `vendor/…:line`. */ + source: string +} + +/** Repo-relative source pointer `file:line` for a node's first character. */ +function pointer(rel: string, sf: ts.SourceFile, node: ts.Node): string { + const { line } = sf.getLineAndCharacterOfPosition(node.getStart(sf)) + return `${rel}:${line + 1}` +} + +/** The raw `/** … *​/` JSDoc block immediately preceding a node, or '' if none. */ +function rawJsDoc(text: string, node: ts.Node): string { + const ranges = ts.getLeadingCommentRanges(text, node.getFullStart()) ?? [] + const jsdoc = ranges.filter(r => text.slice(r.pos, r.pos + 3) === '/**').at(-1) + return jsdoc ? text.slice(jsdoc.pos, jsdoc.end) : '' +} + +/** + * Parse a raw JSDoc block into description prose + the `@mode` tag (when + * present). Output obeys the repo's markdown conventions so the generated file + * passes verify-md-wrap: each prose paragraph collapses to ONE physical line, + * and a `-` bullet list is preserved with each item on its own single line + * (continuation lines folded in). `{@link Foo}` unwraps to `Foo`; `@`-tag lines + * other than `@mode` end the current prose run. + */ +function parseJsDoc(raw: string): { doc: string; mode: Mode | null } { + const inner = raw + .replace(/^\/\*\*/, '') + .replace(/\*\/$/, '') + .split('\n') + .map(l => l.replace(/^\s*\*?\s?/, '').replace(/\s+$/, '')) + let mode: Mode | null = null + const blocks: string[] = [] + let para: string[] = [] + let list: string[] = [] + let item: string[] = [] + const join = (parts: string[]): string => parts.join(' ').replace(/\s+/g, ' ').trim() + const flushItem = (): void => { + if (item.length) list.push(join(item)) + item = [] + } + const flushList = (): void => { + flushItem() + if (list.length) blocks.push(list.join('\n')) // one block, items on own lines + list = [] + } + const flushPara = (): void => { + flushList() + if (para.length) blocks.push(join(para)) + para = [] + } + for (const line of inner) { + const m = /^@mode\s+(emit|waterfall|parallel)\s*$/.exec(line) + if (m) { mode = m[1] as Mode; continue } + if (line.startsWith('@')) { flushPara(); continue } // other tags end the prose + if (line.trim() === '') { flushPara(); continue } + if (/^-\s+/.test(line)) { + // A list item starts: a pending paragraph (e.g. an intro line directly + // above the list, no blank between) flushes FIRST so it renders above. + flushItem() + if (para.length) { blocks.push(join(para)); para = [] } + item.push(line) + continue + } + if (item.length) { item.push(line); continue } // continuation of current item + para.push(line) + } + flushPara() + const doc = blocks.join('\n\n').replace(/\{@link\s+([^}]+)\}/g, '$1').trim() + return { doc, mode } +} + +/** Find the `declare module 'cordis'` body in a source file, or null. */ +function cordisModuleBody(sf: ts.SourceFile): ts.ModuleBlock | null { + for (const stmt of sf.statements) { + if (ts.isModuleDeclaration(stmt) && ts.isStringLiteral(stmt.name) && stmt.name.text === 'cordis') { + if (stmt.body && ts.isModuleBlock(stmt.body)) return stmt.body + } + } + return null +} + +/** The signature text of a method-signature member (everything but a body). */ +function memberSignature(member: ts.TypeElement | ts.ClassElement, sf: ts.SourceFile): string { + const full = member.getText(sf) + const body = (member as { body?: ts.Node }).body + const sig = body ? full.slice(0, full.length - body.getText(sf).length) : full + return sig.replace(/\s*;?\s*$/, '').replace(/\s+/g, ' ').trim() +} + +/** Walk every harness `interface Events` block and extract its events. + * `scanRoot` defaults to the repo root; tests pass a fixture dir. */ +export function collectEvents(scanRoot: string = root): EventEntry[] { + const entries: EventEntry[] = [] + for (const rel of globSync('packages/*/*/src/*.ts', { cwd: scanRoot }).sort()) { + const abs = resolve(scanRoot, rel) + const text = readFileSync(abs, 'utf8') + if (!text.includes('interface Events')) continue + const sf = ts.createSourceFile(abs, text, ts.ScriptTarget.Latest, true) + const body = cordisModuleBody(sf) + if (!body) continue + for (const stmt of body.statements) { + if (!ts.isInterfaceDeclaration(stmt) || stmt.name.text !== 'Events') continue + for (const member of stmt.members) { + if (!ts.isMethodSignature(member)) continue + const name = ts.isStringLiteral(member.name) ? member.name.text : member.name.getText(sf) + const signature = memberSignature(member, sf) + const { doc, mode } = parseJsDoc(rawJsDoc(text, member)) + const src = pointer(rel, sf, member) + if (!mode) { + throw new Error(`gen-cordis-catalog: event '${name}' (${src}) is missing an @mode tag. Add '@mode emit|waterfall|parallel' to its JSDoc (see AGENTS.md).`) + } + // Conclusive structural check: a trailing `next: () => …` parameter is a + // waterfall. (emit vs parallel is not structurally distinguishable, so + // it is trusted from the tag.) + const last = member.parameters.at(-1) + const hasNext = !!last && last.name.getText(sf) === 'next' + if (hasNext && mode !== 'waterfall') { + throw new Error(`gen-cordis-catalog: event '${name}' (${src}) has a trailing 'next' parameter (structurally a waterfall) but is tagged '@mode ${mode}'. Fix the tag or the signature.`) + } + if (!hasNext && mode === 'waterfall') { + throw new Error(`gen-cordis-catalog: event '${name}' (${src}) is tagged '@mode waterfall' but has no trailing 'next' parameter. A waterfall delegates via next().`) + } + entries.push({ name, scope: name.split('/')[0] ?? name, signature, mode, doc, source: src }) + } + } + } + return entries +} + +/** Walk every harness `interface Context` block + its service class. + * `scanRoot` defaults to the repo root; tests pass a fixture dir. */ +export function collectServices(scanRoot: string = root): ServiceEntry[] { + const entries: ServiceEntry[] = [] + for (const rel of globSync('packages/*/*/src/index.ts', { cwd: scanRoot }).sort()) { + const abs = resolve(scanRoot, rel) + const text = readFileSync(abs, 'utf8') + if (!text.includes('interface Context')) continue + const sf = ts.createSourceFile(abs, text, ts.ScriptTarget.Latest, true) + const body = cordisModuleBody(sf) + if (!body) continue + // The ctx key → type mapping(s) declared in this file's interface Context. + const keyToType = new Map() + for (const stmt of body.statements) { + if (!ts.isInterfaceDeclaration(stmt) || stmt.name.text !== 'Context') continue + for (const member of stmt.members) { + if (!ts.isPropertySignature(member) || !member.type) continue + const key = member.name.getText(sf) + keyToType.set(key, member.type.getText(sf)) + } + } + if (keyToType.size === 0) continue + // Find each service class declared in the same file and emit an entry. + for (const [key, type] of keyToType) { + const cls = sf.statements.find( + (s): s is ts.ClassDeclaration => ts.isClassDeclaration(s) && s.name?.text === type, + ) + if (!cls) continue // a Pick-mixin member (e.g. timer helpers), not a class here + const abstract = cls.modifiers?.some(m => m.kind === ts.SyntaxKind.AbstractKeyword) ?? false + const methods: string[] = [] + for (const member of cls.members) { + if (!ts.isMethodDeclaration(member)) continue + // Only the PUBLIC callable surface a `ctx.` consumer sees. Drop + // private/protected (a protected method like `notifyTaskDone` is a + // subclass hook, not something a plugin calls through `ctx.bash`) and + // static (not reachable through the instance). + const nonPublic = member.modifiers?.some(m => + m.kind === ts.SyntaxKind.PrivateKeyword + || m.kind === ts.SyntaxKind.ProtectedKeyword + || m.kind === ts.SyntaxKind.StaticKeyword) + || ts.isPrivateIdentifier(member.name) + if (nonPublic) continue + const memberName = member.name.getText(sf) + if (memberName.startsWith('[')) continue // computed/symbol members + methods.push(memberSignature(member, sf)) + } + entries.push({ + key, + type, + abstract, + doc: parseJsDoc(rawJsDoc(text, cls)).doc, + methods, + source: pointer(rel, sf, cls), + }) + } + } + return entries.sort((a, b) => a.key.localeCompare(b.key)) +} + +/** + * The inherited tier — cordis core + loader/hmr/timer. Curated, terse, and + * hand-summarized because (a) it is pinned vendor source that changes only on a + * deliberate vendor sync, (b) the cordis-core `Context` mixes true ctx members + * with non-service fields (`root`, `baseUrl`, `logger`) that a blind walk would + * wrongly surface as services, and (c) the internal/* events carry no JSDoc to + * render. Source pointers are verified against vendor by `verify-md-links`' + * sibling check is N/A; keep them current on a vendor bump. + */ +const INHERITED_EVENTS: InheritedEntry[] = [ + { name: 'internal/plugin', summary: 'A plugin fiber was created.', source: 'vendor/cordis/src/events.ts:197' }, + { name: 'internal/status', summary: 'A fiber changed lifecycle state.', source: 'vendor/cordis/src/events.ts:198' }, + { name: 'internal/service', summary: 'Interception hook for a service binding (no core producer).', source: 'vendor/cordis/src/events.ts:199' }, + { name: 'internal/update', summary: 'Waterfall: a fiber config update is being applied.', source: 'vendor/cordis/src/events.ts:200' }, + { name: 'internal/get', summary: 'Waterfall: a service is being read from the store.', source: 'vendor/cordis/src/events.ts:201' }, + { name: 'internal/set', summary: 'Waterfall: a service is being written to the store.', source: 'vendor/cordis/src/events.ts:202' }, + { name: 'internal/listener', summary: 'A listener was registered.', source: 'vendor/cordis/src/events.ts:203' }, + { name: 'internal/dispatch', summary: 'An event is being dispatched to listeners.', source: 'vendor/cordis/src/events.ts:204' }, + { name: 'hmr/change', summary: 'A watched source file changed on disk.', source: 'vendor/hmr/src/index.ts:20' }, + { name: 'hmr/reload', summary: 'Plugins are being reloaded after a change.', source: 'vendor/hmr/src/index.ts:21' }, + { name: 'exit', summary: 'The process is exiting on a signal.', source: 'vendor/loader/src/index.ts:23' }, + { name: 'loader/config-update', summary: 'The loader config tree changed.', source: 'vendor/loader/src/index.ts:24' }, + { name: 'loader/entry-init', summary: 'A config entry is being initialized.', source: 'vendor/loader/src/index.ts:25' }, + { name: 'loader/partial-dispose', summary: 'An entry is being partially disposed on reload.', source: 'vendor/loader/src/index.ts:26' }, + { name: 'loader/patch-context', summary: 'A context is being patched during a reload.', source: 'vendor/loader/src/index.ts:27' }, +] + +const INHERITED_SERVICES: InheritedEntry[] = [ + { name: 'ctx.on / ctx.once', summary: 'Register an event listener (disposable).', source: 'vendor/cordis/src/events.ts:29' }, + { name: 'ctx.emit / ctx.parallel / ctx.serial / ctx.bail / ctx.waterfall', summary: 'Dispatch an event (sync / awaited / first-non-nullish / veto-chain).', source: 'vendor/cordis/src/events.ts:29' }, + { name: 'ctx.plugin / ctx.inject', summary: 'Load a plugin / declare required services.', source: 'vendor/cordis/src/registry.ts:144' }, + { name: 'ctx.effect', summary: 'Register a disposable side effect tied to the fiber.', source: 'vendor/cordis/src/fiber.ts:9' }, + { name: 'ctx.get / ctx.set / ctx.provide / ctx.accessor / ctx.mixin', summary: 'Low-level service-store access and binding.', source: 'vendor/cordis/src/reflect.ts:7' }, + { name: 'ctx.extend / ctx.isolate / ctx.intercept', summary: 'Derive a child context (scoped services / isolation / interception).', source: 'vendor/cordis/src/context.ts:35' }, + { name: 'ctx.root / ctx.scope / ctx.fiber / ctx.registry / ctx.reflect / ctx.events / ctx.logger', summary: 'Ambient handles onto the running context graph.', source: 'vendor/cordis/src/context.ts:16' }, + { name: 'ctx.timer (+ interval / timeout / throttle / debounce / setTimeout / setInterval)', summary: 'Disposable timer helpers. The `timer` key is provided at runtime; the six helpers are mixed onto ctx directly (declared via Pick).', source: 'vendor/timer/src/index.ts:4' }, + { name: 'ctx.loader', summary: 'The config Loader that booted the app (present under the loader).', source: 'vendor/loader/src/index.ts:30' }, + { name: 'ctx.hmr', summary: 'The hot-module-reload watcher (present under the hmr plugin).', source: 'vendor/hmr/src/index.ts:15' }, +] + +/** Render the cross-link "Types:" line for a signature, or '' if none apply. */ +function typeLinks(signature: string): string { + const seen = new Set() + for (const name of Object.keys(LINK_MAP)) { + if (new RegExp(`\\b${name}\\b`).test(signature)) seen.add(name) + } + if (seen.size === 0) return '' + const links = [...seen].sort().map(n => `[${n}](../core-data-structures/${LINK_MAP[n]})`) + return `Types: ${links.join(' · ')}` +} + +/** Render one harness event entry. */ +function renderEvent(e: EventEntry): string[] { + const out = [`#### \`${e.name}\` — ${e.mode}`, ''] + if (e.doc) out.push(e.doc, '') + out.push('```' + FENCE, e.signature, '```', '') + const links = typeLinks(e.signature) + if (links) out.push(links, '') + out.push(`Source: [\`${e.source}\`](../../${e.source.split(':')[0]})`, '') + return out +} + +/** Render one harness service entry. */ +function renderService(s: ServiceEntry): string[] { + const kind = s.abstract ? ' (abstract seam)' : '' + const out = [`### \`ctx.${s.key}\` — \`${s.type}\`${kind}`, ''] + if (s.doc) out.push(s.doc, '') + if (s.methods.length) { + out.push('```' + FENCE, ...s.methods, '```', '') + const links = typeLinks(s.methods.join('\n')) + if (links) out.push(links, '') + } + out.push(`Source: [\`${s.source}\`](../../${s.source.split(':')[0]})`, '') + return out +} + +/** Render the full catalog (pure, deterministic given sorted inputs). */ +function render(events: EventEntry[], services: ServiceEntry[]): string { + const lines: string[] = [ + '', + '', + '# Cordis Events & Services Catalog', + '', + 'An index reference to the **wiring** a plugin author works against: every cordis event you can listen to (exact signature + dispatch mode) and every `ctx.` service you can call (exact public interface). It complements [core-data-structures/](../core-data-structures/core.md), which catalogs the *data structures* these signatures move around — this page is the verbs, that page is the nouns.', + '', + 'This file is GENERATED from source (`scripts/gen-cordis-catalog.ts`) and verified fresh by `pnpm run verify-cordis-catalog` (part of `doc-sync`) — do not edit it by hand. Signature blocks use a `ts cordis-catalog` fence (skipped by doc-typecheck, since a bare signature is not standalone-compilable). Type names in a signature link to the page that documents them.', + '', + 'The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary this repo owns. The **inherited tier** at the end is the cordis-core + loader/hmr/timer surface a plugin also sees — pinned vendor source, summarized tersely.', + '', + '## Events', + '', + `Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets \`next()\` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares ${events.length} events across ${new Set(events.map(e => e.scope)).size} scopes.`, + '', + ] + const scopes = [...new Set(events.map(e => e.scope))].sort() + for (const scope of scopes) { + lines.push(`### \`${scope}/*\``, '') + for (const e of events.filter(x => x.scope === scope).sort((a, b) => a.name.localeCompare(b.name))) { + lines.push(...renderEvent(e)) + } + } + lines.push( + '## Services', + '', + `The ${services.length} \`ctx.\` services the harness provides. An abstract seam (e.g. \`ctx.bash\`) is implemented by a separate package; the interface is what consumers code against.`, + '', + ) + for (const s of services) lines.push(...renderService(s)) + lines.push( + '## Inherited tier (cordis core + loader/hmr/timer)', + '', + 'The framework surface every plugin inherits, beyond the harness vocabulary above. This is pinned vendor source ([vendoring policy](../../vendor/README.md)); it is summarized here so the catalog is a complete picture of what `ctx` and the event bus offer, without elevating framework internals to the harness tier\'s prominence.', + '', + '### Inherited events', + '', + ) + for (const e of INHERITED_EVENTS) { + lines.push(`- \`${e.name}\` — ${e.summary} ([\`${e.source}\`](../../${e.source.split(':')[0]}))`) + } + lines.push('', '### Inherited `ctx` members', '') + for (const s of INHERITED_SERVICES) { + lines.push(`- \`${s.name}\` — ${s.summary} ([\`${s.source}\`](../../${s.source.split(':')[0]}))`) + } + lines.push('') + return lines.join('\n') +} + +/** CLI entry: `--write` (default) writes the catalog, `--check` fails if stale. + * Guarded behind an entry-point check so importing this module for tests neither + * regenerates the committed file nor calls process.exit. */ +function main(): void { + const content = render(collectEvents(), collectServices()) + if (process.argv.includes('--check')) { + let committed: string | null = null + try { + committed = readFileSync(resolve(root, OUT), 'utf8') + } catch { + // Only ENOENT (not yet generated) is expected; a present-but-unreadable + // file is not a state this repo produces. Either way the remedy is the + // same — regenerate — so treat a read failure as "stale". + committed = null + } + if (committed === content) { + console.log(`gen-cordis-catalog: ${OUT} is up to date.`) + process.exit(0) + } + console.error(`gen-cordis-catalog: ${OUT} is stale. Run \`pnpm run gen-cordis-catalog\` and commit ${OUT}.`) + process.exit(1) + } + + writeFileSync(resolve(root, OUT), content) + console.log(`gen-cordis-catalog: wrote ${OUT}.`) +} + +// Run only when invoked as a script, not when imported by a test. +if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { + main() +} diff --git a/scripts/gen-module-graph.ts b/scripts/gen-module-graph.ts index 6314b39399..ebaf7d5db8 100644 --- a/scripts/gen-module-graph.ts +++ b/scripts/gen-module-graph.ts @@ -4,7 +4,7 @@ * The architectural shape of the harness lives implicitly in each package's * `peerDependencies` — the canonical runtime-dependency signal (devDeps mirror * these as `workspace:^` plus test-only extras, which would add noise). This - * script reads every `packages/* /package.json`, keeps only the + * script reads every `packages/* /* /package.json`, keeps only the * `@deepseek-ai/dsh-*` peer edges (dropping the `cordis` peer), and renders a * GitHub-viewable Mermaid graph plus a dependency table. * @@ -34,7 +34,7 @@ interface Pkg { /** Read every workspace package and its `@deepseek-ai/dsh-*` peer edges. */ function collect(): Pkg[] { const pkgs: Pkg[] = [] - for (const rel of globSync('packages/*/package.json', { cwd: root })) { + for (const rel of globSync('packages/*/*/package.json', { cwd: root })) { const json = JSON.parse(readFileSync(resolve(root, rel), 'utf8')) as { name: string peerDependencies?: Record diff --git a/scripts/publint-all.ts b/scripts/publint-all.ts index 3a38be3693..df13d87caa 100644 --- a/scripts/publint-all.ts +++ b/scripts/publint-all.ts @@ -1,28 +1,22 @@ import { execFileSync } from 'node:child_process' +import { readdirSync } from 'node:fs' import { resolve } from 'node:path' -// publint every publishable package (vendor/ is private upstream code and -// examples/ are not packages; both are out of scope). -const packages = [ - 'packages/llm', - 'packages/session', - 'packages/session-persistence', - 'packages/session-persistence-jsonl', - 'packages/session-persistence-sqlite', - 'packages/system-prompt', - 'packages/tools', - 'packages/agent', - 'packages/agent-loop', - 'packages/bash', - 'packages/llm-deepseek', - 'packages/llm-pi-ai', - 'packages/bash-local', - 'packages/tool-bash', - 'packages/invariants', - 'packages/acp', -] - +// publint every harness package. Packages live at packages// +// (the group dirs — core/llm/bash/… — are pure containers); vendor/ is private +// upstream code and examples/ are not packages, both out of scope. Derived +// from the hierarchy so a new package needs no edit here. const root = resolve(import.meta.dirname, '..') +const packagesRoot = resolve(root, 'packages') + +const packages = readdirSync(packagesRoot, { withFileTypes: true }) + .filter(group => group.isDirectory()) + .flatMap(group => + readdirSync(resolve(packagesRoot, group.name), { withFileTypes: true }) + .filter(pkg => pkg.isDirectory()) + .map(pkg => `packages/${group.name}/${pkg.name}`), + ) + for (const path of packages) { execFileSync('node_modules/.bin/publint', [path], { cwd: root, stdio: 'inherit' }) } diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json new file mode 100644 index 0000000000..f46c4fca6b --- /dev/null +++ b/scripts/type-equiv.manifest.json @@ -0,0 +1,40 @@ +{ + "comment": "Maps each ` ```ts type-equiv ` block (by doc + declared symbol) to the source symbol it must match verbatim. verify-type-equiv.ts enforces a 1:1 correspondence: every type-equiv block has exactly one entry here, and every entry resolves to exactly one block. Add an entry when you add a type-equiv block; remove it when you remove the block.", + "entries": [ + { "doc": "docs/core-data-structures/core.md", "symbol": "Branded", "source": "packages/util/brand/src/index.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "ContentBlockMap", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "Message", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "MessageSourceMap", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "FinishReasonMap", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "GenerateOptions", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "ToolSchema", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "SessionEvent", "source": "packages/core/session/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "Agent", "source": "packages/core/agent/src/types.ts" }, + + { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "StreamChunk", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "TokenUsage", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "ContentBlockMap", "source": "packages/llm/llm/src/types.ts" }, + + { "doc": "docs/core-data-structures/session.md", "symbol": "SessionEventMap", "source": "packages/core/session/src/types.ts" }, + { "doc": "docs/core-data-structures/session.md", "symbol": "SessionEvent", "source": "packages/core/session/src/types.ts" }, + { "doc": "docs/core-data-structures/session.md", "symbol": "TurnTriggerMap", "source": "packages/core/session/src/types.ts" }, + { "doc": "docs/core-data-structures/session.md", "symbol": "TurnEndReasonMap", "source": "packages/core/session/src/types.ts" }, + + { "doc": "docs/core-data-structures/persistence.md", "symbol": "SessionHeader", "source": "packages/core/session/src/types.ts" }, + { "doc": "docs/core-data-structures/persistence.md", "symbol": "CreateSessionOptions", "source": "packages/core/session/src/types.ts" }, + + { "doc": "docs/core-data-structures/tools.md", "symbol": "ToolDefinition", "source": "packages/core/tools/src/index.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "SchemaProp", "source": "packages/core/tools/src/schema.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "SchemaSpec", "source": "packages/core/tools/src/schema.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "InferArgs", "source": "packages/core/tools/src/schema.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "ToolExecution", "source": "packages/core/tools/src/index.ts" }, + { "doc": "docs/core-data-structures/tools.md", "symbol": "ToolExecutionResult", "source": "packages/core/tools/src/index.ts" }, + + { "doc": "docs/core-data-structures/bash.md", "symbol": "BashExecRequest", "source": "packages/bash/bash/src/types.ts" }, + { "doc": "docs/core-data-structures/bash.md", "symbol": "BashExecSpec", "source": "packages/bash/bash/src/types.ts" }, + { "doc": "docs/core-data-structures/bash.md", "symbol": "BashRunResult", "source": "packages/bash/bash/src/types.ts" }, + { "doc": "docs/core-data-structures/bash.md", "symbol": "CollectedOutput", "source": "packages/bash/bash/src/types.ts" }, + { "doc": "docs/core-data-structures/bash.md", "symbol": "BashTask", "source": "packages/bash/bash/src/types.ts" }, + { "doc": "docs/core-data-structures/bash.md", "symbol": "BashTaskRead", "source": "packages/bash/bash/src/types.ts" } + ] +} diff --git a/scripts/verify-doc-refs.ts b/scripts/verify-doc-refs.ts new file mode 100644 index 0000000000..322368b724 --- /dev/null +++ b/scripts/verify-doc-refs.ts @@ -0,0 +1,96 @@ +/** + * Doc-sync gate: verify that doc references written in TypeScript COMMENTS + * resolve to a file that exists. Source comments cite docs by root-relative + * prose path — `see docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md`, + * `docs/architecture.md § plugin checklist`. `verify-md-links` parses Markdown + * link AST and never sees these, so a doc rename or move could silently orphan + * a `.ts` comment that points at it. The RFC classification reorg + * ([the classification RFC](../docs/rfc/implemented/process/2026-06-20-rfc-classification.md)) + * is the motivating case: it moved every RFC under a `{class}/` folder, and + * several `.ts` doc comments cite RFC paths that changed. + * + * Detection is a token scan, NOT an AST walk: doc refs live in free prose inside + * comments, not in a structured form. We match `docs/.md` tokens and + * REQUIRE the `.md` extension, so extensionless prose (`docs/postmortem/0001`, + * `docs/architecture.md § plugin checklist` — the section suffix is outside the + * token) is left alone rather than misread as a path. Each token is resolved + * ROOT-RELATIVE (the way the comments are written) and must exist on disk. This + * is checker, not fixer: it reports and never rewrites. + * + * Scope is repo-authored TypeScript under `packages/**` and `examples/**`, + * excluding built output (`lib/`, `*.d.ts`) and `vendor/` (pinned upstream + * source we do not own). The scan is purely textual, so it does not distinguish + * a token in a comment from one in a string literal — a `docs/….md` string in + * code is checked too, which is harmless (such a path should resolve anyway). + * + * Run: `tsx scripts/verify-doc-refs.ts`. + */ + +import { existsSync, readFileSync } from 'node:fs' +import { relative, resolve } from 'node:path' +import { glob } from 'node:fs/promises' + +const root = resolve(import.meta.dirname, '..') + +/** Repo-authored TypeScript that may cite docs in comments. */ +const PATTERNS = ['packages/**/*.ts', 'examples/**/*.ts'] + +/** Paths excluded from the scan: built output and vendored upstream source. */ +const isExcluded = (p: string): boolean => + p.includes('/lib/') || p.endsWith('.d.ts') || p.startsWith('vendor/') + +/** + * Match a `docs/…​.md` reference token. The `.md` extension is required so a + * bare `docs/postmortem/0001` (no extension) does not register as a path. The + * character class stops at whitespace, backticks, parens, and the section sign, + * so trailing prose (`… .md § plugin checklist`) is not swallowed into the path. + */ +const DOC_REF = /\bdocs\/[A-Za-z0-9._/-]+\.md/g + +/** A broken doc reference: a root-relative `docs/….md` token with no file. */ +interface Violation { + file: string + /** 1-based line where the reference appears. */ + line: number + ref: string +} + +/** Find every broken `docs/….md` reference in one TypeScript file. */ +function findViolations(absPath: string): Violation[] { + const file = relative(root, absPath) + const source = readFileSync(absPath, 'utf8') + const out: Violation[] = [] + const lines = source.split('\n') + for (let i = 0; i < lines.length; i++) { + const line = lines[i] + if (line === undefined) continue + for (const m of line.matchAll(DOC_REF)) { + const ref = m[0] + if (!existsSync(resolve(root, ref))) { + out.push({ file, line: i + 1, ref }) + } + } + } + return out +} + +const all: Violation[] = [] +let checked = 0 +for (const pattern of PATTERNS) { + for await (const match of glob(pattern, { cwd: root })) { + if (isExcluded(match)) continue + checked++ + all.push(...findViolations(resolve(root, match))) + } +} + +if (all.length === 0) { + console.log(`verify-doc-refs: ${checked} file(s) checked, all docs/*.md references resolve.`) + process.exit(0) +} + +console.error('verify-doc-refs: broken docs/*.md references found in source comments (target does not exist):') +for (const v of all) { + console.error(` ${v.file}:${v.line} ${v.ref}`) +} +process.exit(1) diff --git a/scripts/verify-event-taxonomy.ts b/scripts/verify-event-taxonomy.ts deleted file mode 100644 index f133d9cf46..0000000000 --- a/scripts/verify-event-taxonomy.ts +++ /dev/null @@ -1,114 +0,0 @@ -/** - * Doc-sync gate (doc-sync-enforcement RFC, part 2): verify the event-taxonomy table in - * docs/architecture.md against the events actually declared in source. - * - * The table duplicates the `declare module 'cordis' { interface Events }` - * blocks across packages/* /src. This script extracts both sets of event names - * and asserts they match exactly — every declared event appears in the table, - * and the table names no event that isn't declared. Verify, don't generate - * (per the RFC): the table keeps its hand-written Mode/Purpose columns; only - * the set of names is checked. - * - * Run: `tsx scripts/verify-event-taxonomy.ts`. - */ - -import { readFileSync } from 'node:fs' -import { join, relative, resolve } from 'node:path' -import { glob } from 'node:fs/promises' - -const root = resolve(import.meta.dirname, '..') - -/** - * Remove `/* *​/` block comments and `//` line comments from TS source. Used to - * de-risk the brace walk in {@link declaredEvents} — a JSDoc `{@link}` tag would - * otherwise throw off the `{`/`}` depth counter. Good enough for our own source - * (no string literals contain `//` or comment-like brace sequences in an Events - * block); it is not a general tokenizer. - */ -function stripComments(text: string): string { - return text - .replace(/\/\*[\s\S]*?\*\//g, '') - .replace(/(^|[^:])\/\/.*$/gm, '$1') -} - -/** - * Event names declared in source: the keys inside every `interface Events` - * block under packages/* /src. A declared event is a quoted `'scope/name'(` - * method signature at the start of a line within such a block. - */ -async function declaredEvents(): Promise> { - const found = new Map() - for await (const match of glob('packages/*/src/**/*.ts', { cwd: root })) { - const abs = resolve(root, match) - // Strip comments first so a JSDoc `{@link …}` tag (or a `// {` line) inside - // an Events block can't unbalance the brace walk below. Event names live in - // code, never in comments, so this loses nothing. - const text = stripComments(readFileSync(abs, 'utf8')) - // Walk `interface Events {` blocks brace-balanced and pull quoted keys. - const re = /interface\s+Events\s*\{/g - let m: RegExpExecArray | null - while ((m = re.exec(text)) !== null) { - let depth = 1 - let i = m.index + m[0].length - const start = i - while (i < text.length && depth > 0) { - const ch = text[i] - if (ch === '{') depth++ - else if (ch === '}') depth-- - i++ - } - const body = text.slice(start, i - 1) - // A declaration is a quoted event name followed by `(` (method form). - for (const k of body.matchAll(/['"]([a-z][a-z-]*\/[a-z-]+)['"]\s*\(/g)) { - const name = k[1] - if (name) found.set(name, relative(root, abs)) - } - } - } - return found -} - -/** Event names referenced in the architecture-doc taxonomy table (in `code`). */ -function tableEvents(): Set { - const text = readFileSync(join(root, 'docs/architecture.md'), 'utf8') - const lines = text.split('\n') - const heading = lines.findIndex(l => /^###\s+Event taxonomy/.test(l)) - if (heading === -1) throw new Error('verify-event-taxonomy: "### Event taxonomy" heading not found') - const names = new Set() - for (let i = heading + 1; i < lines.length; i++) { - const line = lines[i] ?? '' - if (/^###\s/.test(line)) break // next section ends the table - if (!line.includes('|')) continue - for (const code of line.matchAll(/`([^`]+)`/g)) { - // A cell may read "`a/b` / `c/d` (pkg)" — pull each scoped name. - for (const name of (code[1] ?? '').matchAll(/[a-z][a-z-]*\/[a-z-]+/g)) names.add(name[0]) - } - } - return names -} - -const declared = await declaredEvents() -const table = tableEvents() - -const declaredNames = new Set(declared.keys()) -const missingFromTable = [...declaredNames].filter(n => !table.has(n)).sort() -const missingFromSource = [...table].filter(n => !declaredNames.has(n)).sort() - -if (missingFromTable.length === 0 && missingFromSource.length === 0) { - console.log(`verify-event-taxonomy: ${declaredNames.size} events match the architecture-doc table.`) - process.exit(0) -} - -if (missingFromTable.length > 0) { - console.error('verify-event-taxonomy: declared in source but MISSING from the docs/architecture.md table:') - for (const n of missingFromTable) { - console.error(` ${n} (declared in ${declared.get(n) ?? '?'})`) - } -} -if (missingFromSource.length > 0) { - console.error('verify-event-taxonomy: named in the table but NOT declared in source (stale doc):') - for (const n of missingFromSource) { - console.error(` ${n}`) - } -} -process.exit(1) diff --git a/scripts/verify-md-links.ts b/scripts/verify-md-links.ts index 35e42452ca..57bb824b32 100644 --- a/scripts/verify-md-links.ts +++ b/scripts/verify-md-links.ts @@ -48,7 +48,8 @@ const root = resolve(import.meta.dirname, '..') const PATTERNS = [ 'README.md', 'docs/**/*.md', - 'packages/*/README.md', + 'packages/*/*.md', + 'packages/*/*/*.md', 'AGENTS.md', 'packages/AGENTS.md', '.agents/skills/**/*.md', diff --git a/scripts/verify-md-wrap.ts b/scripts/verify-md-wrap.ts index 169b06b15b..f8acb26d78 100644 --- a/scripts/verify-md-wrap.ts +++ b/scripts/verify-md-wrap.ts @@ -18,7 +18,7 @@ * A wrapped paragraph inside a list item or blockquote is still a `paragraph` * node, so those are caught too. Scope mirrors doc-typecheck plus the two * AGENTS.md files that doc-sync does NOT otherwise cover (the convention itself - * lives there): README.md, docs/** /*.md, packages/* /README.md, AGENTS.md, + * lives there): README.md, docs/** /*.md, packages/* /*.md, AGENTS.md, * packages/AGENTS.md. The root and packages/ CLAUDE.md are symlinks to the * AGENTS.md files, so they are deduped by real path. * @@ -36,7 +36,7 @@ import type { Nodes } from 'mdast' const root = resolve(import.meta.dirname, '..') /** Files to check: doc-typecheck's scope plus the AGENTS.md pair. */ -const PATTERNS = ['README.md', 'docs/**/*.md', 'packages/*/README.md', 'AGENTS.md', 'packages/AGENTS.md'] +const PATTERNS = ['README.md', 'docs/**/*.md', 'packages/*/*.md', 'packages/*/*/*.md', 'AGENTS.md', 'packages/AGENTS.md'] /** A located hard-wrap: a prose paragraph spanning more than one source line. */ interface Violation { diff --git a/scripts/verify-node-next-types.ts b/scripts/verify-node-next-types.ts new file mode 100644 index 0000000000..7883a855c3 --- /dev/null +++ b/scripts/verify-node-next-types.ts @@ -0,0 +1,160 @@ +/** + * Verify that built package declarations are consumable by a standard external + * TypeScript ESM project using NodeNext resolution. + * + * Run after `pnpm run build` has emitted declaration files under package + * `lib/types` directories. + */ + +import { execFileSync } from 'node:child_process' +import { existsSync, globSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' +import { dirname, resolve } from 'node:path' + +const root = resolve(import.meta.dirname, '..') + +interface ExportTarget { + types?: string +} + +interface PackageManifest { + name?: string + types?: string + exports?: Record +} + +interface WorkspacePackage { + dir: string + name: string + manifest: PackageManifest +} + +function readPackage(path: string): WorkspacePackage | null { + const manifest = JSON.parse(readFileSync(path, 'utf8')) as PackageManifest + if (!manifest.name) return null + return { dir: dirname(path), name: manifest.name, manifest } +} + +function workspacePackages(): WorkspacePackage[] { + return [ + ...globSync('vendor/*/package.json', { cwd: root }), + ...globSync('packages/*/*/package.json', { cwd: root }), + ] + .map(path => readPackage(resolve(root, path))) + .filter(pkg => pkg !== null) + .sort((a, b) => a.name.localeCompare(b.name)) +} + +const declarationSpecifierPattern = /(?:from\s*|import\s*\(\s*|import\s+|declare\s+module\s*)["'](\.{0,2}(?:\/[^"']*)?)["']/g +const hasExtension = /\.[^/.]+$/ + +function relativeSpecifiersMissingExtensions(): string[] { + const errors: string[] = [] + const files = [ + ...globSync('vendor/*/lib/types/**/*.d.ts', { cwd: root }), + ...globSync('packages/*/*/lib/types/**/*.d.ts', { cwd: root }), + ].sort() + + for (const file of files) { + const text = readFileSync(resolve(root, file), 'utf8') + for (const match of text.matchAll(declarationSpecifierPattern)) { + const specifier = match[1] + if (!specifier) continue + const isRelative = specifier === '.' || specifier.startsWith('./') || specifier.startsWith('../') + if (isRelative && !hasExtension.test(specifier)) errors.push(`${file}: ${specifier}`) + } + } + + return errors +} + +function publicSpecifiers(pkg: WorkspacePackage): string[] { + const specifiers = new Set() + if (pkg.manifest.types) specifiers.add(pkg.name) + + for (const [key, target] of Object.entries(pkg.manifest.exports ?? {})) { + if (key.includes('*') || key === './package.json') continue + if (typeof target !== 'object' || target === null || !target.types) continue + specifiers.add(key === '.' ? pkg.name : `${pkg.name}/${key.slice(2)}`) + } + + return [...specifiers].sort() +} + +function linkPackage(pkg: WorkspacePackage, nodeModules: string): void { + const parts = pkg.name.split('/') + const link = resolve(nodeModules, ...parts) + mkdirSync(dirname(link), { recursive: true }) + symlinkSync(pkg.dir, link, 'dir') +} + +const packages = workspacePackages() +const badSpecifiers = relativeSpecifiersMissingExtensions() +if (badSpecifiers.length > 0) { + console.error('verify-node-next-types: declaration files still contain relative specifiers without file extensions.') + console.error(badSpecifiers.join('\n')) + process.exit(1) +} + +const missingOutputs = packages + .filter(pkg => pkg.manifest.types && !existsSync(resolve(pkg.dir, pkg.manifest.types))) + .map(pkg => `${pkg.name}: missing ${pkg.manifest.types}`) + +if (missingOutputs.length > 0) { + console.error('verify-node-next-types: build outputs are missing; run `pnpm run build` first.') + console.error(missingOutputs.join('\n')) + process.exit(1) +} + +const tmp = mkdtempSync(resolve(root, '.node-next-types-')) +let failed = false + +try { + const nodeModules = resolve(tmp, 'node_modules') + mkdirSync(nodeModules, { recursive: true }) + for (const pkg of packages) linkPackage(pkg, nodeModules) + + const rootTypes = resolve(root, 'node_modules/@types/node') + if (existsSync(rootTypes)) { + const typesDir = resolve(nodeModules, '@types') + mkdirSync(typesDir, { recursive: true }) + symlinkSync(rootTypes, resolve(typesDir, 'node'), 'dir') + } + + writeFileSync(resolve(tmp, 'package.json'), `${JSON.stringify({ type: 'module', private: true }, null, 2)}\n`) + writeFileSync(resolve(tmp, 'tsconfig.json'), `${JSON.stringify({ + compilerOptions: { + target: 'es2024', + module: 'NodeNext', + moduleResolution: 'NodeNext', + strict: true, + // Third-party SDK declarations can have their own lib-check noise under a + // symlinked temp install. The explicit scan above owns our regression: + // relative specifiers without file extensions in built declarations. + skipLibCheck: true, + preserveSymlinks: true, + noEmit: true, + types: ['node'], + }, + include: ['index.ts'], + }, null, 2)}\n`) + + const imports = packages.flatMap(publicSpecifiers) + .map((specifier, index) => `import * as mod${index} from ${JSON.stringify(specifier)};\nvoid mod${index};`) + .join('\n') + writeFileSync(resolve(tmp, 'index.ts'), `${imports}\n`) + + execFileSync(resolve(root, 'node_modules/.bin/tsc'), ['-p', resolve(tmp, 'tsconfig.json'), '--pretty', 'false'], { + cwd: root, + stdio: 'pipe', + }) + console.log(`verify-node-next-types: ${packages.length} workspace package declaration surface(s) compile under NodeNext.`) +} catch (error: unknown) { + failed = true + const output = error as { stdout?: Buffer; stderr?: Buffer } + console.error('verify-node-next-types: NodeNext consumer typecheck failed.\n') + console.error(`${output.stdout?.toString() ?? ''}${output.stderr?.toString() ?? ''}`) +} finally { + rmSync(tmp, { recursive: true, force: true }) +} + +if (failed) process.exit(1) diff --git a/scripts/verify-package-paths.ts b/scripts/verify-package-paths.ts new file mode 100644 index 0000000000..7bec754dba --- /dev/null +++ b/scripts/verify-package-paths.ts @@ -0,0 +1,167 @@ +/** + * Doc-sync gate: catch DRIFTED `packages/` references — a path to a + * package that has MOVED, written as prose in Markdown or in a TypeScript + * comment/string. Docs and comments cite package locations by root-relative + * path (`packages/core/tools/src/index.ts`, `see packages/ui/acp`); + * `verify-md-links` only parses Markdown LINK targets and `verify-doc-refs` + * only checks `docs/*.md` tokens, so a `packages/…` path sitting in backtick + * prose or a code comment goes unchecked. The package-hierarchy reorg is the + * motivating case: it moved every package under a `{group}/` folder, so a stale + * `packages/tools` (now `packages/core/tools`) reads fine to a human but points + * at nothing. + * + * The check is drift-scoped, NOT a blanket existence test: a broken + * `packages/` token is a violation ONLY when one of its path segments is + * the directory name of a package that actually exists on disk — i.e. the + * package is real and the path is merely stale. A token naming a package that + * exists NOWHERE (`packages/code-runtime` in a forward-looking proposal, an + * illustrative `packages//` skeleton) is left alone: this gate reports + * MOVED paths, not hypothetical or future ones, so it applies uniformly to + * proposed/implemented/rejected docs without per-lifecycle exclusions. This is + * checker, not fixer: it reports and never rewrites. + * + * Detection is a token scan, NOT an AST walk: package refs live in free prose, + * backticks, and comments. We match `packages/` tokens whose path is made + * of plain path characters, so a glob, a ``, or a `{brace,expansion}` + * terminates the match before those chars and is never probed. + * + * Scope mirrors the other doc gates plus repo-authored TypeScript: Markdown + * across README/docs/packages/AGENTS, and `.ts` under packages/** and + * examples/** (excluding built `lib/`, `*.d.ts`, and vendored upstream source). + * A reference to a package's build OUTPUT (`packages///lib/…`, + * e.g. `packages/ui/acp-agent/lib/bin.js` cited by a built-bin smoke) is also + * skipped — it is emitted only by `pnpm run build`, which CI runs AFTER this + * gate, so flagging it would be a false positive on a path that is correct but + * not yet on disk. That skip is scoped to a REAL package root: a stale + * group-less `packages/acp-agent/lib/bin.js` is still flagged (its root does not + * exist — exactly the moved-package drift this gate catches). + * + * Run: `tsx scripts/verify-package-paths.ts`. + */ + +import { existsSync, readdirSync, readFileSync, realpathSync } from 'node:fs' +import { relative, resolve } from 'node:path' +import { glob } from 'node:fs/promises' + +const root = resolve(import.meta.dirname, '..') + +/** Markdown + repo-authored TypeScript that may cite package paths. */ +const PATTERNS = [ + 'README.md', + 'docs/**/*.md', + 'packages/*/*.md', + 'packages/*/*/*.md', + 'AGENTS.md', + 'packages/AGENTS.md', + 'packages/**/*.ts', + 'examples/**/*.ts', +] + +/** Paths excluded from the scan: built output and vendored upstream source. */ +const isExcluded = (p: string): boolean => + p.includes('/lib/') || p.endsWith('.d.ts') || p.startsWith('vendor/') + +/** + * Directory names of every real package, `packages//`. A broken + * reference is only flagged when one of its segments is in this set — that is + * what scopes the gate to DRIFT (a moved real package) rather than typos or + * not-yet-existing packages named in a proposal. + */ +function realPackageNames(): Set { + const names = new Set() + const pkgRoot = resolve(root, 'packages') + for (const group of readdirSync(pkgRoot, { withFileTypes: true })) { + if (!group.isDirectory()) continue + for (const pkg of readdirSync(resolve(pkgRoot, group.name), { withFileTypes: true })) { + if (pkg.isDirectory()) names.add(pkg.name) + } + } + return names +} + +const packageNames = realPackageNames() + +/** + * Match a `packages/` reference token. The character class is plain path + * characters only, so a glob (`*`), placeholder (`<`, `>`), or brace expansion + * (`{`, `}`, `,`) terminates the match before those chars and is never probed — + * those are patterns, not real paths. A trailing `.`/`/` (e.g. a sentence-ending + * period) is trimmed before the existence check. + */ +const PKG_REF = /\bpackages\/[A-Za-z0-9._/-]+/g + +/** A broken package reference: a stale root-relative `packages/…` path. */ +interface Violation { + file: string + /** 1-based line where the reference appears. */ + line: number + ref: string +} + +/** + * Find every DRIFTED `packages/…` reference in one file: a token that does not + * resolve on disk AND names a real package in one of its segments (so it is a + * moved path, not a typo or a not-yet-existing package). The same real-package + * test also screens out a bare `packages` (no segment) and illustrative + * skeletons whose segment is not a package. + */ +function findViolations(absPath: string): Violation[] { + const file = relative(root, absPath) + const source = readFileSync(absPath, 'utf8') + const out: Violation[] = [] + const lines = source.split('\n') + for (let i = 0; i < lines.length; i++) { + const line = lines[i] + if (line === undefined) continue + for (const m of line.matchAll(PKG_REF)) { + // Trim a trailing path separator or sentence punctuation that the greedy + // class may have swallowed (`packages/core/tools.` / `…/tools/`). + const ref = m[0].replace(/[./]+$/, '') + if (existsSync(resolve(root, ref))) continue + // A reference INTO a package's built `lib/` is a build OUTPUT, not an + // authored-source location: it does not exist until `pnpm run build` emits + // it, and CI runs this gate BEFORE the build step. Skip it — but ONLY when + // the `packages//` ROOT it sits under is real and on disk, so + // `packages/ui/acp-agent/lib/bin.js` (correct, just not yet built) is + // exempt while a stale `packages/acp-agent/lib/bin.js` (group-less, the + // exact moved-package drift this gate exists to catch) still flags. A bare + // `lib` segment is not a blanket escape hatch. + const parts = ref.split('/') + const libAt = parts.indexOf('lib') + if (libAt === 3 && existsSync(resolve(root, parts.slice(0, 3).join('/')))) continue + // Only a stale path to a REAL (moved) package is a violation; a segment + // matching a live package name is the drift signal. + const segments = ref.split('/').slice(1) + if (segments.some(seg => packageNames.has(seg))) { + out.push({ file, line: i + 1, ref }) + } + } + } + return out +} + +const all: Violation[] = [] +let checked = 0 +const seen = new Set() +for (const pattern of PATTERNS) { + for await (const match of glob(pattern, { cwd: root })) { + if (isExcluded(match)) continue + // Dedup by real path: the root/packages CLAUDE.md are symlinks to AGENTS.md. + const real = realpathSync(resolve(root, match)) + if (seen.has(real)) continue + seen.add(real) + checked++ + all.push(...findViolations(real)) + } +} + +if (all.length === 0) { + console.log(`verify-package-paths: ${checked} file(s) checked, all packages/* references resolve.`) + process.exit(0) +} + +console.error('verify-package-paths: broken packages/* references found (target does not exist):') +for (const v of all) { + console.error(` ${v.file}:${v.line} ${v.ref}`) +} +process.exit(1) diff --git a/scripts/verify-rfc-classification.ts b/scripts/verify-rfc-classification.ts new file mode 100644 index 0000000000..011ce9591a --- /dev/null +++ b/scripts/verify-rfc-classification.ts @@ -0,0 +1,160 @@ +/** + * Doc-sync gate: enforce the RFC classification scheme + * ([the classification RFC](../docs/rfc/implemented/process/2026-06-20-rfc-classification.md)). + * Every RFC is filed at `docs/rfc/{lifecycle}/{class}/yyyy-mm-dd-topic.md`; the + * folder IS the label. This gate is the machine source of truth for the closed + * class set and keeps the README index honest. + * + * Two checks: + * + * 1. STRUCTURE — every `.md` under a lifecycle folder lives in a class folder + * from CLASSES, named `yyyy-mm-dd-*.md`. A loose `.md` directly under a + * lifecycle root (other than the README/AGENTS allowlist) fails; an unknown + * class folder fails; a stray file at an unexpected depth fails. This is what + * makes the set CLOSED: a new class folder can't appear without amending + * CLASSES here (and the README's Classification section, per the RFC). + * + * 2. COMPLETENESS — `docs/rfc/README.md` lists every RFC exactly once, under the + * `### {Class}` heading inside the `## {Lifecycle}` section that matches the + * file's path. A missing entry, a duplicate, or an entry under the wrong + * heading fails. This mirrors `verify-event-taxonomy`: a curated doc table + * checked against the on-disk source of truth, so the index can't drift. + * + * The class DESCRIPTIONS in the README prose are not checked (they are + * explanatory text); only the per-class index tables are. This is checker, not + * fixer: it reports and never rewrites. + * + * Run: `tsx scripts/verify-rfc-classification.ts`. + */ + +import { readFileSync } from 'node:fs' +import { relative, resolve } from 'node:path' +import { glob } from 'node:fs/promises' + +const root = resolve(import.meta.dirname, '..') +const rfcRoot = resolve(root, 'docs/rfc') + +/** The closed set of RFC lifecycles (top-level folders under docs/rfc/). */ +const LIFECYCLES = ['proposed', 'implemented', 'rejected'] as const + +/** + * The closed set of RFC classes (nested folder under each lifecycle). Adding a + * class is a deliberate act: extend this list AND the README's Classification + * section. The gate rejects any folder not listed here. + */ +const CLASSES = ['feature', 'bug-fix', 'simplification', 'architecture', 'process', 'testing'] as const + +/** Non-RFC Markdown allowed to sit directly at a lifecycle root. */ +const ROOT_ALLOWLIST = new Set(['AGENTS.md', 'CLAUDE.md']) + +/** Title-case a class/lifecycle folder name for README heading comparison. */ +const heading = (s: string): string => s.charAt(0).toUpperCase() + s.slice(1) + +const errors: string[] = [] + +// --- Check 1: structure ----------------------------------------------------- +// Every Markdown file anywhere under a lifecycle folder, at any depth. +interface Rfc { + lifecycle: string + cls: string + base: string + /** Path relative to docs/rfc, for the README link check. */ + rel: string +} +const rfcs: Rfc[] = [] + +for (const lifecycle of LIFECYCLES) { + for await (const match of glob(`${lifecycle}/**/*.md`, { cwd: rfcRoot })) { + const segs = match.split('/') + // Allowlisted file directly at the lifecycle root (e.g. implemented/AGENTS.md). + if (segs.length === 2 && ROOT_ALLOWLIST.has(segs[1] ?? '')) continue + const cls = segs[1] + const base = segs[2] + if (segs.length !== 3 || cls === undefined || base === undefined) { + errors.push(`structure: ${match} — expected {lifecycle}/{class}/file.md (got depth ${segs.length})`) + continue + } + if (!(CLASSES as readonly string[]).includes(cls)) { + errors.push(`structure: ${match} — unknown class folder "${cls}" (allowed: ${CLASSES.join(', ')})`) + continue + } + if (!/^\d{4}-\d{2}-\d{2}-.+\.md$/.test(base)) { + errors.push(`structure: ${match} — filename must be yyyy-mm-dd-topic.md`) + continue + } + rfcs.push({ lifecycle, cls, base, rel: match }) + } +} + +// --- Check 2: README completeness ------------------------------------------- +// Parse the index into (lifecycle, class) -> set of linked rel paths, by +// tracking the current `## {Lifecycle}` and `### {Class}` headings and reading +// every `](path)` link target underneath. A link target is normalized to its +// path relative to docs/rfc. +const readmePath = resolve(rfcRoot, 'README.md') +const readme = readFileSync(readmePath, 'utf8') +const lifecycleByHeading = new Map(LIFECYCLES.map((l): [string, string] => [heading(l), l])) +const classByHeading = new Map(CLASSES.map((c): [string, string] => [heading(c), c])) + +/** README-listed RFC link targets, keyed `lifecycle/class` -> set of rel paths. */ +const listed = new Map>() +let curLifecycle: string | null = null +let curClass: string | null = null + +for (const line of readme.split('\n')) { + const h2 = /^##\s+(.+?)\s*$/.exec(line) + if (h2?.[1] !== undefined) { + curLifecycle = lifecycleByHeading.get(h2[1].trim()) ?? null + curClass = null + continue + } + const h3 = /^###\s+(.+?)\s*$/.exec(line) + if (h3?.[1] !== undefined) { + curClass = classByHeading.get(h3[1].trim()) ?? null + continue + } + if (!curLifecycle || !curClass) continue + // Collect every relative .md link target on this line. + for (const m of line.matchAll(/\]\(([^)]+\.md)[^)]*\)/g)) { + const target = m[1] + if (target === undefined) continue + // README links are relative to docs/rfc; normalize and key by location. + const rel = relative(rfcRoot, resolve(rfcRoot, target)) + const key = `${curLifecycle}/${curClass}` + const set = listed.get(key) ?? new Set() + set.add(rel) + listed.set(key, set) + } +} + +// Every on-disk RFC must be listed under the heading matching its path. +const seenOnDisk = new Set() +for (const rfc of rfcs) { + seenOnDisk.add(rfc.rel) + const key = `${rfc.lifecycle}/${rfc.cls}` + if (!listed.get(key)?.has(rfc.rel)) { + errors.push( + `index: ${rfc.rel} is not listed in README under "## ${heading(rfc.lifecycle)}" → "### ${heading(rfc.cls)}"`, + ) + } +} + +// Every README entry must point at a real RFC under that same heading (catches a +// misfiled or stale row). +for (const [key, targets] of listed) { + for (const rel of targets) { + if (!seenOnDisk.has(rel)) { + errors.push(`index: README lists "${rel}" under "${key}", but no such RFC exists`) + } + } +} + +// --- Report ----------------------------------------------------------------- +if (errors.length === 0) { + console.log(`verify-rfc-classification: ${rfcs.length} RFC(s) checked, structure and index consistent.`) + process.exit(0) +} + +console.error('verify-rfc-classification: violations found:') +for (const e of errors) console.error(` ${e}`) +process.exit(1) diff --git a/scripts/verify-type-equiv.ts b/scripts/verify-type-equiv.ts new file mode 100644 index 0000000000..c93383e40f --- /dev/null +++ b/scripts/verify-type-equiv.ts @@ -0,0 +1,227 @@ +/** + * Doc-sync gate: verify every ` ```ts type-equiv ` block in the docs is a + * VERBATIM copy of the source type definition it documents. + * + * The core-data-structures docs paste real type definitions so a reader sees + * the exact shape. A paste drifts the moment source changes — this script is + * the drift guard. For each block it extracts the documented symbol's + * declaration from source via the TypeScript compiler API, whitespace- + * normalizes both the source text and the block, and asserts they are equal. + * + * Provenance lives in a central manifest (`scripts/type-equiv.manifest.json`), + * NOT in the doc prose: each entry names `{ doc, symbol, source }`. The script + * enforces a 1:1 correspondence — every type-equiv block in the docs has + * exactly one manifest entry (keyed by doc + declared symbol), and every + * manifest entry resolves to exactly one block. An orphan on either side fails, + * so a block can never be silently unchecked and an entry can never rot. + * + * doc-typecheck.ts recognizes the same ` ```ts type-equiv ` fence and skips it + * (it is not standalone-compilable and is not counted in the opt-out ratio); + * the two scripts share the fence, this one owns the verification. + * + * Run: `tsx scripts/verify-type-equiv.ts`. + */ + +import { readFileSync, existsSync } from 'node:fs' +import { resolve } from 'node:path' +import { glob } from 'node:fs/promises' +import ts from 'typescript' + +const root = resolve(import.meta.dirname, '..') + +/** + * Markdown globs scanned for ` ```ts type-equiv ` blocks — the SAME scope + * doc-typecheck uses. Scanning every doc (not only the docs the manifest names) + * is what makes the 1:1 guarantee real in both directions: a type-equiv block + * added to a doc with NO manifest entry is still discovered here and reported as + * an orphan, instead of being silently skipped. + */ +const MARKDOWN_GLOBS = ['README.md', 'docs/**/*.md', 'packages/*/*.md', 'packages/*/*/*.md'] + +/** One manifest entry: a documented type-equiv block and its source symbol. */ +interface ManifestEntry { + /** Doc file (repo-relative) containing the ` ```ts type-equiv ` block. */ + doc: string + /** The declared symbol the block must match (e.g. `SessionEvent`). */ + symbol: string + /** Source file (repo-relative) that exports the symbol. */ + source: string +} + +/** One extracted ` ```ts type-equiv ` block. */ +interface EquivBlock { + doc: string + /** 1-based line of the opening fence (for diagnostics). */ + line: number + /** Symbol name parsed from the block's declaration. */ + symbol: string + /** Block body (the pasted declaration). */ + code: string +} + +/** Collapse a declaration to its structural form for comparison: drop comments + * (block + line), then collapse all whitespace runs to single spaces. This lets + * a doc block show a CLEAN definition (without source's verbose inline JSDoc) + * while still guaranteeing the field shapes match — drift in a field name or + * type fails; a reworded inline comment does not. Adequate for our own type + * source (no string literal contains `//` or `/* *​/`); not a general tokenizer. */ +function normalize(code: string): string { + return code + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/(^|[^:])\/\/.*$/gm, '$1') + .replace(/\s+/g, ' ') + .trim() +} + +/** Strip a leading `export ` / `export default ` modifier — the doc block shows + * the bare declaration, the source carries the export modifier. */ +function stripExport(code: string): string { + return code.replace(/^export\s+(default\s+)?/, '') +} + +/** Parse the declared symbol name from a type-equiv block body. */ +function blockSymbol(code: string): string | null { + const m = /(?:export\s+(?:default\s+)?)?(?:abstract\s+)?(?:interface|type|class|enum)\s+([A-Za-z0-9_]+)/.exec(code) + return m?.[1] ?? null +} + +/** Extract every ` ```ts type-equiv ` block from one Markdown file. */ +function extractEquivBlocks(docRel: string): EquivBlock[] { + const text = readFileSync(resolve(root, docRel), 'utf8') + const lines = text.split('\n') + const blocks: EquivBlock[] = [] + let open: { line: number; body: string[] } | null = null + + for (let i = 0; i < lines.length; i++) { + const raw = lines[i] ?? '' + const fence = /^```(\s*)(\S.*)?$/.exec(raw) + if (!fence) { + if (open) open.body.push(raw) + continue + } + if (open) { + const code = open.body.join('\n') + const symbol = blockSymbol(code) + if (!symbol) { + throw new Error(`verify-type-equiv: ${docRel}:${open.line} — type-equiv block has no parseable interface/type/class declaration`) + } + blocks.push({ doc: docRel, line: open.line, symbol, code }) + open = null + continue + } + if ((fence[2] ?? '').trim() === 'ts type-equiv') open = { line: i + 1, body: [] } + } + if (open) throw new Error(`verify-type-equiv: ${docRel}:${open.line} — unterminated type-equiv block`) + return blocks +} + +/** The declaration text of `symbol` in `sourceRel`, with `export` stripped, or + * null when the symbol is not declared there. Uses the TS parser so it spans + * interfaces, type aliases (including mapped/generic ones), classes, and enums + * uniformly, and excludes the leading JSDoc (getStart skips leading trivia) + * while keeping inline member comments. */ +function sourceDeclaration(sourceRel: string, symbol: string): string | null { + const abs = resolve(root, sourceRel) + const text = readFileSync(abs, 'utf8') + const sf = ts.createSourceFile(abs, text, ts.ScriptTarget.Latest, /* setParentNodes */ true) + for (const stmt of sf.statements) { + const named = + ts.isInterfaceDeclaration(stmt) || ts.isTypeAliasDeclaration(stmt) + || ts.isClassDeclaration(stmt) || ts.isEnumDeclaration(stmt) + if (named && stmt.name?.text === symbol) { + return stripExport(stmt.getText(sf)) + } + } + return null +} + +const manifestRaw = readFileSync(resolve(root, 'scripts/type-equiv.manifest.json'), 'utf8') +const manifest = JSON.parse(manifestRaw) as { entries: ManifestEntry[] } +const entries = manifest.entries + +// Key a block/entry by doc + symbol (a symbol may be documented in more than one +// doc, but at most once per doc). +const keyOf = (x: { doc: string; symbol: string }): string => `${x.doc}::${x.symbol}` + +// Collect every type-equiv block across ALL docs in scope — not only the docs +// the manifest names — so a block in an unmanifested doc is found and reported +// as an orphan rather than silently skipped. +const docSet = new Set() +for (const pattern of MARKDOWN_GLOBS) { + for await (const match of glob(pattern, { cwd: root })) docSet.add(match) +} +const blocks: EquivBlock[] = [...docSet].sort().flatMap(extractEquivBlocks) + +const errors: string[] = [] +// A manifest entry naming a doc that does not exist (or is outside the scanned +// scope, so no block could ever match it) is an error in its own right. +for (const d of [...new Set(entries.map(e => e.doc))]) { + if (!existsSync(resolve(root, d))) errors.push(`manifest references ${d}, which does not exist`) + else if (!docSet.has(d)) errors.push(`manifest references ${d}, which is outside the scanned markdown scope (${MARKDOWN_GLOBS.join(', ')})`) +} + +// Duplicate-block guard: the same symbol twice in one doc is ambiguous. +const blockByKey = new Map() +for (const b of blocks) { + const k = keyOf(b) + const prior = blockByKey.get(k) + if (prior) { + errors.push(`duplicate type-equiv block for ${b.symbol} in ${b.doc} (lines ${prior.line} and ${b.line})`) + continue + } + blockByKey.set(k, b) +} + +// Duplicate-entry guard in the manifest. +const entryByKey = new Map() +for (const e of entries) { + const k = keyOf(e) + if (entryByKey.has(k)) { + errors.push(`duplicate manifest entry for ${e.symbol} in ${e.doc}`) + continue + } + entryByKey.set(k, e) +} + +// 1:1 correspondence: orphan blocks (no entry) and orphan entries (no block). +for (const b of blocks) { + if (!entryByKey.has(keyOf(b))) { + errors.push(`type-equiv block ${b.symbol} (${b.doc}:${b.line}) has no manifest entry — add one to scripts/type-equiv.manifest.json`) + } +} +for (const e of entries) { + if (!blockByKey.has(keyOf(e))) { + errors.push(`manifest entry ${e.symbol} (${e.doc}) has no matching type-equiv block — remove it or add the block`) + } +} + +// Verbatim check: each matched block must equal its source declaration. +let verified = 0 +for (const e of entries) { + const b = blockByKey.get(keyOf(e)) + if (!b) continue // already reported as an orphan entry + const decl = sourceDeclaration(e.source, e.symbol) + if (decl === null) { + errors.push(`symbol ${e.symbol} not found in ${e.source} (manifest entry for ${e.doc})`) + continue + } + if (normalize(decl) !== normalize(stripExport(b.code))) { + errors.push( + `DRIFT: ${e.doc}:${b.line} — type-equiv block for ${e.symbol} does not match ${e.source}.\n` + + ` source: ${normalize(decl)}\n` + + ` doc: ${normalize(stripExport(b.code))}`, + ) + continue + } + verified++ +} + +if (errors.length === 0) { + console.log(`verify-type-equiv: ${verified} type-equiv block(s) match source (1:1 with manifest).`) + process.exit(0) +} + +console.error('verify-type-equiv: type-equiv verification failed:') +for (const e of errors) console.error(` ${e}`) +console.error(`\n(checked ${blocks.length} block(s) across ${new Set(blocks.map(b => b.doc)).size} doc(s); manifest at scripts/type-equiv.manifest.json)`) +process.exit(1) diff --git a/tsconfig.base.json b/tsconfig.base.json index d8a2352188..3d6d1fc42a 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -4,12 +4,14 @@ "module": "esnext", "moduleResolution": "bundler", "declaration": true, - "emitDeclarationOnly": true, + "sourceMap": true, + "declarationMap": true, "composite": true, "incremental": true, "skipLibCheck": true, "esModuleInterop": true, "allowImportingTsExtensions": true, + "rewriteRelativeImportExtensions": true, "verbatimModuleSyntax": false, "strict": true, "noUncheckedIndexedAccess": true, @@ -19,11 +21,9 @@ "noUnusedLocals": true, "noUnusedParameters": true, "types": ["node"], - // Source-level resolution for the build graph: without this, a fresh - // checkout's first `tsc -b` resolves sibling vendor plugins through their - // package.json types (vendor/*/lib/*.d.ts) which don't exist yet — TS2307 - // until a second run. Derived configs that want lib resolution - // (tsconfig.typecheck.json) override this map wholesale. + // Source-level resolution for every repo-local graph. Project references, + // not declaration path aliases, keep each package/vendor source compiled + // under its own tsconfig boundary. "paths": { "cordis": ["./vendor/cordis/src"], "cosmokit": ["./vendor/cosmokit/src"], @@ -34,22 +34,20 @@ "@cordisjs/plugin-timer": ["./vendor/timer/src"], "@cordisjs/plugin-hmr": ["./vendor/hmr/src"], "@cordisjs/plugin-logger-console": ["./vendor/logger-console/src"], - "@deepseek-ai/dsh-llm": ["./packages/llm/src"], - "@deepseek-ai/dsh-session": ["./packages/session/src"], - "@deepseek-ai/dsh-session-persistence": ["./packages/session-persistence/src"], - "@deepseek-ai/dsh-session-persistence-jsonl": ["./packages/session-persistence-jsonl/src"], - "@deepseek-ai/dsh-session-persistence-sqlite": ["./packages/session-persistence-sqlite/src"], - "@deepseek-ai/dsh-system-prompt": ["./packages/system-prompt/src"], - "@deepseek-ai/dsh-tools": ["./packages/tools/src"], - "@deepseek-ai/dsh-agent": ["./packages/agent/src"], - "@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"], - "@deepseek-ai/dsh-bash": ["./packages/bash/src"], - "@deepseek-ai/dsh-llm-deepseek": ["./packages/llm-deepseek/src"], - "@deepseek-ai/dsh-llm-pi-ai": ["./packages/llm-pi-ai/src"], - "@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"], - "@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"], - "@deepseek-ai/dsh-invariants": ["./packages/invariants/src"], - "@deepseek-ai/dsh-acp": ["./packages/acp/src"] + // One wildcard maps every @deepseek-ai/dsh- to its source. Package + // dir names are unique across groups, so first-on-disk-wins resolution is + // unambiguous; adding a package under an existing group needs no edit + // here. The build graph's project references (tsconfig.build.json) stay + // explicit — TS project references have no wildcard form. + "@deepseek-ai/dsh-*": [ + "./packages/core/*/src", + "./packages/llm/*/src", + "./packages/bash/*/src", + "./packages/session-persistence/*/src", + "./packages/ui/*/src", + "./packages/util/*/src", + "./packages/support/*/src" + ] } } } diff --git a/tsconfig.build.json b/tsconfig.build.json index f2b5778abb..27a17a3f17 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -10,21 +10,27 @@ { "path": "./vendor/timer" }, { "path": "./vendor/hmr" }, { "path": "./vendor/logger-console" }, - { "path": "./packages/llm" }, - { "path": "./packages/session" }, - { "path": "./packages/session-persistence" }, - { "path": "./packages/session-persistence-jsonl" }, - { "path": "./packages/session-persistence-sqlite" }, - { "path": "./packages/system-prompt" }, - { "path": "./packages/agent" }, - { "path": "./packages/tools" }, - { "path": "./packages/agent-loop" }, - { "path": "./packages/bash" }, - { "path": "./packages/llm-deepseek" }, - { "path": "./packages/llm-pi-ai" }, - { "path": "./packages/bash-local" }, - { "path": "./packages/tool-bash" }, - { "path": "./packages/invariants" }, - { "path": "./packages/acp" } + { "path": "./packages/util/brand" }, + { "path": "./packages/llm/llm" }, + { "path": "./packages/core/session" }, + { "path": "./packages/session-persistence/session-persistence" }, + { "path": "./packages/session-persistence/session-persistence-jsonl" }, + { "path": "./packages/session-persistence/session-persistence-sqlite" }, + { "path": "./packages/core/system-prompt" }, + { "path": "./packages/core/agent" }, + { "path": "./packages/core/tools" }, + { "path": "./packages/core/agent-loop" }, + { "path": "./packages/core/agent-core" }, + { "path": "./packages/bash/bash" }, + { "path": "./packages/llm/llm-deepseek" }, + { "path": "./packages/llm/llm-pi-ai" }, + { "path": "./packages/bash/bash-local" }, + { "path": "./packages/bash/tool-bash" }, + { "path": "./packages/support/invariants" }, + { "path": "./packages/ui/acp" }, + { "path": "./packages/ui/acp-agent" }, + { "path": "./packages/ui/stdio-agent" }, + { "path": "./packages/support/ui-stdio" }, + { "path": "./packages/support/llm-replay" } ] } diff --git a/tsconfig.json b/tsconfig.json index 725f31659f..bc5aee0bb9 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -1,4 +1,47 @@ { "extends": "./tsconfig.base.json", - "files": [] + "compilerOptions": { + "noEmit": true, + "rewriteRelativeImportExtensions": false + }, + "include": [ + "examples/*/src/**/*.ts", + "examples/*/start.ts", + "examples/*/tests/**/*.ts", + "packages/*/*/tests/**/*.ts", + "scripts/**/*.ts" + ], + "references": [ + { "path": "./vendor/cosmokit" }, + { "path": "./vendor/schemastery" }, + { "path": "./vendor/cordis" }, + { "path": "./vendor/loader" }, + { "path": "./vendor/include" }, + { "path": "./vendor/group" }, + { "path": "./vendor/timer" }, + { "path": "./vendor/hmr" }, + { "path": "./vendor/logger-console" }, + { "path": "./packages/util/brand" }, + { "path": "./packages/llm/llm" }, + { "path": "./packages/core/session" }, + { "path": "./packages/session-persistence/session-persistence" }, + { "path": "./packages/session-persistence/session-persistence-jsonl" }, + { "path": "./packages/session-persistence/session-persistence-sqlite" }, + { "path": "./packages/core/system-prompt" }, + { "path": "./packages/core/agent" }, + { "path": "./packages/core/tools" }, + { "path": "./packages/core/agent-loop" }, + { "path": "./packages/core/agent-core" }, + { "path": "./packages/bash/bash" }, + { "path": "./packages/llm/llm-deepseek" }, + { "path": "./packages/llm/llm-pi-ai" }, + { "path": "./packages/bash/bash-local" }, + { "path": "./packages/bash/tool-bash" }, + { "path": "./packages/support/invariants" }, + { "path": "./packages/ui/acp" }, + { "path": "./packages/ui/acp-agent" }, + { "path": "./packages/ui/stdio-agent" }, + { "path": "./packages/support/ui-stdio" }, + { "path": "./packages/support/llm-replay" } + ] } diff --git a/tsconfig.test.json b/tsconfig.test.json deleted file mode 100644 index 5976d5b43c..0000000000 --- a/tsconfig.test.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "extends": "./tsconfig.json", - "compilerOptions": { - "noEmit": true, - "emitDeclarationOnly": false, - "composite": false, - "types": ["node"] - }, - "include": ["vendor/*/src", "packages/*/src", "packages/*/tests", "examples"] -} diff --git a/tsconfig.typecheck.json b/tsconfig.typecheck.json deleted file mode 100644 index e925a665aa..0000000000 --- a/tsconfig.typecheck.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "extends": "./tsconfig.base.json", - "compilerOptions": { - "noEmit": true, - "emitDeclarationOnly": false, - "composite": false, - "incremental": false, - "types": ["node"], - "paths": { - "cordis": ["./vendor/cordis/lib"], - "cosmokit": ["./vendor/cosmokit/lib"], - "schemastery": ["./vendor/schemastery/lib"], - "@cordisjs/plugin-loader": ["./vendor/loader/lib"], - "@cordisjs/plugin-include": ["./vendor/include/lib"], - "@cordisjs/plugin-group": ["./vendor/group/lib"], - "@cordisjs/plugin-timer": ["./vendor/timer/lib"], - "@cordisjs/plugin-hmr": ["./vendor/hmr/lib"], - "@cordisjs/plugin-logger-console": ["./vendor/logger-console/lib/shared"], - "@deepseek-ai/dsh-llm": ["./packages/llm/src"], - "@deepseek-ai/dsh-session": ["./packages/session/src"], - "@deepseek-ai/dsh-session-persistence": ["./packages/session-persistence/src"], - "@deepseek-ai/dsh-session-persistence-jsonl": ["./packages/session-persistence-jsonl/src"], - "@deepseek-ai/dsh-session-persistence-sqlite": ["./packages/session-persistence-sqlite/src"], - "@deepseek-ai/dsh-system-prompt": ["./packages/system-prompt/src"], - "@deepseek-ai/dsh-tools": ["./packages/tools/src"], - "@deepseek-ai/dsh-agent": ["./packages/agent/src"], - "@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"], - "@deepseek-ai/dsh-bash": ["./packages/bash/src"], - "@deepseek-ai/dsh-llm-deepseek": ["./packages/llm-deepseek/src"], - "@deepseek-ai/dsh-llm-pi-ai": ["./packages/llm-pi-ai/src"], - "@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"], - "@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"], - "@deepseek-ai/dsh-invariants": ["./packages/invariants/src"], - "@deepseek-ai/dsh-acp": ["./packages/acp/src"] - } - }, - "include": ["packages/*/src", "packages/*/tests", "examples", "scripts"] -} diff --git a/tsdown.config.ts b/tsdown.config.ts index 3b9039cc36..7b172180d1 100644 --- a/tsdown.config.ts +++ b/tsdown.config.ts @@ -1,19 +1,20 @@ import { defineConfig } from 'tsdown' /** - * JS bundling for all workspace packages (vendor/* + packages/*). - * Declarations are NOT produced here — `tsc -b tsconfig.build.json` owns - * .d.ts output (composite project references); hence `dts: false` and - * `clean: false` (lib/ already holds tsc's declarations). + * JS bundling for all workspace packages (vendor and the packages hierarchy). + * TypeScript source is compiled first by `tsc -b tsconfig.build.json`; tsdown + * reads only the emitted JS under lib/types and writes lib/index.* runtime + * bundles. Declarations are NOT produced here, hence `dts: false`. * * Per-package shape overrides live in `/tsdown.config.ts` * (schemastery: dual ESM+CJS; logger-console: extra browser entry). */ export default defineConfig({ - // Explicit globs: `workspace: true` would also discover examples/* (any - // package.json), but only vendor/* and packages/* are pnpm workspaces. - workspace: ['vendor/*', 'packages/*'], - entry: ['src/index.ts'], + // Explicit globs: `workspace: true` would also discover examples (any + // package.json), but only vendor and the packages hierarchy are pnpm + // workspaces. + workspace: ['vendor/*', 'packages/*/*'], + entry: ['lib/types/index.js'], outDir: 'lib', format: ['esm'], platform: 'node', diff --git a/vendor/README.md b/vendor/README.md index 04e63f26c1..bf0f0b5a8c 100644 --- a/vendor/README.md +++ b/vendor/README.md @@ -31,9 +31,10 @@ Intentionally **not** vendored (verified unused by this set): `reggol`, `@cordis Keep this log exhaustive — every divergence from upstream must be listed. 1. **`hmr/src/index.ts`**: removed the `./locales/en-US.yml` / `./locales/zh-CN.yml` imports, the `.i18n({...})` call on the `Config` schema, and the `src/locales/` directory. Rationale: those imports require a runtime YAML loader hook (`@cordisjs/unyaml`) that we do not vendor; the i18n texts only localize config descriptions. -2. **All `package.json` files**: regenerated — added `private: true`, added `src` to `files` and a `./src/*` export where missing, removed upstream `devDependencies`/`scripts`/`repository` fields. Dependency and peer-dependency ranges preserved, except `hmr` declares `esbuild` as a direct dev dependency because its source imports the `BuildFailure` type and pnpm's strict workspace resolution requires the owner package to name that dependency. -3. **All `tsconfig.json` files**: regenerated to extend the repo-root `tsconfig.base.json` and declare project references. -4. **`schemastery/tsdown.config.ts` and `logger-console/tsdown.config.ts`**: ours, not upstream files — per-package build-shape overrides (dual ESM+CJS output; separate node/browser entries) for the repo-root tsdown build. Like the regenerated tsconfigs, they are not part of the upstream sync surface. +2. **All `package.json` files**: regenerated — added `private: true`, added precise `files` entries for bundled runtime files and `lib/types/**/*.d.ts` / `.d.ts.map`, preserved `src` in `files` only for packages whose previous file list already shipped it, added a `./src/*` export where missing, pointed declaration metadata at `lib/types`, and removed upstream `devDependencies`/`scripts`/`repository` fields. Dependency and peer-dependency ranges preserved, except `hmr` declares `esbuild` as a direct dev dependency because its source imports the `BuildFailure` type and pnpm's strict workspace resolution requires the owner package to name that dependency. +3. **All `tsconfig.json` files**: regenerated to extend the repo-root `tsconfig.base.json`, emit TypeScript intermediates to `lib/types`, and declare project references. +4. **Vendored TypeScript source internal specifiers**: changed local relative imports/exports from upstream's specifier shape to explicit `.ts` specifiers so TypeScript rewrites emitted JS to `.js` while declarations keep explicit, NodeNext-safe `.ts` specifiers. This includes `loader/src/config/isolate.ts` using `declare module './entry.ts'`. +5. **`schemastery/tsdown.config.ts` and `logger-console/tsdown.config.ts`**: ours, not upstream files — per-package build-shape overrides (dual ESM+CJS output; separate node/browser entries) for the repo-root tsdown build. They read the JS emitted under `lib/types` and then write the publish runtime entries under `lib/`. Like the regenerated tsconfigs, they are not part of the upstream sync surface. ## Sync procedure diff --git a/vendor/cordis/package.json b/vendor/cordis/package.json index 6b9e59a00b..9d9ac07a34 100644 --- a/vendor/cordis/package.json +++ b/vendor/cordis/package.json @@ -6,18 +6,20 @@ "sideEffects": false, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "bin": "bin.js", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "bin.js" ], "author": "Shigma ", diff --git a/vendor/cordis/src/context.ts b/vendor/cordis/src/context.ts index 768ba52d6f..8b21c464b2 100644 --- a/vendor/cordis/src/context.ts +++ b/vendor/cordis/src/context.ts @@ -1,10 +1,10 @@ import { Dict } from 'cosmokit' -import { EventsService } from './events' -import { LoggerService } from './logger' -import { ReflectService } from './reflect' -import { InjectKey, RegistryService } from './registry' -import { getTraceable, symbols } from './utils' -import { Fiber } from './fiber' +import { EventsService } from './events.ts' +import { LoggerService } from './logger.ts' +import { ReflectService } from './reflect.ts' +import { InjectKey, RegistryService } from './registry.ts' +import { getTraceable, symbols } from './utils.ts' +import { Fiber } from './fiber.ts' /** * Public shape of a Cordis context. diff --git a/vendor/cordis/src/events.ts b/vendor/cordis/src/events.ts index f7dcf011f4..4461816537 100644 --- a/vendor/cordis/src/events.ts +++ b/vendor/cordis/src/events.ts @@ -1,7 +1,7 @@ import { defineProperty, Promisify } from 'cosmokit' -import { Context } from './context' -import { Fiber, FiberState } from './fiber' -import { DisposableList, symbols } from './utils' +import { Context } from './context.ts' +import { Fiber, FiberState } from './fiber.ts' +import { DisposableList, symbols } from './utils.ts' /** Return whether an event result should stop a bail-style dispatch. */ export function isBailed(value: any) { @@ -25,7 +25,7 @@ export type ThisType = F extends (this: infer T, ...args: any) => any ? T : n */ export type DispatchMode = 'emit' | 'parallel' | 'serial' | 'bail' | 'waterfall' -declare module './context' { +declare module './context.ts' { export interface Context { /* eslint-disable max-len */ parallel(name: K, ...args: Parameters): Promise diff --git a/vendor/cordis/src/fiber.ts b/vendor/cordis/src/fiber.ts index 840bc54352..fd472e7733 100644 --- a/vendor/cordis/src/fiber.ts +++ b/vendor/cordis/src/fiber.ts @@ -1,11 +1,11 @@ import { Awaitable, defineProperty, Dict, isNullable } from 'cosmokit' -import { Context } from './context' -import { Plugin } from './registry' -import { buildOuterStack, composeError, DisposableList, getTraceable, isConstructor, isObject, symbols } from './utils' -import { Impl } from './reflect' +import { Context } from './context.ts' +import { Plugin } from './registry.ts' +import { buildOuterStack, composeError, DisposableList, getTraceable, isConstructor, isObject, symbols } from './utils.ts' +import { Impl } from './reflect.ts' import { StandardSchemaV1 } from '@standard-schema/spec' -declare module './context' { +declare module './context.ts' { export interface Context extends Pick { fiber: Fiber } diff --git a/vendor/cordis/src/index.ts b/vendor/cordis/src/index.ts index 83d160395e..d0814213e0 100644 --- a/vendor/cordis/src/index.ts +++ b/vendor/cordis/src/index.ts @@ -1,14 +1,14 @@ /** Core context type and root context implementation. */ -export * from './context' +export * from './context.ts' /** Event bus, dispatch modes, and event augmentation types. */ -export * from './events' +export * from './events.ts' /** Plugin fiber lifecycle, effects, and config validation helpers. */ -export * from './fiber' +export * from './fiber.ts' /** Logger facade, logger service, message, exporter, and formatting types. */ -export * from './logger' +export * from './logger.ts' /** Plugin registry, dependency injection, and plugin entrypoint types. */ -export * from './registry' +export * from './registry.ts' /** Base service class and service lifecycle symbols. */ -export * from './service' +export * from './service.ts' /** Shared internal helpers used by context, services, and plugin fibers. */ -export * from './utils' +export * from './utils.ts' diff --git a/vendor/cordis/src/logger.ts b/vendor/cordis/src/logger.ts index f76ac2cdb7..a1e97c165a 100644 --- a/vendor/cordis/src/logger.ts +++ b/vendor/cordis/src/logger.ts @@ -1,9 +1,9 @@ import { defineProperty, hyphenate } from 'cosmokit' -import { Context } from './context' -import { Fiber } from './fiber' -import { createCallable, joinPrototype, symbols, Tracker } from './utils' +import { Context } from './context.ts' +import { Fiber } from './fiber.ts' +import { createCallable, joinPrototype, symbols, Tracker } from './utils.ts' -declare module './context' { +declare module './context.ts' { interface Intercept { logger: LoggerService.Intercept } diff --git a/vendor/cordis/src/reflect.ts b/vendor/cordis/src/reflect.ts index 4bc9fb44db..212ec4e779 100644 --- a/vendor/cordis/src/reflect.ts +++ b/vendor/cordis/src/reflect.ts @@ -1,9 +1,9 @@ import { defineProperty, Dict, isNullable } from 'cosmokit' -import { Context } from './context' -import { getTraceable, symbols, withProps } from './utils' -import { Fiber, FiberState } from './fiber' +import { Context } from './context.ts' +import { getTraceable, symbols, withProps } from './utils.ts' +import { Fiber, FiberState } from './fiber.ts' -declare module './context' { +declare module './context.ts' { interface Context { get(name: K, strict?: boolean): undefined | this[K] get(name: string, strict?: boolean): any diff --git a/vendor/cordis/src/registry.ts b/vendor/cordis/src/registry.ts index fae7712df9..9dfa10a06b 100644 --- a/vendor/cordis/src/registry.ts +++ b/vendor/cordis/src/registry.ts @@ -1,8 +1,8 @@ import { defineProperty, Dict } from 'cosmokit' import { StandardSchemaV1 } from '@standard-schema/spec' -import { Context } from './context' -import { Fiber } from './fiber' -import { buildOuterStack, DisposableList, symbols, withProps } from './utils' +import { Context } from './context.ts' +import { Fiber } from './fiber.ts' +import { buildOuterStack, DisposableList, symbols, withProps } from './utils.ts' function isApplicable(object: Plugin) { return object && typeof object === 'object' && typeof object.apply === 'function' @@ -140,7 +140,7 @@ type GetPluginConfig

= ? S : GetPluginParameters

[0] -declare module './context' { +declare module './context.ts' { export interface Context { inject(deps: Inject, callback: Plugin.Function): Fiber & PromiseLike plugin

(plugin: P, ...args: Spread>): Fiber & PromiseLike diff --git a/vendor/cordis/src/service.ts b/vendor/cordis/src/service.ts index 4cc9f307f2..30895247c1 100644 --- a/vendor/cordis/src/service.ts +++ b/vendor/cordis/src/service.ts @@ -1,6 +1,6 @@ import { defineProperty } from 'cosmokit' -import { Context } from './context' -import { createCallable, joinPrototype, symbols, Tracker } from './utils' +import { Context } from './context.ts' +import { createCallable, joinPrototype, symbols, Tracker } from './utils.ts' /** * Base class for services that expose a named API on `ctx`. diff --git a/vendor/cordis/src/utils.ts b/vendor/cordis/src/utils.ts index 46dd962c6f..2fd499bd0c 100644 --- a/vendor/cordis/src/utils.ts +++ b/vendor/cordis/src/utils.ts @@ -1,5 +1,5 @@ import { defineProperty } from 'cosmokit' -import type { Context, Service } from '.' +import type { Context, Service } from './index.ts' /** Ordered collection of disposable values with O(1) deletion by value. */ export class DisposableList { diff --git a/vendor/cordis/tsconfig.json b/vendor/cordis/tsconfig.json index b9829bf1df..c7357481fd 100644 --- a/vendor/cordis/tsconfig.json +++ b/vendor/cordis/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noImplicitAny": false, "noImplicitThis": false, "strictFunctionTypes": false, diff --git a/vendor/cosmokit/package.json b/vendor/cosmokit/package.json index 92fdf8e903..940fcdb539 100644 --- a/vendor/cosmokit/package.json +++ b/vendor/cosmokit/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/cosmokit/src/array.ts b/vendor/cosmokit/src/array.ts index ccbc4b2752..18ed5e407f 100644 --- a/vendor/cosmokit/src/array.ts +++ b/vendor/cosmokit/src/array.ts @@ -1,4 +1,4 @@ -import { isNullable } from './misc' +import { isNullable } from './misc.ts' /** Return true when every item in `array2` is present in `array1`. */ export function contain(array1: readonly any[], array2: readonly any[]) { diff --git a/vendor/cosmokit/src/index.ts b/vendor/cosmokit/src/index.ts index 088e81c54f..9fe48de069 100644 --- a/vendor/cosmokit/src/index.ts +++ b/vendor/cosmokit/src/index.ts @@ -1,10 +1,10 @@ /** Array set and normalization helpers. */ -export * from './array' +export * from './array.ts' /** Runtime type, binary, clone, and equality helpers. */ -export * from './types' +export * from './types.ts' /** Shared utility types and object helpers. */ -export * from './misc' +export * from './misc.ts' /** String case, path, and property formatting helpers. */ -export * from './string' +export * from './string.ts' /** Time constants, parsing, and formatting helpers. */ -export * from './time' +export * from './time.ts' diff --git a/vendor/cosmokit/src/types.ts b/vendor/cosmokit/src/types.ts index b4d1e5bed8..499a46273a 100644 --- a/vendor/cosmokit/src/types.ts +++ b/vendor/cosmokit/src/types.ts @@ -1,4 +1,4 @@ -import { isNullable } from './misc' +import { isNullable } from './misc.ts' type GlobalConstructorNames = keyof { [K in keyof typeof globalThis as typeof globalThis[K] extends abstract new (...args: any) => any ? K : never]: K diff --git a/vendor/cosmokit/tsconfig.json b/vendor/cosmokit/tsconfig.json index 0db18e0f14..b7411f94f2 100644 --- a/vendor/cosmokit/tsconfig.json +++ b/vendor/cosmokit/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, "noImplicitOverride": false, diff --git a/vendor/group/package.json b/vendor/group/package.json index 86e5043a10..34a8f59ae2 100644 --- a/vendor/group/package.json +++ b/vendor/group/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/group/tsconfig.json b/vendor/group/tsconfig.json index 137b02f7ac..e512d1d84c 100644 --- a/vendor/group/tsconfig.json +++ b/vendor/group/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, "noImplicitOverride": false, diff --git a/vendor/hmr/package.json b/vendor/hmr/package.json index 075968a3ba..28087d5fa8 100644 --- a/vendor/hmr/package.json +++ b/vendor/hmr/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/hmr/tsconfig.json b/vendor/hmr/tsconfig.json index 033f83429f..8464912787 100644 --- a/vendor/hmr/tsconfig.json +++ b/vendor/hmr/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, "noImplicitOverride": false, diff --git a/vendor/include/package.json b/vendor/include/package.json index d42a1c0739..f9314d0c5e 100644 --- a/vendor/include/package.json +++ b/vendor/include/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/include/tsconfig.json b/vendor/include/tsconfig.json index ae2c70f4bc..6fe5099b43 100644 --- a/vendor/include/tsconfig.json +++ b/vendor/include/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noImplicitAny": false, "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, diff --git a/vendor/loader/package.json b/vendor/loader/package.json index ee5dd088ff..fde6d01d27 100644 --- a/vendor/loader/package.json +++ b/vendor/loader/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/loader/tsconfig.json b/vendor/loader/tsconfig.json index 84799662e0..2db62c7b63 100644 --- a/vendor/loader/tsconfig.json +++ b/vendor/loader/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noImplicitAny": false, "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, diff --git a/vendor/logger-console/package.json b/vendor/logger-console/package.json index b4a4c9634e..8c0d8a0bda 100644 --- a/vendor/logger-console/package.json +++ b/vendor/logger-console/package.json @@ -5,10 +5,10 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/shared.d.ts", + "types": "lib/types/shared.d.ts", "exports": { ".": { - "types": "./lib/shared.d.ts", + "types": "./lib/types/shared.d.ts", "node": "./lib/index.js", "default": "./lib/browser.js" }, @@ -16,7 +16,10 @@ "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/browser.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/logger-console/src/browser.ts b/vendor/logger-console/src/browser.ts index bdbeaaf226..b45a15e228 100644 --- a/vendor/logger-console/src/browser.ts +++ b/vendor/logger-console/src/browser.ts @@ -1,8 +1,8 @@ import { Message } from 'cordis' -import { ConsoleExporter as Base } from './shared.js' +import { ConsoleExporter as Base } from './shared.ts' /** Re-export shared console exporter config and base implementation. */ -export * from './shared.js' +export * from './shared.ts' /** Browser console exporter that dispatches to native console methods. */ export class ConsoleExporter extends Base { diff --git a/vendor/logger-console/src/index.ts b/vendor/logger-console/src/index.ts index 87ab53d6dc..d46ac6413f 100644 --- a/vendor/logger-console/src/index.ts +++ b/vendor/logger-console/src/index.ts @@ -1,10 +1,10 @@ import { Formatter } from 'cordis' import { inspect } from 'node:util' import supportsColor from 'supports-color' -import { ConsoleExporter as Base } from './shared.js' +import { ConsoleExporter as Base } from './shared.ts' /** Re-export shared console exporter config and base implementation. */ -export * from './shared.js' +export * from './shared.ts' const inspectFormatter: Formatter = (value, target) => { return inspect(value, { colors: !!target.colors, depth: Infinity, compact: true, breakLength: Infinity }) diff --git a/vendor/logger-console/tsconfig.json b/vendor/logger-console/tsconfig.json index c632badb1b..cba4d151c7 100644 --- a/vendor/logger-console/tsconfig.json +++ b/vendor/logger-console/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, "noImplicitOverride": false, diff --git a/vendor/logger-console/tsdown.config.ts b/vendor/logger-console/tsdown.config.ts index 3d9b213b6b..0df6d4bd0b 100644 --- a/vendor/logger-console/tsdown.config.ts +++ b/vendor/logger-console/tsdown.config.ts @@ -3,9 +3,10 @@ import { defineConfig } from 'tsdown' /** * logger-console ships two entries: the node exporter (index) and the * browser exporter (browser), selected via package.json `exports` - * conditions. They are built as two single-entry passes so the shared - * base class is inlined into each (matching upstream's published shape) - * instead of split into a hash-named chunk. + * conditions. The entries are JS emitted by tsc under lib/types and are + * bundled as two single-entry passes so the shared base class is inlined into + * each (matching upstream's published shape) instead of split into a hash-named + * chunk. */ const shared = { outDir: 'lib', @@ -18,6 +19,6 @@ const shared = { } as const export default defineConfig([ - { ...shared, entry: ['src/index.ts'] }, - { ...shared, entry: ['src/browser.ts'] }, + { ...shared, entry: ['lib/types/index.js'] }, + { ...shared, entry: ['lib/types/browser.js'] }, ]) diff --git a/vendor/schemastery/package.json b/vendor/schemastery/package.json index 71f7744e5e..ec5791f3af 100644 --- a/vendor/schemastery/package.json +++ b/vendor/schemastery/package.json @@ -5,9 +5,12 @@ "private": true, "main": "lib/index.cjs", "module": "lib/index.mjs", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "files": [ - "lib", + "lib/index.mjs", + "lib/index.cjs", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/schemastery/tsconfig.json b/vendor/schemastery/tsconfig.json index 5797e8902b..b25fa05af7 100644 --- a/vendor/schemastery/tsconfig.json +++ b/vendor/schemastery/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "module": "preserve", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, diff --git a/vendor/schemastery/tsdown.config.ts b/vendor/schemastery/tsdown.config.ts index 43b4384a06..57f2f5f6c4 100644 --- a/vendor/schemastery/tsdown.config.ts +++ b/vendor/schemastery/tsdown.config.ts @@ -2,12 +2,12 @@ import { defineConfig } from 'tsdown' /** * schemastery has no `"type": "module"` and publishes dual-format output - * (package.json: main → lib/index.cjs, module → lib/index.mjs). Pin the - * extensions explicitly — the defaults for a CommonJS package would emit - * .mjs/.js instead. + * (package.json: main → lib/index.cjs, module → lib/index.mjs). The entry is + * the JS emitted by tsc under lib/types; pin the bundled extensions + * explicitly because the defaults for a CommonJS package would emit .mjs/.js. */ export default defineConfig({ - entry: ['src/index.ts'], + entry: ['lib/types/index.js'], outDir: 'lib', format: ['esm', 'cjs'], platform: 'node', diff --git a/vendor/timer/package.json b/vendor/timer/package.json index 30bfe58280..07c41150e8 100644 --- a/vendor/timer/package.json +++ b/vendor/timer/package.json @@ -5,17 +5,19 @@ "private": true, "type": "module", "main": "lib/index.js", - "types": "lib/index.d.ts", + "types": "lib/types/index.d.ts", "exports": { ".": { - "types": "./lib/index.d.ts", + "types": "./lib/types/index.d.ts", "default": "./lib/index.js" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ - "lib", + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", "src" ], "author": "Shigma ", diff --git a/vendor/timer/tsconfig.json b/vendor/timer/tsconfig.json index 99c40177cd..843303e870 100644 --- a/vendor/timer/tsconfig.json +++ b/vendor/timer/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.base.json", "compilerOptions": { "rootDir": "src", - "outDir": "lib", + "outDir": "lib/types", "noUncheckedIndexedAccess": false, "exactOptionalPropertyTypes": false, "noImplicitOverride": false, diff --git a/vitest.config.ts b/vitest.config.ts index 0878477fb4..fd37a67278 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -5,9 +5,9 @@ export default defineConfig({ // Vite ≥8 warns that this plugin can be replaced by the native (experimental) // `resolve.tsconfigPaths: true`. It cannot — keep the plugin. Tests run // unbuilt (see AGENTS.md): bare workspace names like `cordis` or - // `@deepseek-ai/dsh-llm` must resolve to src/, and the only place that - // mapping exists is the root tsconfig.json `paths` map inherited by - // tsconfig.test.json. The native option is a bare boolean: for each + // `@deepseek-ai/dsh-llm` must resolve to src/, and that mapping comes from + // the root tsconfig.json paths map. The native option is a bare boolean: + // for each // importing file it discovers the NEAREST tsconfig.json and applies that // file's own `paths`. Every workspace under packages/* and vendor/* has its // own tsconfig.json without `paths`, so native resolution maps nothing, @@ -17,16 +17,21 @@ export default defineConfig({ // 15 workspace tsconfigs — including vendor/* ones, which are pinned // upstream copies (vendor/README.md). The plugin's `projects` option // instead applies the one root map to every importer. - plugins: [tsconfigPaths({ projects: ['./tsconfig.test.json'] })], + plugins: [tsconfigPaths({ projects: ['./tsconfig.json'] })], test: { - include: ['packages/*/tests/**/*.spec.ts', 'examples/*/tests/**/*.spec.ts'], + include: ['packages/*/*/tests/**/*.spec.ts', 'examples/*/tests/**/*.spec.ts'], coverage: { provider: 'v8', // Coverage measures OUR runtime source. Types-only files carry no // executable code; vendor/ and examples/ are out of scope (examples are // exercised by the demo smoke test instead). - include: ['packages/*/src/**/*.ts'], - exclude: ['packages/*/src/types.ts'], + include: ['packages/*/*/src/**/*.ts'], + // Types-only files carry no executable code. `bin.ts` files are + // self-executing CLI entrypoints (a top-level `await main()`): a spec + // can't import one without booting it, so they are driven by the keyless + // Loader-path smoke (a real subprocess) instead of the in-process unit + // suite — the same reason `examples/start.ts` sat out of coverage scope. + exclude: ['packages/*/*/src/types.ts', 'packages/*/*/src/bin.ts'], // 100% or it doesn't merge (AGENTS.md: excessive tests are welcome). // Per-file so a well-covered big file can't subsidize a bare one. // Every v8 ignore comment must carry a reason — see AGENTS.md. diff --git a/vitest.e2e.config.ts b/vitest.e2e.config.ts index 9316d660da..63ccbb8a27 100644 --- a/vitest.e2e.config.ts +++ b/vitest.e2e.config.ts @@ -22,9 +22,9 @@ try { export default defineConfig({ // Same resolution note as vitest.config.ts: bare workspace names resolve // through the root tsconfig paths map; the native option cannot do this. - plugins: [tsconfigPaths({ projects: ['./tsconfig.test.json'] })], + plugins: [tsconfigPaths({ projects: ['./tsconfig.json'] })], test: { - include: ['packages/*/tests/**/*.e2e.ts', 'examples/*/tests/**/*.e2e.ts'], + include: ['packages/*/*/tests/**/*.e2e.ts', 'examples/*/tests/**/*.e2e.ts'], // Real model calls: generous timeouts, and retries for transient flakes // (the shared internal key hits concurrency quotas). No coverage — the // unit suites own the coverage gate. diff --git a/vitest.snapshot.config.ts b/vitest.snapshot.config.ts new file mode 100644 index 0000000000..ecc8d911aa --- /dev/null +++ b/vitest.snapshot.config.ts @@ -0,0 +1,36 @@ +import tsconfigPaths from 'vite-tsconfig-paths' +import { defineConfig } from 'vitest/config' + +// Snapshot tests: `pnpm run test:snapshot`, file pattern *.snapshot.ts. +// REPLAY by default — they boot the real acp-agent subprocess against a +// recorded session JSONL fixture (no API key, no network) and diff the +// normalized stdout transcript + re-persisted log against committed goldens. +// `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the +// fixtures against the real API and refreshes the goldens. +// +// Replay loads no .env (it must never reach the network — a recorded fixture +// drives the model). Record reads DEEPSEEK_API_KEY from the env or a gitignored +// repo-root .env, so a contributor with a key only in .env can still record. +if (process.env.DSH_SNAPSHOT === 'record') { + try { + process.loadEnvFile(new URL('.env', import.meta.url).pathname) + } catch (error) { + // ENOENT (no .env) is fine — the key may already be in the environment. + // Surface any other failure rather than silently recording with wrong env. + if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') throw error + } +} + +export default defineConfig({ + // Same resolution note as vitest.config.ts: bare workspace names resolve + // through the root tsconfig paths map; the native option cannot do this. + plugins: [tsconfigPaths({ projects: ['./tsconfig.json'] })], + test: { + include: ['examples/*/tests/**/*.snapshot.ts'], + // Each test boots a subprocess; give it room, and run files one at a time + // (a record run hits the live API, and replay subprocess boot is heavy). + testTimeout: 120_000, + hookTimeout: 30_000, + fileParallelism: false, + }, +})