Adds docs/rfc/implemented/2026-06-19-real-api-e2e-ci.md covering the rationale
for running the real-API e2e suite in a separate secret-consuming workflow, the
fork/Dependabot/secret threat model, the residual exposure of the pull_request
trigger, and what changes when the repo goes public. Indexes it in the RFC
README.
Also adds a SECURITY comment on the pull_request trigger forbidding a switch to
pull_request_target (an untrusted-code-with-secrets leak vector, especially once
public), pointing at the RFC.
- Rewrite the snapshot-test RFC's replay-plugin section to state current
reality directly (the plugin is the @deepseek-ai/dsh-llm-replay package,
under the coverage gate) instead of keeping the old example-local text with a
"superseded" note bolted on.
- Add docs/rfc/implemented/AGENTS.md (+ CLAUDE.md symlink): an implemented RFC
must be kept current with what actually shipped — update paths/names/structure
in the same change that moves the code, in place, not as an append-only
changelog of its own drift. A reversal of the DECISION is still a new RFC.
- Reconcile docs/rfc/README.md: the "never edited into a different decision"
rule now distinguishes tracking where a decision lives (required) from
flipping the decision (forbidden), and the implemented/ bullet points at the
new convention.
Round 1 of Codex review on the extraction PR.
- (B) EOF-exit race: the 200ms flush-then-exit setTimeout was untracked, so a
fiber/HMR dispose within that window could not cancel it and the process
would still exit. Track the handle and clear it in the disposer; coalesce
re-entrant maybeExit() calls onto the one pending timer. Regression tests for
both (dispose-within-window cancels; repeated idle schedules once).
- (B/doc) ui-stdio rendering is global, not scoped by config.agent (faithful to
the original copies — agent scopes only input + the EOF-exit gate). Corrected
the README + Config JSDoc, which overclaimed "drive and render".
- (C) createStdioChat is exported and driven directly by tests/programmatic
callers that bypass schemastery validation, so default welcome/agent in the
helper (?? 'ready.'/'main') instead of trusting the cast. Test for empty config.
- (A/doc) docs/rfc/.../acp-snapshot-tests.md asserted the replay plugin
deliberately stays in examples/ ("don't split preemptively") — now false since
this PR packages it. Added a superseding note with the why (coverage gate).
All gates green: typecheck, lint, test:coverage (891, 100%), doc-sync,
test:e2e (6 keyless pass), test:snapshot unaffected.
Logic that lived under examples/ was outside the per-file 100% coverage
gate (examples/ are not workspaces) and, in the stdio-UI case, duplicated
across two examples. Move it into packages/ so it is gated and de-duped.
- packages/ui-stdio (new): unify the two diverged stdio-chat.ts copies into
one @deepseek-ai/dsh-ui-stdio plugin (welcome/agent Config). A test-only
I/O seam (createStdioChat(ctx, config, runtime)) keeps process streams out
of the serializable config and makes every render/EOF/disposal branch
unit-testable. Per-file 100%. echo/coding cordis.yml now load the package;
both src/stdio-chat.ts deleted.
- packages/llm-replay (new): move examples/acp-agent/src/llm-replay.ts (+ its
spec) here so its derive/parse/replay branches fall under the coverage gate.
cordis.snapshot.yml + README rewired to the package name; added apply/env
/assertNever/abort tests to reach per-file 100%.
- examples/{echo,coding}-agent: keyless Loader-path e2e smokes that boot the
real cordis.yml (no key) — the guard a hand-mounted unit test cannot be for
the unwrapExports/export-shape class (postmortem 0001). examples/AGENTS.md
codifies the keyless+with-key smoke convention (keyless-by-nature exception
for echo-agent).
- AGENTS.md: a scoped, removal-triggered pre-release stance (foundation over
blast radius). packages/README.md: new rows + a FIXME to later regroup ALL
packages into a hierarchy. Wiring: tsconfig paths/refs, publint, knip,
module-graph.
Verified: typecheck, lint, test:coverage (887 tests, 100%), build, hygiene,
doc-sync, test:snapshot (10), test:e2e (6 keyless pass, with-key self-skip).
Rename the concrete Agent class to make its ReAct-style reasoning loop
explicit in the name. Package name, default-export plugin (`AgentLoop`),
and the `ctx.agentLoop` service key are unchanged.
Establishes the standard way to give a snapshot scenario a non-empty starting
workspace: an optional `<scenario>/workspace/` directory whose contents the
harness copies into the temp cwd before the run (for both record and replay),
so the agent's bash tools see the seeded files. The cwd is normalized in the
goldens, so seeded paths stay stable.
The new `workspace-edit` scenario demonstrates the full read→write→verify cycle
on a seeded file: it ships `workspace/greeting.txt` ("hello"), prompts the agent
to append a WORLD line and cat it back. The recorded log captures the real bash
edits (`echo WORLD >> greeting.txt`, then `cat` showing `hello\nWORLD`), and it
replays deterministically with no key.
Also hardens runScenario teardown (Codex review): workspace seeding and spawn
now run inside the try whose finally removes both temp dirs, so a seeding/spawn
failure can't leak them. Documents the convention in the RFC + example README.
The snapshot replay config duplicated most of base.yml + the acp tail just to
swap llm-deepseek → llm-replay. Factor the shared pieces:
- examples/base-core.yml: the providerless provider/tool core (llm, sessions,
system-prompt, tools, agents, invariants, bash-local, tool-bash). base.yml is
now base-core + the llm-deepseek adapter; the snapshot replay config is
base-core + llm-replay. The replay config no longer hand-copies the core.
- examples/acp-agent/acp-tail.yml: agent-loop (no pre-created agents) +
persistence + the ACP bridge/system-prompt, shared by cordis.yml and the
replay config so the three acp-agent configs can't drift. Its persistence root
is `$DSH_SNAPSHOT_SESSIONS_ROOT ?? ./.sessions`.
- Deleted cordis.snapshot-record.yml: recording now reuses the normal cordis.yml
(real adapter), with the harness redirecting the persistence root via env.
start.ts maps DSH_SNAPSHOT=record → cordis.yml.
Verified: snapshot replay 8/8 keyless; record path works through cordis.yml;
ACP e2e no-key boot green through the doubly-nested include (cordis.yml →
base.yml → base-core.yml); coding-agent boots clean; all gates pass.
The goldens now mirror the shape of the surfaces they capture — one compact
JSON record per line — matching the wire (NDJSON stdout) and disk (JSONL
session log) formats, renamed *.golden.jsonl. They stay grep/jq-able and
faithful to what the agent emits, where the prior pretty-printed .txt was a
reformatted representation. Both normalizers drop the 2-space indent; the
normalizer spec asserts the compact form. All 11 goldens regenerated; replay
remains deterministic (8/8 across runs).
Holistic-review fixes for integration gaps the per-commit reviews missed:
- CI now runs `pnpm run test:snapshot` (a step after the coverage gate). It was
wired into pre-push but not .github/workflows/ci.yml, so the RFC/AGENTS claim
that snapshot replay runs in the default PR gate was only half-true — CI is
the real gate.
- vitest.snapshot.config.ts loads the repo .env ONLY when DSH_SNAPSHOT=record.
Loading it unconditionally contradicted the replay safety story (replay must
never reach the network), and runScenario forwards process.env to the child.
Non-ENOENT load errors now surface instead of being swallowed.
- start.ts: the graceful-shutdown comment said "RECORD runs" but the path
applies to both snapshot modes (replay also closes stdin → dispose → exit).
- docs/development.md: list the new pre-push snapshot job and the CI snapshot
gate.
Per a design revision, the per-scenario snapshot fixture becomes EXACTLY the
persisted session JSONL (<scenario>/session.jsonl) rather than a hand-authored
llm.json. The log already holds all LLM behavior (assistant/chunk carries every
StreamChunk) AND all harness behavior (tool/call, tool/result, turn/*, usage),
so one artifact drives replay and doubles as a behavioral golden.
llm-replay becomes replay-only (the record-tee is removed; recording is now
"run the real agent once and harvest the .jsonl", done by the harness in a
later commit). deriveReplayScript(events) groups assistant/chunk by (turn,step)
in log order — exact because the loop makes one ctx.llm.stream() call per step
and tags each chunk with the current (turn,step). The two failure modes the log
can't express (a thrown stream — no terminal finish; cancel/hang — timing) use
an optional replay.override.json sidecar.
Hardens against a Codex review finding: a derived group is only valid if it
ends in a `finish` chunk. A group without one is the fingerprint of a thrown
stream() and is NOT silently replayed as a clean stop — deriveReplayScript
throws, naming the (turn,step), so a missing sidecar override fails loud.
Updates the unit tests (parse/derive/load helpers, sidecar override, finish-
terminated grouping, HMR), the example README, and the RFC prose to the JSONL
format. Two goldens (stdout transcript + re-persisted JSONL) and the harness
wiring land in the next commit.
Records the decision to add a third test tier: snapshot tests that boot the
real acp-agent subprocess over ACP stdio, record the LLM's streamed responses
once against the real API, then replay them deterministically so the full
stdout transcript can be diffed against a committed golden — keyless in CI.
Captures the design choices hardened in a Codex (xhigh) review: record at the
provider-neutral llm/stream waterfall; a discriminated fixture entry schema
(chunks/throw/hang) that honors both LLM failure branches; positional replay
with a one-in-flight-stream constraint; per-stream atomic fixture flush (the
subprocess is SIGKILLed, so dispose-time flush would never run); a providerless
replay config; normalize-then-snapshot parsed frames; normalization over an OS
sandbox now with the rootless bwrap/sandbox-exec tier reserved via the
BashExecutor capability seam. Cross-links the proposed determinism RFC
(complementary: internal history invariant vs external protocol contract).
Codex + an independent review pass found three real defects in the prior commit:
1. parseExitStatus could misreport a SUCCESSFUL command as a failure: a clean
exit 0 appends no marker, so output ending in "[exit code: 5]" (no trailing
newline) was read as the marker. Anchor the parse to a LEADING newline —
renderResult always inserts one before a real marker, so a body that merely
ends in marker-like text no longer matches. A narrow residual (a clean exit 0
whose final line is exactly the marker) is inherent to the replay-only-sees-
text design and documented; the complete fix (a structured exit on the event)
is the RFC's named escape hatch.
2. A run_in_background start and an isError result were rendered as exited
terminal cards with a false exit-0 pill. A background start returns a task-id
ack (not a streamed terminal) and is no longer marked terminal; an isError
result (spawn failure / abort) carries no exit pill.
3. The terminal capability was re-read live on the result path, so a second
initialize between a call and its result could desync them (orphan
terminal_output or clobbered card). Snapshot the capability per session at
creation (SessionRecord.terminalEnabled) so call and result always agree.
Also reword the reference-parity claim: keeping the description as a content
block in terminal mode is a DELIBERATE divergence (claude-agent-acp drops it).
Tests added for each; with-key e2e still green.
Match claude-agent-acp / codex-acp: the bash tool_call title IS the command
(an execute card hides rawInput), the model description rides as a content
text block above the card, and the completed card carries an exit-status pill
via _meta.terminal_exit.
Bridge fixes found in review of the prior terminal-card commit:
- tool_call_update.content is OMITTED in terminal mode (an ACP update.content
REPLACES the call's content collection in Zed, so the fenced ```console block
would clobber the terminal content block).
- terminal.output preserves RAW newlines (terminal renderers rely on exact
bytes); only the fenced fallback trims trailing blank lines.
- a relative workdir is resolved against the session cwd for the card header,
matching where the command actually ran.
- result-side terminal output is gated on the pending call having registered a
terminal (no orphan _meta.terminal_output for a terminal Zed never made).
The exit pill is recovered by parsing renderResult's status markers (the pure
presentResult seam sees only content blocks); a round-trip test pins the parse
to the marker emission. Neutral ToolTerminal gains exitCode/signal; widened
ToolCallPresentation with a content block. Docs (RFC + 3 READMEs) updated;
with-key e2e verifies the card + exit pill against the real model.
When the client advertises clientCapabilities._meta.terminal_output (Zed), a
bash tool call now renders as a real TERMINAL card — a cwd header + the command
+ its output — instead of the plain ```console text block. Keeps agent-side
dsh-bash execution; rejects the spec's client-side terminal/create (which would
bypass sandbox/env-scrub/ownership/cwd). Matches what claude-agent-acp and
codex-acp do; wire contract verified against Zed's source.
- dsh-tools: a provider-neutral ToolTerminal shape ({ cwd?, output? }) on
ToolCallPresentation/ToolResultPresentation — a tool asks "render me as a
terminal"; no ACP types leak in.
- dsh-tool-bash: bash presentCall marks terminal (cwd from an explicit absolute
workdir, else left for the bridge to fill from the session cwd); presentResult
carries the output alongside the ```console fallback.
- dsh-acp: initialize reads/remembers the _meta.terminal_output capability;
streamSessionEventUpdate maps a terminal presentation to
content:[{type:'terminal',terminalId}] + _meta.terminal_info on the call and
_meta.terminal_output on the update WHEN capable — else the unchanged text
path. terminalId is the callId; cwd defaults to the session header. The pure
translator gained a TerminalRendering {enabled,cwd} param (off by default).
Tests via the REAL tool-bash + bash-local: capability ON -> terminal content +
_meta; OFF -> no _meta (text path). The with-key e2e adds a real-model terminal
card case (echo over ACP with the capability on). 773 tests, 100% coverage.
The exit-status pill (_meta.terminal_exit), live streaming
(_meta.terminal_output_delta), and command classification are RFC follow-ups.
Records the verified design before implementing: keep dsh-bash agent-side
execution and render Zed's terminal tool-call card via the `_meta` convention
(terminal_info/terminal_output/terminal_exit), capability-gated on
clientCapabilities._meta.terminal_output, with the ```console text block as the
no-capability fallback. Rejects the spec's client-side terminal/create path (it
would bypass dsh-bash's sandbox/env-scrub/ownership/cwd). Studied
claude-agent-acp, codex-acp, and Zed's renderer to ground the wire contract.
Live streaming and command classification are noted as separate follow-ups.
- bash presentCall title is now "description — command" (e.g. "List files in
src — ls -la src"). An execute-kind ACP card HIDES rawInput (Zed renders it
only for non-terminal tools), so the command must ride in the always-visible
title to be seen — matching how claude-agent-acp/codex-acp title execute
tools. The command stays in rawInput too for non-execute UIs that show it.
- Rework the acp tool-call presentation tests (turns + load replay) to drive the
REAL dsh-tool-bash + dsh-bash-local via a new makeBridgeHarness({ withBash })
option, running an actual `echo` — instead of an inline fake bash tool. The
mock MODEL still scripts the call (deterministic, no key), but the tool and
executor are real, so the test verifies the shipping presentCall/presentResult.
- AGENTS.md: add the principle "prefer the REAL implementation over a mock/
stand-in in tests" (mock only the expensive/non-deterministic boundary).
- RFC (proposed): the ACP terminal sub-protocol + command classification — the
capability-gated rich rendering (live cwd-header terminal card, classify a
`cat` as a read / `grep` as a search) that the reference adapters do; the
fenced ```console text block stays the no-capability baseline. Studied
codex-acp, claude-agent-acp, and Zed's renderer to ground it.
In Zed the tool-call card showed only "bash" — the bare tool name — instead
of what the command does. Fix it by letting each TOOL own how its calls render,
rather than the bridge special-casing names.
dsh-tools: add an optional two-state presentation seam to ToolDefinition /
defineTool — `presentCall(args)` (pending: title, kind, rawInput) and
`presentResult(args, result)` (completed: title?, content?). Provider-neutral
`ToolCallKind`/`ToolCallPresentation`/`ToolResultPresentation` vocabulary so
tools never depend on ACP. defineTool soft-validates args (display runs on log
replay, so a malformed/old shape returns undefined instead of throwing).
dsh-tool-bash: bash declares presentCall (model `description` → title, exact
`command` → rawInput, kind execute) and presentResult (wrap output in a fenced
```console block — a UI-only affordance kept out of the model-facing result);
bash_output/bash_kill present task-scoped titles.
dsh-acp: inject `tools`; a per-session `ToolPresenter` looks the tool up by name
and maps its neutral presentation to the ACP tool_call/tool_call_update wire
shape, with a generic fallback (title = name) for tools that declare nothing.
Because the `tool/result` event carries only {callId, content, isError}, the
presenter keeps a small bridge-local map of ONLY in-flight calls' (name, args),
keyed by callId and removed as each result is presented — no event-schema or
core change. Replay uses a throwaway presenter so loaded sessions render
identically to live ones.
Tests: dsh-tools defineTool presenters (typed args, soft-validate), tool-bash
bash/bash_output/bash_kill presenters, acp ToolPresenter (tool-owned mapping,
unknown-callId fallback, in-flight-only map), and an end-to-end turn through the
bridge. The key-gated e2e now asserts a real bash call's title is the model
description (not "bash") and rawInput is the command — verified against the real
DeepSeek model. The test harness derives its inject from the bridge's exported
`inject` so it can't drift again.
- AgentLoop.resume uses `this.ctx.get('sessionPersistence')` (strict) instead
of the `, false` overload: still topology-independent, but an inactive/
absent backend reads as undefined (rejected by the existing guard) rather
than being handed back mid-teardown.
- Correct the bridge teardown comment: an ACP-created agent's registry entry
binds to the BRIDGE fiber (the factory is reached through the bridge's
traceable proxy, so AgentLoop.start's `this.ctx.effect` registration uses the
caller context), not the AgentLoop fiber — so an ACP-only HMR dispose
reclaims it. Add a regression test pinning that ownership.
- Sync the ctx.get guidance in the post-mortem, packages/AGENTS.md, and the
dsh-code-review skill to the strict form.
Two independent bugs made the ACP server crash the moment an editor (Zed)
connected, despite 178 green unit tests at 100% coverage:
1. `session/new` threw `cannot get property "agents" without inject`. Root
cause: a stray `export default apply` made the cordis Loader's
`unwrapExports` (`exports.default ?? exports`) collapse the module to the
bare `apply` function, discarding the sibling `inject`/`name`/`Config`
named exports. The plugin fiber was built with empty `inject`, so every
`ctx.<service>` read in `apply` threw at load. Fix: remove the default
export so the Loader uses the namespace.
2. `session/load` threw `cannot get property "sessionPersistence" without
inject`. `AgentLoop.resume` read `this.ctx.sessionPersistence` (a service
it deliberately does NOT inject); the property proxy's ancestor-only fiber
walk fails through the bridge's traceable shadow. Fix: read it via
`this.ctx.get('sessionPersistence', false)`, the topology-independent
global-store lookup.
Why the suite missed both: every test mounted the plugin by hand
(`ctx.plugin({name,inject,apply})`), bypassing `unwrapExports` entirely, and
the only test driving these RPCs was key-gated (skipped in CI). Added a no-key
`session/new` e2e that boots the real example through the real Loader — it
fails loudly on bug #1 without an API key. Set `TSX_TSCONFIG_PATH` in the e2e
spawn so the subprocess resolves workspace `paths` from a temp cwd (it was
silently falling back to a stale built `lib/`).
Docs: post-mortem 0001; AGENTS.md "line coverage is not behavior coverage" +
with-key/smoke-test philosophy; packages/AGENTS.md plugin-export-shape and
ctx.get rules; dsh-code-review SKILL checks.
- Fix two root-AGENTS.md cross-links that the depth bump left pointing at the
new docs/AGENTS.md instead of the root file they cite (capability-seams,
optional-code-mode). These resolved on disk so verify-md-links passed — the
gate checks existence, not which file you meant; corrected to ../../../.
- Broaden verify-md-links scope to .agents/skills/**/*.md: this PR rewrote the
dsh-code-review skill's links into the RFC tree, but the skill dir was outside
the gate, so a broken skill link would have passed silently.
- Percent-decode the path component before the existence check, so a valid
encoded relative target (My%20File.md) is not falsely reported broken; a
malformed escape (%zz) is reported broken rather than crashing the gate.
- Drop the merged property-testing RFC's "nightly CI job 100x" claim: that line
came from the original proposal, not the accepted decision, and CI has only
push/pull_request triggers — note it as possible future work instead.
doc-sync (incl. verify-md-links over 58 files), doc-typecheck, lint pass.
Collapse docs/adr/ and docs/rfc/ into a single docs/rfc/ with proposed/,
implemented/, and rejected/ subfolders. Every file is renamed to
yyyy-mm-dd-topic-title.md, where the date is when the topic was first
proposed (from git history). ADRs and RFCs that covered exactly the same
topic are merged (property-based testing, session persistence); the
umbrella RFC 005 stays split across its three implemented decisions, and
RFC 006's deferred part-3 (API extractor reports) splits into its own
proposed RFC. All cross-references become machine-checkable relative
links instead of bare "ADR NNNN" / "RFC NNN" prose.
Add a verify-md-links doc-sync gate (scripts/verify-md-links.ts) that
checks every relative Markdown cross-link resolves, wired into doc-sync
alongside verify-md-wrap. This makes the reorganization self-verifying:
the same change that rewrote ~forty inter-doc links adds the check that
proves none dangle. Document the cross-link convention in a new
docs/AGENTS.md and record the gate as an implemented RFC.
doc-sync, typecheck, lint, and the full test suite (667) all pass.
Lifts the RFC 010 § Deferred restriction that the server had to launch in the
workspace ("cwd must equal the launch directory"). An editor can now open any
project folder, and N concurrent sessions over one connection can each target a
different directory.
- packages/acp: drop the `cwd === process.cwd()` guard in validateWorkspaceParams
(keep "must be absolute" — the cwd becomes the session header / bash workdir),
and drop the persisted-cwd-vs-launch-dir check in session/load (a resumed
session keeps its original header.cwd, so its bash tools run in its workspace).
- packages/tool-bash: the missing link — default the bash workdir to the calling
agent's session cwd (`exec.agent.session.header.cwd`) via a new resolveWorkdir
helper. An explicit model `workdir` still wins; a relative one resolves against
the session cwd. This is the only correct spot for multi-session: N sessions
share one ctx.bash executor, so the workdir must come per-call from exec.agent,
not executor config. Falls back to the executor default when no session cwd is
available (preserves non-ACP behavior).
- Trust: the cwd originates from the ACP client (the user's editor) at
session/new — same trust level as the old launch dir; no new untrusted-input
path. `additionalDirectories` (scope widening / sandbox) stays rejected.
- Tests: bridge accepts any absolute cwd + records it on the header; session/load
honors the persisted cwd; bash defaults to / resolves relative against the
session cwd; two sessions with different cwds each run bash in their own dir;
non-absolute cwd still rejected. 100% per-file coverage maintained.
- Docs: RFC 010 status + § Deferred cwd bullet marked RESOLVED; acp README adds a
Per-session cwd section; tool-bash + example READMEs and e2e comments updated.
Adding verify-md-wrap to the shared doc-sync gate left its defining docs
stale (Codex review):
- ADR 0014 described doc-sync as two gates; add a dated amendment for the
third (verify-md-wrap) and drop the "two checkable classes" wording.
- CI step label/comment said "doc code blocks + event taxonomy"; include
the markdown wrap check.
The loop logs the assistant/message (carrying tool-call blocks) BEFORE running
the tools, so a crash mid-tool leaves durable tool calls with no matching
tool/result. interruptedTurnClosers only added step/end + turn/end, so a resumed
session's deriveMessages() replayed a dangling assistant tool-call — which every
provider rejects as an invalid transcript on the next request.
interruptedTurnClosers now scans the interrupted turn for tool-call blocks
without a matching tool/result and synthesizes an error tool/result for each
(before the step/end), so the rehydrated history is a valid transcript. Adds a
dedicated repair.spec.ts and a shared-contract case proving both backends pair
every orphaned call with a result. Docs (ADR 0018, both persistence READMEs,
load() JSDoc) updated.
Also fixes the echo-agent README session-cleanup path: demo:echo runs from the
repo root, so sessions land in <repo-root>/.sessions/_no-cwd/, not
examples/echo-agent/.sessions/ (review #33).
imccyu
Tianyi Cui