Commit Graph
24 Commits
Author SHA1 Message Date
Tianyi Cui 90a19f072d docs(acp,rfc): fix stale ownership wording + propose unifying agent/session id (review)
Review follow-ups on the bash owner-token PR:

- packages/acp/README.md still described task isolation in object-identity terms
  ("records each background task's owning agent", "a different agent"). Rewrite
  to the session-token model: ownership is by `session.header.id`, stored on the
  executor's task, so a different Agent object on the same session may access it
  and ownership survives a tool-bash HMR reload.

- The reviewer flagged that the notice routes by `session.header.id` while the
  registry only enforces unique `agent.id`, so a programmatic caller could
  register two agents sharing a session token and mis-route a notice (not
  reachable via ACP). Rather than bolt a session-id invariant onto the generic
  registry, add a proposed RFC (2026-06-20-unify-agent-and-session-id) to remove
  the precondition by construction — an agent IS its session, one id — with a
  full risks discussion (forecloses multi-session-actor / fork futures, makes the
  config resume-or-create policy load-bearing, migration churn). The actual
  unification ships as its own Codex-converged PR. Cross-linked from the
  agent-lifecycle RFC's seam-precondition note.

- Reframe the tool-bash module-doc ownership paragraph to current-state (per the
  new AGENTS.md doc convention): contrast storing the token on the executor vs
  in the plugin as a standing rationale, not as "closing the old gap".
2026-06-20 13:38:48 +08:00
Tianyi Cui b58f1dd5c8 refactor(tool-bash): own background tasks by session token, not a plugin-local Map
Delete the `taskOwner: Map<string, Agent>` entirely — it served two roles
(access control AND holding a live Agent for completion notices), both now
stateless:

- Access control: `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to
  the caller's token (`exec.agent?.session.header.id`) with `!== undefined`
  semantics (an empty-string token is still a real owner). The owner is stamped
  at spawn via `resolve({ …, owner })`. Ownership now lives on the task in the
  executor, so it SURVIVES a tool-bash HMR reload — closing the old
  XXX(tool-bash-owner-hmr) gap.
- Completion notice: `onTaskDone` reads `ctx.bash.ownerOf(task.id)` and finds
  the live agent by scanning `ctx.get('agents')?.list()` for a matching
  `session.header.id` (read via `ctx.get` — the listener runs on the bash
  fiber, a foreign fiber, where the `ctx.agents` proxy would throw). No
  registry / owner gone → drop the notice cleanly.

Token is `session.header.id` (NOT `session.id`): every other subsystem keys off
the header id, and the test fakes populate only `session.header.id`, so reading
`session.id` would make every fake unowned and pass the isolation tests for the
wrong reason.

Tests give A and B DISTINCT real session tokens (a same-token-different-Agent
case is now ALLOWED — identity no longer matters); the HMR test inverts to
assert ownership SURVIVES a tool-bash reload; a new test covers the
owner-agent-gone-before-completion drop. Migrates the agent-lifecycle RFC
proposed->implemented (recording all three seams + the session-id-uniqueness
precondition) and updates the tool-bash README + the now-implemented RFC's
cross-links.
2026-06-20 08:14:27 +08:00
Tianyi Cui ab02e9acec refactor(session-persistence): extract a shared write coordinator
The JSONL and SQLite backends were byte-identical (or same-algorithm) for ALL
of their write-path orchestration — the four maps (states/buffers/chains/inits),
installWritePath, initFor, onCreated's four adoption cases, flush, drain,
serialize, adopt/adoptLivePrefix, assertVersion, and the create/append/load/
has/delete skeletons. Only the storage primitives (write bytes vs INSERT rows)
differed, so every fix landed twice.

Extract that orchestration into a PersistenceCoordinator in the seam package.
Each backend composes one (new PersistenceCoordinator(ctx, this)), implements a
small PersistenceBackend hook interface (loadStored, loadLive, appendBatch,
commitRepair, deleteStored, list, optional close), and delegates its six public
service methods to it. Composition, not inheritance — a backend exposes only the
hooks, can't reach the coordinator's private state, and the public
SessionPersistence API is unchanged so a third-party backend may still implement
it directly.

The crash-repair torn-tail token is OPAQUE: the coordinator computes the
synthetic closers (it owns interruptedTurnClosers) but only tests
`tornMarker !== undefined` and round-trips it to commitRepair, never inspecting
it (JSONL = byte offset, SQLite = seq). loadStored vs loadLive stay distinct so
HMR adoption is cwd-scoped (a same-id log at a different cwd is a collision, not
a resume). appendBatch carries meta so lazy-materialize + first-batch commit
atomically (no separate materialize hook).

Tests: the duplicated orchestration tests (adoption, HMR, collision,
dispose-drain, crash-tail) move into one runCoordinatorContract suite run once
per backend (memory + jsonl + sqlite) via hook fixtures; per-backend specs keep
only storage mechanics. A through-coordinator torn-tail test per real backend
keeps the commitRepair-with-marker branch covered under the 100% gate.

Net -112 lines (the dedup outweighs the new coordinator + shared suite); 100%
coverage; backends shrank ~1200 lines of duplicated churn. Migrates the
write-coordinator RFC proposed -> implemented.
2026-06-20 03:47:28 +08:00
Tianyi Cui 815bac7de9 refactor(session): drop the dead mutable SessionSummary
SessionSummary (updatedAt/title/firstPrompt) and SessionPersistence.update()
were dead state: zero production callers of update(), no production reader of
updatedAt/firstPrompt, and ACP's title comes from a tool-call presenter, not
storage. The live Session.header was already typed SessionHeader, so the
summary only ever existed in the persistence layer, written and read by nothing
but its own contract test.

Delete it entirely (no SessionMeta alias — SessionMeta collapses to
SessionHeader everywhere). This removes the JSONL .summary.json sidecar
machinery, the SQLite title/first_prompt/updated_at columns and per-append
updated_at bump, and the update() method from the abstract service and both
backends. SQLite SCHEMA_VERSION goes 1->2 and openDatabase now rejects any
non-current user_version (older or newer) — no migration, unreleased software.

Net -400 lines, and it erases the JSONL-sidecar-vs-SQLite-column durability
divergence that the upcoming write coordinator would otherwise have to model.

Records the decision in docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md
and migrates the 2026-06-14 session-persistence RFC's facts to current truth.
Adds a standalone AGENTS.md section "Tests document behavior, not golden truth"
(a passing test pins current behavior, not necessarily correct behavior) with
the summary-drop as its worked example, and reinforces the no-migration
pre-release stance.
2026-06-20 01:03:57 +08:00
Tianyi Cui 0561fb47b6 Merge pull request #64 from deepseek-ai/worktree-e2e-real-api-workflow
ci: add real-API e2e workflow against external DeepSeek API
2026-06-19 18:53:32 +08:00
Tianyi Cui 27721f9f45 docs(rfc): record real-API e2e CI decision + security model
Adds docs/rfc/implemented/2026-06-19-real-api-e2e-ci.md covering the rationale
for running the real-API e2e suite in a separate secret-consuming workflow, the
fork/Dependabot/secret threat model, the residual exposure of the pull_request
trigger, and what changes when the repo goes public. Indexes it in the RFC
README.

Also adds a SECURITY comment on the pull_request trigger forbidding a switch to
pull_request_target (an untrusted-code-with-secrets leak vector, especially once
public), pointing at the RFC.
2026-06-19 18:42:27 +08:00
Tianyi Cui 5bccda19d3 docs(rfc): keep implemented RFCs current; rewrite the llm-replay section in place
- Rewrite the snapshot-test RFC's replay-plugin section to state current
  reality directly (the plugin is the @deepseek-ai/dsh-llm-replay package,
  under the coverage gate) instead of keeping the old example-local text with a
  "superseded" note bolted on.
- Add docs/rfc/implemented/AGENTS.md (+ CLAUDE.md symlink): an implemented RFC
  must be kept current with what actually shipped — update paths/names/structure
  in the same change that moves the code, in place, not as an append-only
  changelog of its own drift. A reversal of the DECISION is still a new RFC.
- Reconcile docs/rfc/README.md: the "never edited into a different decision"
  rule now distinguishes tracking where a decision lives (required) from
  flipping the decision (forbidden), and the implemented/ bullet points at the
  new convention.
2026-06-19 17:06:20 +08:00
Tianyi Cui 4f291953ea fix(ui-stdio): cancel EOF-exit timer on dispose; harden config defaults; doc fixes
Round 1 of Codex review on the extraction PR.

- (B) EOF-exit race: the 200ms flush-then-exit setTimeout was untracked, so a
  fiber/HMR dispose within that window could not cancel it and the process
  would still exit. Track the handle and clear it in the disposer; coalesce
  re-entrant maybeExit() calls onto the one pending timer. Regression tests for
  both (dispose-within-window cancels; repeated idle schedules once).
- (B/doc) ui-stdio rendering is global, not scoped by config.agent (faithful to
  the original copies — agent scopes only input + the EOF-exit gate). Corrected
  the README + Config JSDoc, which overclaimed "drive and render".
- (C) createStdioChat is exported and driven directly by tests/programmatic
  callers that bypass schemastery validation, so default welcome/agent in the
  helper (?? 'ready.'/'main') instead of trusting the cast. Test for empty config.
- (A/doc) docs/rfc/.../acp-snapshot-tests.md asserted the replay plugin
  deliberately stays in examples/ ("don't split preemptively") — now false since
  this PR packages it. Added a superseding note with the why (coverage gate).

All gates green: typecheck, lint, test:coverage (891, 100%), doc-sync,
test:e2e (6 keyless pass), test:snapshot unaffected.
2026-06-19 14:53:07 +08:00
Tianyi Cui 7731bc80c3 Merge remote-tracking branch 'origin/master' into feat/acp-snapshot-tests 2026-06-19 10:03:04 +08:00
Tianyi Cui 76fceae5b3 feat(acp-example): per-scenario workspace/ seeding + a real file-edit scenario
Establishes the standard way to give a snapshot scenario a non-empty starting
workspace: an optional `<scenario>/workspace/` directory whose contents the
harness copies into the temp cwd before the run (for both record and replay),
so the agent's bash tools see the seeded files. The cwd is normalized in the
goldens, so seeded paths stay stable.

The new `workspace-edit` scenario demonstrates the full read→write→verify cycle
on a seeded file: it ships `workspace/greeting.txt` ("hello"), prompts the agent
to append a WORLD line and cat it back. The recorded log captures the real bash
edits (`echo WORLD >> greeting.txt`, then `cat` showing `hello\nWORLD`), and it
replays deterministically with no key.

Also hardens runScenario teardown (Codex review): workspace seeding and spawn
now run inside the try whose finally removes both temp dirs, so a seeding/spawn
failure can't leak them. Documents the convention in the RFC + example README.
2026-06-19 10:01:42 +08:00
Tianyi Cui 679aaacfc4 refactor(examples): DRY the acp-agent configs via base-core.yml + acp-tail.yml
The snapshot replay config duplicated most of base.yml + the acp tail just to
swap llm-deepseek → llm-replay. Factor the shared pieces:

- examples/base-core.yml: the providerless provider/tool core (llm, sessions,
  system-prompt, tools, agents, invariants, bash-local, tool-bash). base.yml is
  now base-core + the llm-deepseek adapter; the snapshot replay config is
  base-core + llm-replay. The replay config no longer hand-copies the core.
- examples/acp-agent/acp-tail.yml: agent-loop (no pre-created agents) +
  persistence + the ACP bridge/system-prompt, shared by cordis.yml and the
  replay config so the three acp-agent configs can't drift. Its persistence root
  is `$DSH_SNAPSHOT_SESSIONS_ROOT ?? ./.sessions`.
- Deleted cordis.snapshot-record.yml: recording now reuses the normal cordis.yml
  (real adapter), with the harness redirecting the persistence root via env.
  start.ts maps DSH_SNAPSHOT=record → cordis.yml.

Verified: snapshot replay 8/8 keyless; record path works through cordis.yml;
ACP e2e no-key boot green through the doubly-nested include (cordis.yml →
base.yml → base-core.yml); coding-agent boots clean; all gates pass.
2026-06-19 09:41:10 +08:00
Tianyi Cui ec73e11c9b refactor(acp-example): snapshot goldens are JSONL, not pretty-printed .txt
The goldens now mirror the shape of the surfaces they capture — one compact
JSON record per line — matching the wire (NDJSON stdout) and disk (JSONL
session log) formats, renamed *.golden.jsonl. They stay grep/jq-able and
faithful to what the agent emits, where the prior pretty-printed .txt was a
reformatted representation. Both normalizers drop the 2-space indent; the
normalizer spec asserts the compact form. All 11 goldens regenerated; replay
remains deterministic (8/8 across runs).
2026-06-19 09:29:41 +08:00
Tianyi Cui 4a1c64663b docs: retire completed tagged-envelope review TODO 2026-06-19 09:28:23 +08:00
Tianyi Cui c182543dd5 refactor(acp-example): derive llm-replay script from the session JSONL
Per a design revision, the per-scenario snapshot fixture becomes EXACTLY the
persisted session JSONL (<scenario>/session.jsonl) rather than a hand-authored
llm.json. The log already holds all LLM behavior (assistant/chunk carries every
StreamChunk) AND all harness behavior (tool/call, tool/result, turn/*, usage),
so one artifact drives replay and doubles as a behavioral golden.

llm-replay becomes replay-only (the record-tee is removed; recording is now
"run the real agent once and harvest the .jsonl", done by the harness in a
later commit). deriveReplayScript(events) groups assistant/chunk by (turn,step)
in log order — exact because the loop makes one ctx.llm.stream() call per step
and tags each chunk with the current (turn,step). The two failure modes the log
can't express (a thrown stream — no terminal finish; cancel/hang — timing) use
an optional replay.override.json sidecar.

Hardens against a Codex review finding: a derived group is only valid if it
ends in a `finish` chunk. A group without one is the fingerprint of a thrown
stream() and is NOT silently replayed as a clean stop — deriveReplayScript
throws, naming the (turn,step), so a missing sidecar override fails loud.

Updates the unit tests (parse/derive/load helpers, sidecar override, finish-
terminated grouping, HMR), the example README, and the RFC prose to the JSONL
format. Two goldens (stdout transcript + re-persisted JSONL) and the harness
wiring land in the next commit.
2026-06-19 02:44:33 +08:00
Tianyi Cui 49bb6a88eb fix(tool-bash): handle unavailable spill paths 2026-06-19 01:54:57 +08:00
Tianyi Cui bef9386591 docs(rfc): add ACP snapshot tests RFC (record-once / replay-deterministic)
Records the decision to add a third test tier: snapshot tests that boot the
real acp-agent subprocess over ACP stdio, record the LLM's streamed responses
once against the real API, then replay them deterministically so the full
stdout transcript can be diffed against a committed golden — keyless in CI.

Captures the design choices hardened in a Codex (xhigh) review: record at the
provider-neutral llm/stream waterfall; a discriminated fixture entry schema
(chunks/throw/hang) that honors both LLM failure branches; positional replay
with a one-in-flight-stream constraint; per-stream atomic fixture flush (the
subprocess is SIGKILLed, so dispose-time flush would never run); a providerless
replay config; normalize-then-snapshot parsed frames; normalization over an OS
sandbox now with the rootless bwrap/sandbox-exec tier reserved via the
BashExecutor capability seam. Cross-links the proposed determinism RFC
(complementary: internal history invariant vs external protocol contract).
2026-06-19 00:19:54 +08:00
Tianyi Cui 7fa113be0e Merge remote-tracking branch 'origin/master' into codex/pr48-repo-hardening-rfcs
# Conflicts:
#	docs/adr/README.md
#	docs/rfc/009-session-persistence-and-resumability.md
#	docs/rfc/README.md
#	docs/rfc/implemented/2026-06-11-doc-sync-enforcement.md
#	docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md
#	examples/acp-agent/tests/acp.e2e.ts
#	packages/acp/README.md
#	packages/acp/src/index.ts
#	packages/acp/tests/stream-update.spec.ts
#	packages/agent-loop/src/loop.ts
#	packages/tools/src/index.ts
2026-06-18 23:41:14 +08:00
Tianyi Cui 9f6b96c555 fix(acp): address review of the terminal-card alignment (exit parse, background/error, capability snapshot)
Codex + an independent review pass found three real defects in the prior commit:

1. parseExitStatus could misreport a SUCCESSFUL command as a failure: a clean
   exit 0 appends no marker, so output ending in "[exit code: 5]" (no trailing
   newline) was read as the marker. Anchor the parse to a LEADING newline —
   renderResult always inserts one before a real marker, so a body that merely
   ends in marker-like text no longer matches. A narrow residual (a clean exit 0
   whose final line is exactly the marker) is inherent to the replay-only-sees-
   text design and documented; the complete fix (a structured exit on the event)
   is the RFC's named escape hatch.

2. A run_in_background start and an isError result were rendered as exited
   terminal cards with a false exit-0 pill. A background start returns a task-id
   ack (not a streamed terminal) and is no longer marked terminal; an isError
   result (spawn failure / abort) carries no exit pill.

3. The terminal capability was re-read live on the result path, so a second
   initialize between a call and its result could desync them (orphan
   terminal_output or clobbered card). Snapshot the capability per session at
   creation (SessionRecord.terminalEnabled) so call and result always agree.

Also reword the reference-parity claim: keeping the description as a content
block in terminal mode is a DELIBERATE divergence (claude-agent-acp drops it).
Tests added for each; with-key e2e still green.
2026-06-18 19:35:15 +08:00
Tianyi Cui e51dabbb8b feat(acp): align bash terminal card with reference adapters (command title, description block, exit pill)
Match claude-agent-acp / codex-acp: the bash tool_call title IS the command
(an execute card hides rawInput), the model description rides as a content
text block above the card, and the completed card carries an exit-status pill
via _meta.terminal_exit.

Bridge fixes found in review of the prior terminal-card commit:
- tool_call_update.content is OMITTED in terminal mode (an ACP update.content
  REPLACES the call's content collection in Zed, so the fenced ```console block
  would clobber the terminal content block).
- terminal.output preserves RAW newlines (terminal renderers rely on exact
  bytes); only the fenced fallback trims trailing blank lines.
- a relative workdir is resolved against the session cwd for the card header,
  matching where the command actually ran.
- result-side terminal output is gated on the pending call having registered a
  terminal (no orphan _meta.terminal_output for a terminal Zed never made).

The exit pill is recovered by parsing renderResult's status markers (the pure
presentResult seam sees only content blocks); a round-trip test pins the parse
to the marker emission. Neutral ToolTerminal gains exitCode/signal; widened
ToolCallPresentation with a content block. Docs (RFC + 3 READMEs) updated;
with-key e2e verifies the card + exit pill against the real model.
2026-06-18 18:54:32 +08:00
Tianyi Cui c8dbe6567a docs(acp): clarify _meta is a spec extensibility point; the terminal keys are the Zed convention 2026-06-18 17:34:50 +08:00
Tianyi Cui 149ab1bba4 feat(acp): render bash as a terminal card via the _meta convention
When the client advertises clientCapabilities._meta.terminal_output (Zed), a
bash tool call now renders as a real TERMINAL card — a cwd header + the command
+ its output — instead of the plain ```console text block. Keeps agent-side
dsh-bash execution; rejects the spec's client-side terminal/create (which would
bypass sandbox/env-scrub/ownership/cwd). Matches what claude-agent-acp and
codex-acp do; wire contract verified against Zed's source.

- dsh-tools: a provider-neutral ToolTerminal shape ({ cwd?, output? }) on
  ToolCallPresentation/ToolResultPresentation — a tool asks "render me as a
  terminal"; no ACP types leak in.
- dsh-tool-bash: bash presentCall marks terminal (cwd from an explicit absolute
  workdir, else left for the bridge to fill from the session cwd); presentResult
  carries the output alongside the ```console fallback.
- dsh-acp: initialize reads/remembers the _meta.terminal_output capability;
  streamSessionEventUpdate maps a terminal presentation to
  content:[{type:'terminal',terminalId}] + _meta.terminal_info on the call and
  _meta.terminal_output on the update WHEN capable — else the unchanged text
  path. terminalId is the callId; cwd defaults to the session header. The pure
  translator gained a TerminalRendering {enabled,cwd} param (off by default).

Tests via the REAL tool-bash + bash-local: capability ON -> terminal content +
_meta; OFF -> no _meta (text path). The with-key e2e adds a real-model terminal
card case (echo over ACP with the capability on). 773 tests, 100% coverage.

The exit-status pill (_meta.terminal_exit), live streaming
(_meta.terminal_output_delta), and command classification are RFC follow-ups.
2026-06-18 17:25:09 +08:00
Tianyi Cui 386ee14af3 docs(acp): RFC for the terminal-card rendering (implemented design)
Records the verified design before implementing: keep dsh-bash agent-side
execution and render Zed's terminal tool-call card via the `_meta` convention
(terminal_info/terminal_output/terminal_exit), capability-gated on
clientCapabilities._meta.terminal_output, with the ```console text block as the
no-capability fallback. Rejects the spec's client-side terminal/create path (it
would bypass dsh-bash's sandbox/env-scrub/ownership/cwd). Studied
claude-agent-acp, codex-acp, and Zed's renderer to ground the wire contract.
Live streaming and command classification are noted as separate follow-ups.
2026-06-18 14:36:02 +08:00
Tianyi Cui 27f5f84e3b docs: address Codex review of the RFC reorg
- Fix two root-AGENTS.md cross-links that the depth bump left pointing at the
  new docs/AGENTS.md instead of the root file they cite (capability-seams,
  optional-code-mode). These resolved on disk so verify-md-links passed — the
  gate checks existence, not which file you meant; corrected to ../../../.
- Broaden verify-md-links scope to .agents/skills/**/*.md: this PR rewrote the
  dsh-code-review skill's links into the RFC tree, but the skill dir was outside
  the gate, so a broken skill link would have passed silently.
- Percent-decode the path component before the existence check, so a valid
  encoded relative target (My%20File.md) is not falsely reported broken; a
  malformed escape (%zz) is reported broken rather than crashing the gate.
- Drop the merged property-testing RFC's "nightly CI job 100x" claim: that line
  came from the original proposal, not the accepted decision, and CI has only
  push/pull_request triggers — note it as possible future work instead.

doc-sync (incl. verify-md-links over 58 files), doc-typecheck, lint pass.
2026-06-18 02:41:19 +08:00
Tianyi Cui 7c400e9c02 docs: unify ADR/RFC trees into one lifecycle-organized RFC tree
Collapse docs/adr/ and docs/rfc/ into a single docs/rfc/ with proposed/,
implemented/, and rejected/ subfolders. Every file is renamed to
yyyy-mm-dd-topic-title.md, where the date is when the topic was first
proposed (from git history). ADRs and RFCs that covered exactly the same
topic are merged (property-based testing, session persistence); the
umbrella RFC 005 stays split across its three implemented decisions, and
RFC 006's deferred part-3 (API extractor reports) splits into its own
proposed RFC. All cross-references become machine-checkable relative
links instead of bare "ADR NNNN" / "RFC NNN" prose.

Add a verify-md-links doc-sync gate (scripts/verify-md-links.ts) that
checks every relative Markdown cross-link resolves, wired into doc-sync
alongside verify-md-wrap. This makes the reorganization self-verifying:
the same change that rewrote ~forty inter-doc links adds the check that
proves none dangle. Document the cross-link convention in a new
docs/AGENTS.md and record the gate as an implemented RFC.

doc-sync, typecheck, lint, and the full test suite (667) all pass.
2026-06-18 02:18:24 +08:00