Commit Graph
25 Commits
Author SHA1 Message Date
Tianyi Cui cc47f76cea docs: propose simplification RFCs 2026-06-20 16:33:03 +08:00
Tianyi Cui 90a19f072d docs(acp,rfc): fix stale ownership wording + propose unifying agent/session id (review)
Review follow-ups on the bash owner-token PR:

- packages/acp/README.md still described task isolation in object-identity terms
  ("records each background task's owning agent", "a different agent"). Rewrite
  to the session-token model: ownership is by `session.header.id`, stored on the
  executor's task, so a different Agent object on the same session may access it
  and ownership survives a tool-bash HMR reload.

- The reviewer flagged that the notice routes by `session.header.id` while the
  registry only enforces unique `agent.id`, so a programmatic caller could
  register two agents sharing a session token and mis-route a notice (not
  reachable via ACP). Rather than bolt a session-id invariant onto the generic
  registry, add a proposed RFC (2026-06-20-unify-agent-and-session-id) to remove
  the precondition by construction — an agent IS its session, one id — with a
  full risks discussion (forecloses multi-session-actor / fork futures, makes the
  config resume-or-create policy load-bearing, migration churn). The actual
  unification ships as its own Codex-converged PR. Cross-linked from the
  agent-lifecycle RFC's seam-precondition note.

- Reframe the tool-bash module-doc ownership paragraph to current-state (per the
  new AGENTS.md doc convention): contrast storing the token on the executor vs
  in the plugin as a standing rationale, not as "closing the old gap".
2026-06-20 13:38:48 +08:00
Tianyi Cui b58f1dd5c8 refactor(tool-bash): own background tasks by session token, not a plugin-local Map
Delete the `taskOwner: Map<string, Agent>` entirely — it served two roles
(access control AND holding a live Agent for completion notices), both now
stateless:

- Access control: `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to
  the caller's token (`exec.agent?.session.header.id`) with `!== undefined`
  semantics (an empty-string token is still a real owner). The owner is stamped
  at spawn via `resolve({ …, owner })`. Ownership now lives on the task in the
  executor, so it SURVIVES a tool-bash HMR reload — closing the old
  XXX(tool-bash-owner-hmr) gap.
- Completion notice: `onTaskDone` reads `ctx.bash.ownerOf(task.id)` and finds
  the live agent by scanning `ctx.get('agents')?.list()` for a matching
  `session.header.id` (read via `ctx.get` — the listener runs on the bash
  fiber, a foreign fiber, where the `ctx.agents` proxy would throw). No
  registry / owner gone → drop the notice cleanly.

Token is `session.header.id` (NOT `session.id`): every other subsystem keys off
the header id, and the test fakes populate only `session.header.id`, so reading
`session.id` would make every fake unowned and pass the isolation tests for the
wrong reason.

Tests give A and B DISTINCT real session tokens (a same-token-different-Agent
case is now ALLOWED — identity no longer matters); the HMR test inverts to
assert ownership SURVIVES a tool-bash reload; a new test covers the
owner-agent-gone-before-completion drop. Migrates the agent-lifecycle RFC
proposed->implemented (recording all three seams + the session-id-uniqueness
precondition) and updates the tool-bash README + the now-implemented RFC's
cross-links.
2026-06-20 08:14:27 +08:00
Tianyi Cui ab02e9acec refactor(session-persistence): extract a shared write coordinator
The JSONL and SQLite backends were byte-identical (or same-algorithm) for ALL
of their write-path orchestration — the four maps (states/buffers/chains/inits),
installWritePath, initFor, onCreated's four adoption cases, flush, drain,
serialize, adopt/adoptLivePrefix, assertVersion, and the create/append/load/
has/delete skeletons. Only the storage primitives (write bytes vs INSERT rows)
differed, so every fix landed twice.

Extract that orchestration into a PersistenceCoordinator in the seam package.
Each backend composes one (new PersistenceCoordinator(ctx, this)), implements a
small PersistenceBackend hook interface (loadStored, loadLive, appendBatch,
commitRepair, deleteStored, list, optional close), and delegates its six public
service methods to it. Composition, not inheritance — a backend exposes only the
hooks, can't reach the coordinator's private state, and the public
SessionPersistence API is unchanged so a third-party backend may still implement
it directly.

The crash-repair torn-tail token is OPAQUE: the coordinator computes the
synthetic closers (it owns interruptedTurnClosers) but only tests
`tornMarker !== undefined` and round-trips it to commitRepair, never inspecting
it (JSONL = byte offset, SQLite = seq). loadStored vs loadLive stay distinct so
HMR adoption is cwd-scoped (a same-id log at a different cwd is a collision, not
a resume). appendBatch carries meta so lazy-materialize + first-batch commit
atomically (no separate materialize hook).

Tests: the duplicated orchestration tests (adoption, HMR, collision,
dispose-drain, crash-tail) move into one runCoordinatorContract suite run once
per backend (memory + jsonl + sqlite) via hook fixtures; per-backend specs keep
only storage mechanics. A through-coordinator torn-tail test per real backend
keeps the commitRepair-with-marker branch covered under the 100% gate.

Net -112 lines (the dedup outweighs the new coordinator + shared suite); 100%
coverage; backends shrank ~1200 lines of duplicated churn. Migrates the
write-coordinator RFC proposed -> implemented.
2026-06-20 03:47:28 +08:00
Tianyi Cui 815bac7de9 refactor(session): drop the dead mutable SessionSummary
SessionSummary (updatedAt/title/firstPrompt) and SessionPersistence.update()
were dead state: zero production callers of update(), no production reader of
updatedAt/firstPrompt, and ACP's title comes from a tool-call presenter, not
storage. The live Session.header was already typed SessionHeader, so the
summary only ever existed in the persistence layer, written and read by nothing
but its own contract test.

Delete it entirely (no SessionMeta alias — SessionMeta collapses to
SessionHeader everywhere). This removes the JSONL .summary.json sidecar
machinery, the SQLite title/first_prompt/updated_at columns and per-append
updated_at bump, and the update() method from the abstract service and both
backends. SQLite SCHEMA_VERSION goes 1->2 and openDatabase now rejects any
non-current user_version (older or newer) — no migration, unreleased software.

Net -400 lines, and it erases the JSONL-sidecar-vs-SQLite-column durability
divergence that the upcoming write coordinator would otherwise have to model.

Records the decision in docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md
and migrates the 2026-06-14 session-persistence RFC's facts to current truth.
Adds a standalone AGENTS.md section "Tests document behavior, not golden truth"
(a passing test pins current behavior, not necessarily correct behavior) with
the summary-drop as its worked example, and reinforces the no-migration
pre-release stance.
2026-06-20 01:03:57 +08:00
Tianyi Cui 0561fb47b6 Merge pull request #64 from deepseek-ai/worktree-e2e-real-api-workflow
ci: add real-API e2e workflow against external DeepSeek API
2026-06-19 18:53:32 +08:00
Tianyi Cui 27721f9f45 docs(rfc): record real-API e2e CI decision + security model
Adds docs/rfc/implemented/2026-06-19-real-api-e2e-ci.md covering the rationale
for running the real-API e2e suite in a separate secret-consuming workflow, the
fork/Dependabot/secret threat model, the residual exposure of the pull_request
trigger, and what changes when the repo goes public. Indexes it in the RFC
README.

Also adds a SECURITY comment on the pull_request trigger forbidding a switch to
pull_request_target (an untrusted-code-with-secrets leak vector, especially once
public), pointing at the RFC.
2026-06-19 18:42:27 +08:00
Tianyi Cui 5bccda19d3 docs(rfc): keep implemented RFCs current; rewrite the llm-replay section in place
- Rewrite the snapshot-test RFC's replay-plugin section to state current
  reality directly (the plugin is the @deepseek-ai/dsh-llm-replay package,
  under the coverage gate) instead of keeping the old example-local text with a
  "superseded" note bolted on.
- Add docs/rfc/implemented/AGENTS.md (+ CLAUDE.md symlink): an implemented RFC
  must be kept current with what actually shipped — update paths/names/structure
  in the same change that moves the code, in place, not as an append-only
  changelog of its own drift. A reversal of the DECISION is still a new RFC.
- Reconcile docs/rfc/README.md: the "never edited into a different decision"
  rule now distinguishes tracking where a decision lives (required) from
  flipping the decision (forbidden), and the implemented/ bullet points at the
  new convention.
2026-06-19 17:06:20 +08:00
Tianyi Cui 7731bc80c3 Merge remote-tracking branch 'origin/master' into feat/acp-snapshot-tests 2026-06-19 10:03:04 +08:00
Tianyi Cui bef9386591 docs(rfc): add ACP snapshot tests RFC (record-once / replay-deterministic)
Records the decision to add a third test tier: snapshot tests that boot the
real acp-agent subprocess over ACP stdio, record the LLM's streamed responses
once against the real API, then replay them deterministically so the full
stdout transcript can be diffed against a committed golden — keyless in CI.

Captures the design choices hardened in a Codex (xhigh) review: record at the
provider-neutral llm/stream waterfall; a discriminated fixture entry schema
(chunks/throw/hang) that honors both LLM failure branches; positional replay
with a one-in-flight-stream constraint; per-stream atomic fixture flush (the
subprocess is SIGKILLed, so dispose-time flush would never run); a providerless
replay config; normalize-then-snapshot parsed frames; normalization over an OS
sandbox now with the rootless bwrap/sandbox-exec tier reserved via the
BashExecutor capability seam. Cross-links the proposed determinism RFC
(complementary: internal history invariant vs external protocol contract).
2026-06-19 00:19:54 +08:00
Tianyi Cui 7fa113be0e Merge remote-tracking branch 'origin/master' into codex/pr48-repo-hardening-rfcs
# Conflicts:
#	docs/adr/README.md
#	docs/rfc/009-session-persistence-and-resumability.md
#	docs/rfc/README.md
#	docs/rfc/implemented/2026-06-11-doc-sync-enforcement.md
#	docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md
#	examples/acp-agent/tests/acp.e2e.ts
#	packages/acp/README.md
#	packages/acp/src/index.ts
#	packages/acp/tests/stream-update.spec.ts
#	packages/agent-loop/src/loop.ts
#	packages/tools/src/index.ts
2026-06-18 23:41:14 +08:00
Tianyi Cui 386ee14af3 docs(acp): RFC for the terminal-card rendering (implemented design)
Records the verified design before implementing: keep dsh-bash agent-side
execution and render Zed's terminal tool-call card via the `_meta` convention
(terminal_info/terminal_output/terminal_exit), capability-gated on
clientCapabilities._meta.terminal_output, with the ```console text block as the
no-capability fallback. Rejects the spec's client-side terminal/create path (it
would bypass dsh-bash's sandbox/env-scrub/ownership/cwd). Studied
claude-agent-acp, codex-acp, and Zed's renderer to ground the wire contract.
Live streaming and command classification are noted as separate follow-ups.
2026-06-18 14:36:02 +08:00
Tianyi Cui 8acafe918f feat(acp): show the command in execute titles; test via the real bash tool; RFC for terminal rendering
- bash presentCall title is now "description — command" (e.g. "List files in
  src — ls -la src"). An execute-kind ACP card HIDES rawInput (Zed renders it
  only for non-terminal tools), so the command must ride in the always-visible
  title to be seen — matching how claude-agent-acp/codex-acp title execute
  tools. The command stays in rawInput too for non-execute UIs that show it.
- Rework the acp tool-call presentation tests (turns + load replay) to drive the
  REAL dsh-tool-bash + dsh-bash-local via a new makeBridgeHarness({ withBash })
  option, running an actual `echo` — instead of an inline fake bash tool. The
  mock MODEL still scripts the call (deterministic, no key), but the tool and
  executor are real, so the test verifies the shipping presentCall/presentResult.
- AGENTS.md: add the principle "prefer the REAL implementation over a mock/
  stand-in in tests" (mock only the expensive/non-deterministic boundary).
- RFC (proposed): the ACP terminal sub-protocol + command classification — the
  capability-gated rich rendering (live cwd-header terminal card, classify a
  `cat` as a read / `grep` as a search) that the reference adapters do; the
  fenced ```console text block stays the no-capability baseline. Studied
  codex-acp, claude-agent-acp, and Zed's renderer to ground it.
2026-06-18 11:23:12 +08:00
Tianyi Cui 7c400e9c02 docs: unify ADR/RFC trees into one lifecycle-organized RFC tree
Collapse docs/adr/ and docs/rfc/ into a single docs/rfc/ with proposed/,
implemented/, and rejected/ subfolders. Every file is renamed to
yyyy-mm-dd-topic-title.md, where the date is when the topic was first
proposed (from git history). ADRs and RFCs that covered exactly the same
topic are merged (property-based testing, session persistence); the
umbrella RFC 005 stays split across its three implemented decisions, and
RFC 006's deferred part-3 (API extractor reports) splits into its own
proposed RFC. All cross-references become machine-checkable relative
links instead of bare "ADR NNNN" / "RFC NNN" prose.

Add a verify-md-links doc-sync gate (scripts/verify-md-links.ts) that
checks every relative Markdown cross-link resolves, wired into doc-sync
alongside verify-md-wrap. This makes the reorganization self-verifying:
the same change that rewrote ~forty inter-doc links adds the check that
proves none dangle. Document the cross-link convention in a new
docs/AGENTS.md and record the gate as an implemented RFC.

doc-sync, typecheck, lint, and the full test suite (667) all pass.
2026-06-18 02:18:24 +08:00
Tianyi Cui 7a5886da2d docs(rfc): record follow-up cleanup seams 2026-06-17 21:26:57 +08:00
Tianyi Cui efee449cfe feat(session-persistence): preserve interrupted turns on crash; don't truncate (review #33)
A crash can leave a durable log whose final turn never closed. The old
behavior truncated everything after the last turn/end as a "crash tail".
But a single turn can be HUGE in a long-horizon task (many steps, large
tool output), so truncating it silently destroys real, durably-written
work — truncating a turn is wrong.

New crash recovery (ADR 0018): load() PRESERVES the interrupted turn's
events and CLOSES the orphaned turn by durably appending synthetic
boundary events — a step/end if a step was open, then a turn/end carrying
the new merge-extensible TurnEndReason {kind:'interrupted'}. load()
returns the balanced log, so a resumed session is immediately usable. Only
a never-fully-written TORN tail fragment is discarded; corruption in the
committed region is still unloadable.

- dsh-session: TurnEndReason {kind:'interrupted'} + shared
  interruptedTurnClosers() repair helper.
- JSONL backend: scanLog preserves the longest contiguous prefix
  (including a partial final turn); loadCore truncates a torn fragment and
  durably writes the closers, returning the balanced log.
- runPersistenceContract gains a crash-recovery test (both backends + mock).
- Docs: ADR 0018/0017, architecture.md, package READMEs.

Also (review #33): RFC 013 records the "move event vocabulary to Zod"
question (merge-extensible maps → runtime schema registry) + blast radius;
deferred, not done here.
2026-06-16 21:27:50 +08:00
Tianyi Cui 1cc6e1caf7 docs: add RFC 012 (optional Code Mode for all tools)
Proposes an optional Code Mode where the model writes a TypeScript program
against a generated SDK wrapping every registered tool, instead of emitting
one native tool-call per step. Implemented Cordis-style as a capability-seam
trio (code-runtime interface / code-runtime-vm node:vm reference stub /
code-mode consumer plugin) with zero core-package changes; the hardened
execution substrate is deferred to a follow-up RFC.
2026-06-15 01:20:09 +08:00
Tianyi Cui a44f7f3486 docs: add RFCs 009-011 (session persistence + ACP support)
Three proposal documents, numbered in dependency order:

- RFC 009: an abstract, append-only, event-based SessionPersistence
  service over the existing SessionEvent log (no parallel persisted
  type), a JSONL impl, a SessionMeta header seam, and an async
  AgentLoop.resume path. Design informed by Codex/Claude Code/
  opencode/pi. Core design point; unblocks resume + ACP session/load.
- RFC 010: ACP (Agent Client Protocol) support as a dsh-acp
  client-driver plugin on @agentclientprotocol/sdk, mapping ACP onto
  the agent/* events and the tools/execute permission seam. Builds on
  009 for session/load; single active session.
- RFC 011: multiplex concurrent ACP sessions over one connection
  (bridge-layer change; downstream of 010).
2026-06-14 21:14:51 +08:00
Tianyi Cui 825b57aff9 feat(llm): structured error taxonomy with a shared HarnessError base (RFC 005 pt 2)
Introduce HarnessError in dsh-llm (the leaf package): a stable machine-routable
code distinct from the message, cause chaining, name from the subclass, plus
isHarnessError. LlmError, ToolArgsError, and InvariantError now extend it.

Tool failures carry the structure end-to-end: ToolExecutionResult gains
error: { name, code } (populated from a thrown HarnessError), and the loop
forwards it onto the tool/result session event (which gained the same optional
field) for retry/sandbox plugins and replay. The loop's toError wraps non-Error
throws in a HarnessError(code: UNKNOWN, cause) instead of a bare Error.

Landed last and in isolation so it's a pure upgrade over the plain Error+code
the earlier PRs used — independently revertible. Graduates RFC 005 pt 2 ->
ADR 0015; RFC 005 now fully implemented.
2026-06-14 01:07:28 +08:00
Tianyi Cui 6a528be569 build: doc-sync gates — typecheck doc code blocks + verify event taxonomy (RFC 006 pts 1-2)
Two tsx CI gates make doc/code drift fail fast:
- doc-typecheck extracts every fenced ts block from README/docs/package READMEs,
  compiles them with tsc --noEmit against a temp project (vendor->lib, harness->src
  paths from tsconfig.typecheck.json), and fails on errors. Deliberate sketches opt
  out with ```ts ignore-check; the opt-out ratio is reported and capped.
- verify-event-taxonomy asserts the docs/architecture.md taxonomy table names
  exactly the events declared in the interface Events blocks. This surfaced three
  events the table had been missing (tools/change, llm/adapter-change,
  system-prompt/change), now added.

Doc snippets made compilable with stub imports/declares (1 genuine sketch ignored).
Wired into CI after typecheck. API reports (RFC 006 pt 3) deferred. Graduates RFC
006 pts 1-2 -> ADR 0014.
2026-06-14 00:47:38 +08:00
Tianyi Cui 2f6d3b8539 test: property-based tests for protocol-shaped code (RFC 001)
Adds fast-check + one tests/properties.spec.ts per protocol-shaped package
(llm/BlockAssembler, session, tools/schema DSL, agent-loop scheduling). The
tools suite includes the RFC 001<->005 composition property (generated args
satisfying a spec pass validateArgs), closing the validator/InferArgs drift
risk from ADR 0011. Loop properties are deterministic (settle on agent/status,
no sleeps).

The BlockAssembler suite found a real bug on first run: a duplicate block-end
at the same index overwrote an already-flushed block, so the streamed prefix
disagreed with final blocks(). Fixed (first close wins, matching the existing
straggler rule) + regression test. Graduates RFC 001 -> ADR 0013.
2026-06-14 00:06:25 +08:00
Tianyi Cui 11a29fdefe feat(invariants): dev-mode event-contract assertions + session-log freeze (RFC 005 pt 3, RFC 008)
New @deepseek-ai/dsh-invariants plugin (pure listeners, off in prod) asserts
the event taxonomy at runtime — seq monotonicity, turn/step nesting, a
tool/result needs a prior tool/call (NOT the converse), legal agent/status
transitions — and deep-freezes logged event data so mutating history throws.
Seeded sessions are checked + frozen on session/created.

The real RFC 008 fix is always-on: deriveMessages now structured-clones the
content it emits, so the loop's sanctioned request/adapter mutation can no
longer reach back and rewrite the append-only log. The pervasive
DeepReadonly<T> type flip is rejected (compile-only, high-noise, castable) —
recorded in ADR 0012, which folds in RFC 008. Wired into both demos.
2026-06-13 23:25:12 +08:00
Tianyi Cui 36a30180b8 feat(tools): validate model-generated tool args at the boundary (RFC 005 pt 1)
defineTool now runs validateArgs against the SchemaSpec before execute, so a
malformed model call returns a self-correctable isError result listing the
violations instead of reaching the typed body untyped-in-practice. The
validator mirrors schemaSpecToJsonSchema semantics exactly (required from
required:true only, extra keys allowed, default not applied, object/array
without properties/items only type-checks, enum membership).

tool-bash's hand-rolled type/required checks (carrying the TODO(RFC 005)
stopgap note) are slimmed to just the value constraints the DSL can't express
(non-empty strings, positive timeout). Graduates RFC 005 pt 1 to ADR 0011.
2026-06-13 23:00:42 +08:00
Tianyi Cui 066f94c7e0 docs: unwrap hard-wrapped Markdown to one line per paragraph
Hard line breaks mid-paragraph make docs harder to edit and diff — a
one-word change reflows and re-diffs the whole paragraph. Reflow all
tracked non-vendor Markdown (plus vendor/AGENTS.md) so each prose
paragraph is a single line; soft-wrapping is the editor's job. Fenced
code, tables, and list structure are preserved (wrapped list items fold
to one line per bullet). Documents the convention in AGENTS.md.
2026-06-13 20:27:04 +08:00
Tianyi Cui 4dafad4db6 Add RFCs for the remaining quality-proposal ideas
Eight proposals grouped by category, each with problem statement,
concrete plan, and risks: property-based testing over the
protocol-shaped core (chunk streams, event logs, schema DSL);
mutation testing as the counterweight to the 100%-coverage gate;
deterministic tests + a universal replay-invariant fixture + nightly
race stress; architectural conformance (dependency-cruiser rules and
the LlmAdapter conformance kit); runtime arg validation at the model
boundary with a structured error taxonomy and dev-mode invariants;
doc-sync enforcement (typechecked doc snippets, API reports);
supply-chain checks and nightly vendor-drift verification against the
manifest; and deep-readonly public surfaces (logged-vs-in-flight
mutability boundary). AGENTS.md points at docs/adr and docs/rfc.
2026-06-11 15:27:07 +08:00