Tianyi Cui
c94f1563f5
Adds the first cut of snapshot scenarios, each asserting a normalized stdout
transcript golden and (for model turns) a re-persisted session-log golden:
- text-turn, tool-call-turn, multi-turn: RECORDED against the real API — the
committed session.jsonl is a genuine harvested log; replay derives the model
script from it and reproduces deterministically with no key. tool-call-turn
exercises the real bash executor (echo SNAPSHOT_OK → tool/call + tool/result
+ a post-tool answer step).
- error-finish, cancel: AUTHORED via a replay.override.json sidecar (the live
API can't be coaxed into a deterministic 401 or mid-stream cancel). error-
finish replays a {kind:throw} 401 → the bridge answers the prompt with a
JSON-RPC error and the log records turn/end{kind:error}; cancel replays a
{kind:hang} → stopReason:cancelled.
Two input-DSL ops support these: promptExpectError (awaits the prompt, asserts
it rejects — the editor's view of a failed turn — and swallows it) and
promptAndCancel (dispatches the prompt unawaited, waits until the client
OBSERVES the streamed agent_message_chunk, then cancels — pinning frame order
so the cancel transcript is deterministic; fixes a flake Codex caught where the
late chunk and the cancelled response could interleave either way).
Scenarios carry a `recorded` flag so test:snapshot:record only re-runs the
live-API ones. reasoning/max-tokens scenarios are deferred (hard to force
deterministically from the live model). Per docs/rfc/implemented/2026-06-19.
acp-agent example
The DeepSeek Harness coding agent exposed as an Agent Client Protocol (ACP) server over JSON-RPC stdio — drive it from Zed or any other ACP client.
pnpm run demo:acp # needs DEEPSEEK_API_KEY (repo-root .env or env)
This boots @deepseek-ai/dsh-acp over the shared provider/tool core (../base.yml), with agent-loop configured with no pre-created agents (ACP session/new creates them on demand) and JSONL session persistence (so session/load works).
stdout is the protocol
This example loads no stdout logger — stdout carries the JSON-RPC frames, and any other write corrupts them. Do not add @cordisjs/plugin-logger-console or a stdio UI here. Use a stderr exporter if you need logs.
Zed configuration
Add to your Zed settings.json under agent_servers:
{
"agent_servers": {
"DeepSeek Harness": {
"command": "pnpm",
"args": ["run", "demo:acp"],
"env": { "DEEPSEEK_API_KEY": "sk-…" }
}
}
}
The editor sets each session's cwd to the project it opens; the agent's bash tools run there (see the per-session cwd note in packages/acp), so the server does not need to be launched in the workspace.
Snapshot tests (record-once / replay-deterministic)
This example is the home of the harness's snapshot tests — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by src/llm-replay.ts, a function/namespace plugin that installs an llm/stream waterfall listener and short-circuits it, serving model streams reconstructed from a recorded session JSONL fixture (<scenario>/session.jsonl) — so replay needs no API key. The fixture IS the persisted session log: its assistant/chunk events carry every StreamChunk, so grouping them by (turn, step) reconstructs each stream() call (one model call per loop step). Recording is therefore "run the real agent once and harvest the .jsonl". The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional <scenario>/replay.override.json sidecar (a ReplayEntry[] that replaces the derived script). See docs/rfc/implemented/2026-06-19-acp-snapshot-tests.md for the full design.
MVP limitations
The bridge supports N concurrent sessions per connection, each in its own workspace cwd (RFC 011). Remaining limits: text-only prompts, additionalDirectories rejected (a session operates in its single cwd), and the tool-permission gate is deferred (TODO(rfc010-permission-gate) — tools run with the executor's full authority). See packages/acp/README.md for the full contract.