Commit Graph
46 Commits
Author SHA1 Message Date
imccyu bd831db80b single-exe: closure manifest, build pipeline, and CI workflow 2026-07-13 15:50:09 +08:00
imccyu 67fe6c10b6 jsonrpc: SDK serving surface as plugins (dsh-jsonrpc + dsh-jsonrpc-agent) 2026-07-13 15:50:09 +08:00
Tianyi Cui 1e2d8014a5 Merge remote-tracking branch 'origin/master' into feat/sandbox-support
# Conflicts:
#	packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts
2026-07-11 21:43:10 +08:00
kingwl ca39fd89b0 feat(example): sandbox-acp-agent — the live composition; RFCs to implemented
The three-entry cordis.yml (dsh-sandbox-local + dsh-bash-sandbox at a
read-only default + dsh-approval) served over ACP: the first live approval
composition. Recorded snapshot scenarios pin the wire end to end —
config-options advertisement, the mode-switching arc as the suite pinned
header (both switches, the prompt-section delta, one changed-by-the-user
notice per knob, a confined write landing under the switched mode), and
both escalation branches over scripted permissionAnswers (a grant runs
confined under workspace-write; a rejection executes nothing and pins the
fail-closed text). The with-key escalation e2e drives a real model +
real runner + the real bridge answerer, world-verified; ci.yml snapshot
lane and e2e.yml install bubblewrap so the confined replays actually
execute.

Both RFCs move to implemented/ (Decision/Consequences form, deferred
phases tracked in their own sections), with every cross-reference flipped.
2026-07-10 15:44:38 +08:00
kingwl 2eed448acf feat(bash): the sandboxed executor — per-call policy carrier, denial facts, runner-failure classification
dsh-bash grows the per-call policy carrier: BashExecRequest.sandboxMode
(request-optional, spec required-but-nullable — the owner pattern; resolve()
is the one explicit defaulting step) and the BashExecutor.sandboxMode
capability fact (undefined in the base class — composition truth the tool
layer can read). dsh-bash-local carries the field verbatim and confines
nothing.

dsh-bash-sandbox extends LocalBashExecutor and hands ctx.sandbox the exact
argv it is about to spawn. A denial is a RESULT FACT (the command RAN;
result.sandbox.denied is orthogonal to exitCode/signal), classified
conservatively against the wrap own dialect; a RUNNER failure outranks
denial — foreground re-throws the structured SANDBOX_UNAVAILABLE, a settled
background task stamps sandbox.runnerFailed — so a broken sandbox never
reads as a failing command and the command never runs unconfined.
dsh-tool-bash renders the markers and teaches the model not to retry around
a policy denial; escalation and per-session switching are staged follow-ups.
2026-07-10 15:43:02 +08:00
kingwl 7b8c3a9b40 feat(sandbox): the confinement seam and the per-platform native runner chains
ctx.sandbox (dsh-sandbox): confine(argv, policy) returns the argv to spawn
instead — wrapped so the process and its children run confined — plus the
enforcement completeness and the backend denial/runner-failure dialects;
no usable backend throws the fail-closed SANDBOX_UNAVAILABLE. Policy rides
per call. dsh-sandbox-local selects by platform and caches the verdict:
multi-candidate chains probe FUNCTIONALLY in preference order (Linux:
bwrap → the registry-installed node-addon-landlock-run launcher), a sole
candidate is selected unprobed (darwin: sandbox-exec/Seatbelt) and fails
closed at execution via runnerFailureSignatures; win32 is a reserved empty
chain. Profile parity is honest per backend (documented temp-area and ABI
differences; enforcement full|partial is a structured result fact).

CI: the sandbox-e2e matrix proves real-kernel confinement per rung (bwrap,
Landlock per architecture through the registry-installed launcher,
Seatbelt), failing on a silent all-skip; the packed-install rehearsal
installs the launcher family from the registry and asserts the binary
executable apart from kernel enforcement.
2026-07-10 15:43:02 +08:00
pku-xht 3eea868a1b Merge branch 'master' into feat/subagent-process 2026-07-10 10:05:01 +08:00
imccyu 2c812fb84a Merge remote-tracking branch 'origin/master' into workflow-vm-to-workerthread 2026-07-09 22:36:52 +08:00
imccyu 7f5b990aa1 Merge remote-tracking branch 'origin/master' into workflow-vm-to-workerthread 2026-07-09 21:34:52 +08:00
imccyu 0d11a3afc7 workflow: rename dsh-workflow-vm to dsh-workflow-workerthread
Pure mechanical rename now that the package's internals are the
worker-thread engine: directory, package name, spec/e2e filenames,
module tags and logger prefixes, tsconfig/knip/run-gates/AGENTS.md
references, example cordis.yml plugin ids, doc links; catalogs
regenerated and the lockfile refreshed.
2026-07-09 19:33:08 +08:00
pku-xht 607b02a659 Merge branch 'master' into feat/subagent-process 2026-07-09 14:01:28 +08:00
imccyu e51e58e993 chore: register the cordis group across repo gates and docs
Everything outside the package and example that a new top-level group and a
new demo touch: GROUP_ORDER in gen-module-graph and gen-doc-graphs (plus the
tools-service consumers list, the APP_EXAMPLES entry, and the graph-atlas
label/mode rows), the knip e2e entries, the packages/README group row, the
AGENTS.md layout and demo lines, and the regenerated module-graph /
config-catalog / graph-atlas / capability-seams / composition artifacts.
AGENTS.md and examples/AGENTS.md word-budget ceilings rise to current+5%
(1802 / 653): the new group and demo rows are genuine additions to both docs,
not condensable restatements.
2026-07-09 13:57:03 +08:00
pku-xht c321819053 rename: @deepseek-ai/dsh-subagent-process -> @deepseek-ai/dsh-subagent-subprocess
The extracted library's name sat one edit away from @deepseek-ai/dsh-subagent-inprocess
(process/inprocess), inviting a typo'd import to silently resolve to the wrong
package. subagent-subprocess also reads as the deliberate counterpart to
subagent-inprocess (in-process vs. subprocess), matching how the two shared
drivers actually differ.

Package directory, npm name, module doc, JSDoc module tags, test-file name and
its temp-dir prefixes, the subagent-acp import and its Config/tsconfig/package.json
references, root tsconfig.json/tsconfig.build.json/knip.json entries, and the
packages/subagent group README all renamed together; regenerated
docs/module-graph.md and docs/config-catalog.md. Pure rename — no behavior,
export, or Config shape changed.
2026-07-09 13:42:03 +08:00
Dudu-0223 0a3ffe1b04 Merge remote-tracking branch 'origin/master' into timeout-design
# Conflicts:
#	docs/event-producer-consumer.md
#	packages/README.md
2026-07-09 11:52:18 +08:00
pku-xht 6660d2a83e Merge branch 'master' into feat/subagent-process 2026-07-09 11:03:48 +08:00
pku-xht cdd1ce2ad4 feat(subagent): extract dsh-subagent-process shared out-of-process machinery
The credential env scrub (SENSITIVE_ENV_PATTERN/buildChildEnv), the
spawn-failure capture (spawnFailure), the child-exit waits
(waitForExit/exitsWithin), and the stdin-EOF -> SIGTERM -> SIGKILL dispose
ladder move out of subagent-acp into a new pure library package (the
subagent-inprocess shape), with the ladder taking its two grace periods as
parameters — defaults stay in the plugin Config. New isolated-config-dir
helpers (mkdtemp create, best-effort remove; a pinned dir is never removed)
land alongside for the CLAUDE_CONFIG_DIR / CODEX_HOME redirection the RFC
names.

The ACP backend migrates onto the library with no semantic change: its suite
passes with import-path edits only. bash-local keeps its sibling copy, per
the RFC's blast-radius call.

RFC: docs/rfc/proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md
2026-07-08 17:09:53 +08:00
Tianyi Cui 030eebb634 Merge remote-tracking branch 'origin/master' into code-runtime-worker 2026-07-08 14:44:23 +08:00
Dudu-0223 a76285c4e6 Merge origin/master into timeout-design
Resolve conflicts from master's catalog/doc refactors landing alongside the
tool-call timeout work:
- knip.json: keep both new workspace entries (util/timeout + support/acp-snapshot).
- tool-web/src/fetch.ts: keep the timeout_ms removal, adopt master's richer
  JSDoc @param/@returns style on parseFetchArgs/presentFetchCall.
- tools/README.md: keep the tools/execute pipeline wording, adopt master's
  flattened docs/tool-catalog.md path.
- Regenerate every generated doc (cordis-catalog, tool-catalog, config-catalog,
  doc-graphs, module-graph) so they carry both master's changes and the
  tools/execute event + timeout-policy package.
- Add @param/@returns to toolTimeoutResult for master's new verify-export-jsdoc gate.
2026-07-08 11:18:27 +08:00
Tianyi Cui 583704ac1d feat: add the worker-thread code runtime (dsh-code-runtime-worker)
The shipped backend of the code-execution seam, per the Code Mode RFC's
worker-thread section: one fresh Node worker per run, executing the
model's TypeScript after a host-side type-strip (wrapped in an
async-function shell so top-level return/await parse, sliced back out
position-preserved), bindings bridged over the message port under
hostile-peer rules (own-property name lookup, at-most-once replies,
post-settlement drops, null-prototype namespaces), logs streamed eagerly
with an in-band truncation marker, and two independent budgets — measured
event-loop busy time (computeMs) plus a never-pausing wall ceiling
(maxWallMs) — funneling into worker.terminate(). env: {} and execArgv: []
keep the isolate hermetic; disposal aborts in-flight runs and awaits
worker exits.

The worker entry loads unbuilt via Node's native type stripping
(src/worker.ts, erasable-only) and ships built as a sibling tsdown bundle
(lib/worker.js); tests/built-lib.e2e.ts pins the built load path under
plain node and joins the built-artifact smoke gate. Unit suites cover the
bootstrap in-process (fake port) and the runtime over real workers,
per-file 100%.
2026-07-08 11:07:14 +08:00
kingwl 610c8e3709 test(acp-snapshot): fake ACP bin + unit specs to per-file 100% coverage
A scripted fake ACP agent bin (tests/fixtures/fake-acp-agent.ts) speaks
real newline JSON-RPC through the REAL runScenario spawn path (tsx
loader, temp cwd, env plumbing); every behavior — prompt outcome,
session/new rejection, persisted logs, filesystem noise — comes from a
behavior.json beside the fixture, so specs script whole subprocess runs
from data. harness.spec.ts drives every step op, both expect-error arms,
the permission-stub default, env forwarding, workspace seeding, and the
harvest ordering/noise/fallback branches. suite.spec.ts runs the factory
for real at collection time: a replay suite over committed synthetic
fixtures and a record suite over a temp copy (write-back never touches
the committed tree; ACP_SNAPSHOT_SPEC_BOOTSTRAP=1 re-bootstraps it),
plus direct cases for the exported pure helpers. The suite factory's
pure helpers (childFixturePaths, fixtureContext, normalizedHeaders,
headerDeltaCount) are exported for those direct specs.

Two branches carry justified v8 ignores, both structurally unreachable:
the waiter in-bounds guard (noUncheckedIndexedAccess) and waitForExit's
already-exited race guard (both call sites sit one synchronous frame
after stdin.end()/kill()). The fake bin substitutes the session/new cwd,
not process.cwd(), into scripted logs — the realpath difference
(/private on darwin) is exactly what the real bin's header carries.

packages/support/acp-snapshot/src is at 100% statements, branches,
functions, and lines under the per-file gate.
2026-07-08 02:38:25 +08:00
kingwl 556f847064 feat(acp-snapshot): extract the ACP snapshot suite into a support package
The snapshot tier's machinery leaves examples/acp-agent/tests for
packages/support/acp-snapshot (@deepseek-ai/dsh-acp-snapshot), where the
coverage gate measures it and a second example can consume it instead of
forking it: harness.ts (runScenario, parameterized by an AgentUnderTest
{binScript, configPath, tsconfigPath} instead of module constants),
normalize.ts (moved verbatim), and suite.ts (defineAcpSnapshotSuite — the
per-scenario golden/log compares, record write-back, per-suite header pin
with its uniformity guard, and the fixture guard block, lifted from
acp.snapshot.ts). The example file collapses to its scenario table plus
one factory call; env reading (DSH_SNAPSHOT) stays at that edge.

The exactly-one-pin meta-test generalizes from the hardcoded text-turn
name to "exactly one per suite" — which scenario pins is the scenario
table's reviewable choice (per-suite pinning per the proposal RFC).

Extraction parity: pnpm run test:snapshot is 36 passed + fs-policy-reject
failing BEFORE AND AFTER (BSD-sed environment failure, reproduced at the
base commit in a clean worktree — the recorded golden's sed -i syntax is
GNU-only), with zero byte changes under examples/acp-agent/tests/snapshots/.
Coverage for the new src files lands in the next commit.
2026-07-08 02:38:25 +08:00
Tianyi Cui ea58bdf1e8 Merge remote-tracking branch 'origin/master' into worktree-dynamic-workflows
Master's reconstructable-requests overhaul (#179) meets the workflow tool:
- subagent-inprocess structured-output nudge becomes a system-prompt section
  plus logged context (the injected-request waterfall shape is gone upstream)
- snapshot fixtures re-recorded on the merged tree so every request/header
  carries the workflow tool; authored error-finish/cancel headers patched to
  the merged tool list and system text
- architecture.md condensed back under its word ceiling; module graph regenerated
2026-07-06 22:28:57 +08:00
Dudu-0223 208123f252 Merge remote-tracking branch 'origin/master' into timeout-design
# Conflicts:
#	knip.json
2026-07-06 17:57:38 +08:00
Dudu-0223 8615e019d3 feat(timeout): add dsh-timeout and converge bash + web_fetch onto it
Timeout timing/classification was re-implemented three ways across the
tool-bearing capabilities, with the fusion of timeout+cancel and the
timeout-vs-cancel reason recovery being the error-prone parts. Extract that
shared half into a zero-dependency @deepseek-ai/dsh-timeout library
(clampTimeout/deadline/timeoutOf/TimeoutReason) and leave the non-shareable
hard-kill in each capability, per the timeout-library RFC.

bash: run() owns the deadline; runBash drops its killTimer and no longer
classifies (SpawnSpec/SpawnOutcome lose timeoutMs/timedOut/aborted), so the
public timedOut/aborted booleans become mutually-exclusive first-abort
classifications. web_fetch: the hand-rolled controller/timer/listener/
signal.reason dance is replaced by provider-owned deadline/timeoutOf, keeping
the WEB_FETCH_TIMEOUT / WEB_ABORTED contract. fs stays timeout-free (README
states why).
2026-07-06 16:25:04 +08:00
Tianyi Cui 23ed47ba75 chore: knip learns agent-loop's e2e entry (ported from #162 with request-cache.e2e.ts) 2026-07-06 04:01:56 +08:00
Tianyi Cui 8cb5e15544 chore: undo the knip.json reformat noise
Review nit: the feature commit exploded every single-line array in
knip.json to multi-line, burying the one semantic change (the
workflow-vm workspace entry) under ~110 lines of mechanical reformat.
Restore the file to master's formatting with only that entry added —
the diff against master is now the 4 lines that mean something.
2026-07-06 00:53:17 +08:00
Tianyi Cui 1d43ea3cd5 workflow: dynamic workflows — script-driven multi-agent orchestration
A new capability family at packages/workflow/ in the bash seam shape,
modeled on Claude Code's dynamic workflows: the model writes a JavaScript
orchestration script (export const meta = {...} + plain-JS body), a runtime
executes it, and the script — not the conversation — holds the loop, the
branching, and the intermediate results.

- dsh-workflow (ctx.workflows): abstract WorkflowService + run vocabulary
  (WorkflowRun whose result NEVER rejects) + observe-only workflow/* events
  carrying data snapshots (id + meta, never the live run), per-listener
  contained like subagent/*.
- dsh-workflow-vm: in-process node:vm engine. Meta extraction via a
  string/comment-aware scanner (template interpolation rejected; literal
  evaluated alone in an empty timed context; statement blanked line-
  preservingly so stacks keep script line numbers). Hooks: agent(prompt,
  {label, phase, schema, model}) over ctx.subagents, parallel(), pipeline()
  (no cross-stage barrier), phase(), log(), args. Fatal-vs-null discipline:
  hook misuse (unknown/deferred options, bad arguments, unsupported
  schemas, tripped caps, seam start failures, cancellation) throws fatal
  WorkflowErrors the combinators RE-THROW — never dissolved into the
  per-item null reserved for child failures. Realm boundary: inbound values
  materialized by descriptor walks that never invoke accessors (defineProperty
  copies, __proto__-safe); outbound values rebuilt in-realm via the
  context's own JSON.parse. Determinism bans (Date.now/Math.random/argless
  new Date) kept so future resume support cannot break scripts. Caps and
  timeouts are validated Config. Every hook promise carries a no-op
  rejection consumer (app-boot exits on unhandled rejections).
- dsh-tool-workflow: the model-facing workflow tool, synchronous like
  dsh-tool-subagent (start → await → try/finally dispose; abort bridged;
  non-completed → isError). Generic render card titled by a textual
  meta.name sniff. The tool description carries the authoring contract.

Wired into examples/{coding-agent,acp-agent} with explicit-ask-only
guidance. Coverage at every tier: unit (meta scanner, materializer incl.
counting-getter and __proto__ regressions, combinator semantics,
concurrency ceiling, caps, cancellation, no-unhandled-rejection abandon),
integration over the real spawn stack, with-key e2e (real two-phase run +
the tool through the registry pipeline), and a recorded ACP snapshot
scenario (workflow-run, 1 child session). RFC:
docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md (deferred
work explicitly listed). AGENTS.md budget 1575 → 1590 for the new group's
layout line.
2026-07-05 13:29:35 +08:00
Dudu-0223 005db3c610 Merge remote-tracking branch 'origin/master' into web-capability-seam
# Conflicts:
#	docs/architecture.md
#	docs/module-graph.md
#	docs/rfc/README.md
#	packages/README.md
#	tsconfig.build.json
#	tsconfig.json
2026-07-03 16:49:00 +08:00
Tianyi Cui 3aa6b3c77a chore(knip): register tool-fs e2e tests as knip entry points
The new packages/fs/tool-fs/tests/*.e2e.ts (+ its harness.ts) need an explicit
knip workspace entry — mirroring the other e2e-bearing packages — so knip
follows them and does not flag the files or their dsh-agent-loop/dsh-llm-deepseek
devDeps as unused.
2026-07-02 20:47:20 +08:00
Dudu-0223 b92a3c531a feat(web): add DeepSeek-backed web search provider
Add @deepseek-ai/dsh-web-search-deepseek: a WebSearchProvider that calls
DeepSeek's Anthropic-compatible Messages API with the native
web_search_20250305 server tool and parses the structured
web_search_tool_result blocks into the ctx.web seam's WebSearchResult.

- Namespace plugin (inject: ['web']), no default export — registers into
  ctx.web like dsh-llm-deepseek registers into ctx.llm.
- Strict mode: a response with no web_search_tool_result block throws
  WEB_PROVIDER_ERROR rather than scraping URLs from model prose.
- Reuses $DEEPSEEK_API_KEY; baseURL defaults to the Anthropic-compatible
  base (api.deepseek.com/anthropic/v1) and does NOT reuse
  $DEEPSEEK_BASE_URL, which belongs to the chat-completions LLM adapter.
- snippet joined from text-block citations; sources deduped by url.
- Two-stage build layout (outDir lib/types) matching the other web
  packages; registered in tsconfig.json, tsconfig.build.json, knip.json,
  and docs/module-graph.md.
2026-06-29 15:32:03 +08:00
Dudu-0223 d01f5f73b7 Add web capability seam: ctx.web, search/fetch providers, web tools
Introduce web access as a first-class capability seam so the model-facing
web tools stay stable while backends change. dsh-web owns ctx.web as a
provider registry with registration-order-independent selection and the
WebError taxonomy; dsh-web-search-exa, dsh-web-search-perplexity, and
dsh-web-fetch-local register capabilities into it; dsh-tool-web is the sole
owner of the model-facing web_search/web_fetch schemas, prompt sections, and
HTML-to-markdown presentation. Search and fetch are deliberately one seam.

Providers ship as namespace plugins that register into ctx.web (like an
LlmAdapter into ctx.llm), not key-owning services, since multiple search
providers cannot each own the key. Tool registration follows product
enablement, not backend availability, so load order/credentials never enter
the model contract; the seam resolves the provider at execution time and
surfaces a structured WebError otherwise.

Moves the RFC to implemented/ amended to match what shipped. Example/app
configs are intentionally not wired yet (RFC migration step 6).
2026-06-26 19:12:13 +08:00
Tianyi Cui f393043b03 Add the ACP subagent backend: out-of-process delegation (PR3)
The first OUT-OF-PROCESS subagent backend, proving the seam generalizes past the
in-process backends. @deepseek-ai/dsh-subagent-acp runs each child agent in a
spawned subprocess, driven over the Agent Client Protocol as the CLIENT — the
direction-inverted twin of the dsh-acp server bridge. Point the configured
command at the acp-agent example and the harness talks to its own process.

- Fresh process per run: start spawns, runs one ACP session (initialize →
  newSession → prompt), dispose kills the subprocess and awaits its exit.
- Minimal client stub: advertises no fs/terminal; accumulates agent_message_chunk
  text as the result output; auto-answers session/request_permission by a
  configured policy (reject default / allow). No start-time capabilities (an
  out-of-process child can't enforce the parent's depth/tool-filter); ignores
  request.parent; injects only `subagents`.
- StopReason mapping (end_turn→completed, cancelled→aborted, …); result resolves
  error/aborted on a child failure, never rejects (seam contract).
- Security: credential-shaped ambient env vars are scrubbed; the child's own key
  is forwarded only via explicit config.env. A spawn-level error (ENOENT) is
  captured and raced against the ACP drive so a bad command settles error rather
  than crashing the parent.

Testing designed at every tier: keyless integration drives a scripted mock ACP
server subprocess (cancellation incl. the pre-newSession race and a
torn-pipe-after-cancel, permission auto-answer, non-message updates, spawn
failure, HMR, export shape) at 100% coverage; a with-key e2e drives the REAL
acp-agent example process (PONG + real file write, verified on disk) — the
harness driving itself. Snapshot coverage of an ACP child is deferred as
TODO(acp-subagent-replay) (each child is its own process with its own replay).

Stayed on @agentclientprotocol/sdk 0.25.1: the proposed 0.28.x bump only
deprecates the stable ClientSideConnection/AgentSideConnection API this layer
uses (33 sites incl. the server bridge), turning no-deprecated red across code
this PR shouldn't rewrite — that fluent-API migration is its own follow-up. The
backend needs nothing 0.28.x adds.

This completes the subagent seam stack (PR1 interface → PR2 in-process → PR2.5
snapshot infra → PR3 ACP); the seam RFC moves to implemented/, amended.
2026-06-22 10:47:02 +08:00
Tianyi Cui 7aabd2a3df Add in-process subagent backends: spawn (fresh) and fork (seeded)
The second PR of the subagent seam: the two in-process backends that run a
child agent on the same cordis context, reusing the agent factory's quiescent
AgentHandle teardown. Both register on ctx.subagents (PR1's named-provider
registry) and share one run driver.

- dsh-subagent-spawn: a FRESH child via ctx.agents.create — own session, the
  parent's model by default (overridable), zero inherited conversation. Also
  exports the shared in-process run driver (startInProcessRun): mint ids, stamp
  cwd/parentSession-lineage/depth, drive the one-shot (send → whenIdle), read
  the last assistant/message + turn/end reason, dispose to quiescence.
- dsh-subagent-fork: a child SEEDED with the parent's balanced completed-turn
  prefix (the log up to and including its last turn/end), so the child inherits
  context. The in-flight unbalanced turn is excluded — a raw seed would fail the
  invariants replay. Proven: a regression test goes red if the boundary seeds
  the open turn.
- Seam extension: CreateAgentOptions.seed, threaded through AgentLoop.createAgent
  → ctx.sessions.prepare({ seed }) (the primitive resume already used). This is
  the fork-lineage path the TODO(sub-agents) markers anticipated.
- Depth: a merge-extensible AgentOptions.subagentDepth (0 top-level, parent+1 for
  a child); the depthLimit capability refuses a spawn past request.maxDepth.

Tests: real-loop unit tests for both backends (mock MODEL only, real loop +
invariants), a multi-subagent test (one parent drives a fork AND a spawn child
then keeps working), and a with-key e2e (a real parent delegates via the
`subagent` tool to a real child that writes a file on disk — world-verified).
100% per-file coverage. The coding-agent demo wires the spawn backend + tool.

Snapshot coverage of nested agents is deferred to a stacked follow-up
(TODO(subagent-snapshots)): dsh-llm-replay is a single global positional cursor
that cannot route calls to a parent vs. a child on one context. Recorded in the
RFC's deferrals and a new AGENTS.md rule: designing a subsystem must design its
test infrastructure END TO END up front, verifying the snapshot/e2e harness can
express the new shape — a gap this plan hit.
2026-06-22 05:58:40 +08:00
Tianyi Cui 3567808171 fix review findings: harden the app bins + built-bin smokes, arch-exception doc, snapshot fixture-guard
BLOCKER — the published lib/bin.js (stdio + acp) was exercised only via tsx
(demo:* / the src/bin.ts smokes); the built artifact under plain `node` was
unguarded. Root-cause on the BUILT bin:
  1. Settle race: boot() returned once loader.create() registered the include
     ENTRY, but the include loads its child plugins asynchronously — so boot()
     (and main()) resolved while the app plugins (stdin reader, agent loop, ACP
     bridge) were still mounting. A CLI with no attached handles yet exits 0
     silently, and a load error surfaces as an unhandled rejection AFTER boot.
     Fix: `await ctx.loader.await()` after create() — settle the whole tree.
  2. Config-path robustness: hand the include the config's ABSOLUTE file:// URL
     so resolution never depends on ctx.baseUrl / can never fall back to cwd.
Both bins fixed identically. NOTE: the cordis Loader resolves a config's bare
plugin specifiers via its internal module loader, active only under
`node --expose-internals`; the bin cannot add a node flag itself, so this is
documented in the bin JSDoc + both package READMEs (the demos already comply).
The repo `examples/*/cordis.yml` are tsx-only artifacts (workspace plugins
resolve through the tsconfig paths map, not node_modules), so they are not a
valid plain-node bin target — the smokes use a real-install-shaped temp dir.

Fail loud on a load failure: boot() previously exited 0 SILENTLY when a config
path's directory does not exist — the include plugin fails to IMPORT, the cordis
Loader catches+LOGS it and leaves the entry with no fiber (no rejection), and
`loader.await()` does not rethrow (EntryTree.await uses Promise.allSettled). Fix:
boot() now calls assertEntriesLoaded(ctx) after the tree settles and throws on
any entry with no fiber, so a typo'd config dir exits non-zero with a clear
message. main() also installs an unhandledRejection guard (installFailLoud) that
replaces Node's stack dump with a single labelled stderr line for the
companion case (a missing config FILE in a real dir, whose include-init throw
surfaces as a rejection Node already exits non-zero on). Regression tests added
to both built-bin smokes (missing dir + missing file → non-zero exit + stderr);
verified the missing-dir test fails on the pre-fix bin.

Built-bin smokes (the reviewer's ask): packages/ui/{stdio,acp}-agent/tests/
built-bin.e2e.ts run the REAL lib/bin.js under `node` (NOT tsx) in a temp
consumer dir, asserting the stdio echo round-trip / the acp initialize response
+ stdout purity, plus the fail-loud cases above. They build-gate (skip if lib/
absent) and run in a new ci.yml step after the build.

Issue 2 — packages/README.md + docs/architecture.md said "plugins depend on
interfaces, never on the concrete loop", but dsh-agent-core imports the concrete
dsh-agent-loop. Scope the rule to EXTENSION plugins and carve out the sanctioned
COMPOSITION/bundle exception (dsh-agent-core composes the concrete spine); note
it in the implemented RFC too.

Issue 3 — examples/acp-agent/tests/acp.snapshot.ts fixture-guard claimed
no-model scenarios need no session.jsonl, but runScenario() always boots
llm-replay with the session.jsonl path and loadReplayScript() throws when it is
absent. Require session.jsonl for ALL scenarios (no-model ones ship a
header-only fixture) and rewrite the comment to match reality.
2026-06-21 15:39:24 +08:00
Tianyi Cui e2bde2902c refactor(examples): extract the app spine into dsh-agent-core + app packages
Implements docs/rfc/.../2026-06-20-extract-example-app-packages.md. Each
example was thick — a hand-rolled start.ts, an infra preamble, nested
base.yml/base-core.yml/acp-tail.yml includes, and a coupled front-door
cluster enforced only by prose. This moves the composition into packages so
each example is a thin leaf cordis.yml: pick the swappable backends, load one
app package.

New packages:
  - @deepseek-ai/dsh-agent-core (packages/core/agent-core): one bundle plugin
    that loads the providerless/executor-less/UI-less spine (timer + llm +
    sessions + system-prompt + tools + agents + invariants + tool-bash +
    agent-loop) via ctx.plugin(...) inside apply(), and forwards agent-loop's
    `agents` list as its own Config (export const Config = AgentLoop.Config,
    default []).
  - @deepseek-ai/dsh-stdio-agent (packages/ui/stdio-agent): terminal chat APP —
    agent-core + console logger + readline UI + a pre-created `main` agent, with
    a bin. The demo:echo/coding front door.
  - @deepseek-ai/dsh-acp-agent (packages/ui/acp-agent): ACP server APP —
    agent-core + JSONL persistence + the acp bridge, NO stdout logger, with a
    bin. The stdout-purity footgun is structurally unreachable from the leaf.

Amendment to the RFC: hmr stays a LEAF cordis.yml entry, not baked into
dsh-stdio-agent. hmr is a Loader-only dev plugin (throws without
--expose-internals; the in-process test tier can't even import its decorator
form), so a package statically importing it could never carry the per-file
coverage gate. Unlike the console logger, a stray hmr is not a stdout-purity
footgun, so leaving it at the leaf costs no safety. With hmr out, all three new
packages carry in-process unit specs at 100%.

Boot glue (Loader tail, .env load, snapshot-mode selection, stdin-dispose
lifecycle) moves into each app's bin; start.ts and base.yml/base-core.yml/
acp-tail.yml are deleted. Each app package gets a keyless real-load-path test
that boots through its bin + the cordis Loader (guarding the unwrapExports
export-shape bug class, postmortem 0001). ACP snapshot replay stays green
against the existing committed goldens (pure boot restructuring). RFC moved
proposed->implemented with the amendment recorded; package/example/architecture
docs and the module graph updated.
2026-06-21 12:06:38 +08:00
Tianyi Cui d6a2ab30c8 feat(types): brand bash ids + stop brand erosion; extract Branded to dsh-brand
Type-only change (brands are zero-cost casts; no runtime/wire impact). Closes
the two gaps in the "brand ids that cross package boundaries" policy and fixes
the dependency direction so a capability package never pulls in an unrelated one.

- Extract the `Branded<B>` primitive into a new standalone type-only package
  `@deepseek-ai/dsh-brand` (packages/util/brand) with no harness-package deps.
  dsh-llm keeps its owned CallId but imports Branded from dsh-brand; dsh-session,
  dsh-agent, and dsh-bash all import Branded from there. dsh-bash depends on
  dsh-brand ALONE — never on dsh-llm or dsh-session (the architectural fix: a
  generic execution backend must not couple to the LLM or session vocabulary).
- Mint BashTaskId + OwnerToken in dsh-bash and thread them through BashTask.id,
  the get/ownerOf/list/readOutput/kill seam, the bash-local generation site, and
  the dsh-tool-bash validate/access surface. OwnerToken is a DISTINCT brand from
  SessionId so the seam stays decoupled; dsh-tool-bash is the single boundary
  that casts SessionId -> OwnerToken.
- Brand at the SOURCE, not via mid-pipeline casts: agent-loop's Config types
  agents[].id as AgentId and resumeSessionId as SessionId, so the brand enters
  at the config boundary and the inner create()/resume casts disappear (only the
  genuinely-new per-run session-id string is cast).
- Stop brand erosion: propagate CallId/SessionId/AgentId to the registry/store
  Map keys and public params/exports (SessionStore, AgentRegistry + factory
  options, the ACP session-id surface + ToolPresenter CallId map, the
  persistence coordinator, invariants pendingCalls, the pi-ai tool-call maps).
- Docs: document BashTaskId/OwnerToken in bash.md (type-equiv re-pasted), point
  the Branded type-equiv at dsh-brand, fix stale param types in the session/
  agent/bash READMEs, regenerate the cordis catalog + module graph.

Implements docs/rfc/proposed/architecture/2026-06-20-branded-ids.md
2026-06-21 07:19:59 +08:00
Tianyi Cui 08f6f17cc1 Merge remote-tracking branch 'origin/master' into worktree-package-hierarchy 2026-06-20 22:55:38 +08:00
Tianyi Cui d02e9f1bd6 Reorganize packages into a modular hierarchy
Move the 18 flat packages/<name> packages into role-grouped dirs:
core/, llm/, bash/, session-persistence/, ui/, support/. Group dirs are
pure containers; each package keeps its @deepseek-ai/dsh-* name.

Collapse the per-package tsconfig paths maps (base + typecheck) into one
@deepseek-ai/dsh-* wildcard with a candidate per group, and derive the
publint list from the hierarchy. Update all depth-coupled globs/configs
(workspace, tsdown, vitest, eslint, knip, tsconfig includes/refs,
per-package tsconfigs, generators, doc-script scopes, type-equiv manifest)
and the cross-package/script relative imports in tests.

Fix doc-typecheck's workspacePaths() to parse tsconfig JSONC via the
TypeScript API instead of a regex comment-strip, which corrupted the
new wildcard `/*/` path candidates.

WIP: doc cross-links and package/RFC docs still to update.
2026-06-20 22:55:20 +08:00
Tianyi Cui 8c16524b90 chore(knip): drop dead entry pattern and fail on config hints
examples/acp-agent has no src/ directory — its plugins come from real
packages wired via cordis.yml, and its only local entry (start.ts) is
already auto-discovered. The examples/acp-agent/src/*.ts entry pattern
matched nothing, which knip surfaced as a config hint that exited 0 and
so went unnoticed.

Remove the dead pattern, and pass --treat-config-hints-as-errors so a
future stale entry (a renamed/removed path) fails the hygiene gate
instead of degrading to a silent hint.
2026-06-20 22:33:27 +08:00
Tianyi Cui 072f97c184 refactor(examples): extract reusable logic into tested packages
Logic that lived under examples/ was outside the per-file 100% coverage
gate (examples/ are not workspaces) and, in the stdio-UI case, duplicated
across two examples. Move it into packages/ so it is gated and de-duped.

- packages/ui-stdio (new): unify the two diverged stdio-chat.ts copies into
  one @deepseek-ai/dsh-ui-stdio plugin (welcome/agent Config). A test-only
  I/O seam (createStdioChat(ctx, config, runtime)) keeps process streams out
  of the serializable config and makes every render/EOF/disposal branch
  unit-testable. Per-file 100%. echo/coding cordis.yml now load the package;
  both src/stdio-chat.ts deleted.
- packages/llm-replay (new): move examples/acp-agent/src/llm-replay.ts (+ its
  spec) here so its derive/parse/replay branches fall under the coverage gate.
  cordis.snapshot.yml + README rewired to the package name; added apply/env
  /assertNever/abort tests to reach per-file 100%.
- examples/{echo,coding}-agent: keyless Loader-path e2e smokes that boot the
  real cordis.yml (no key) — the guard a hand-mounted unit test cannot be for
  the unwrapExports/export-shape class (postmortem 0001). examples/AGENTS.md
  codifies the keyless+with-key smoke convention (keyless-by-nature exception
  for echo-agent).
- AGENTS.md: a scoped, removal-triggered pre-release stance (foundation over
  blast radius). packages/README.md: new rows + a FIXME to later regroup ALL
  packages into a hierarchy. Wiring: tsconfig paths/refs, publint, knip,
  module-graph.

Verified: typecheck, lint, test:coverage (887 tests, 100%), build, hygiene,
doc-sync, test:snapshot (10), test:e2e (6 keyless pass, with-key self-skip).
2026-06-19 12:42:28 +08:00
Tianyi Cui 81d434896d feat(acp-example): snapshot harness, normalizers, wiring, and handshake scenario
Adds the snapshot-test harness and the keyless replay pipeline end-to-end.

- snapshot-harness.ts: boots the real acp-agent subprocess via the cordis
  Loader (preserving TSX_TSCONFIG_PATH so unbuilt dsh-* imports resolve from a
  temp cwd), tees raw stdout into an SDK ClientSideConnection, interprets a
  per-scenario input.json DSL (initialize / newSession capturing the random
  sessionId / prompt / cancel), closes stdin to trigger graceful shutdown, and
  harvests the persisted session.jsonl. Failure-safe: a finally block SIGKILLs
  a live child, awaits its exit, and removes both temp dirs even on a thrown
  step or harvest. Raw bytes are buffered and decoded once (no multibyte split).
- snapshot-normalize.ts (+ spec): two pure normalizers (stdout frames + session
  JSONL) scrub cwd, session ids / UUIDs, and JSON-RPC ids, and zero time /
  createdAt — but keep `seq` (deterministic by contract). normalizeStdout throws
  on a non-JSON line (the stdout-purity check).
- start.ts: selects cordis.snapshot.yml (replay, providerless) or
  cordis.snapshot-record.yml (record, real adapter) from DSH_SNAPSHOT, skips
  .env in replay, and disposes the ctx on stdin end so persistence flushes
  before exit (harvest-after-flush, not on the prompt response).
- acp.snapshot.ts: asserts the normalized stdout golden (and, for model
  scenarios, the re-persisted JSONL golden) via toMatchFileSnapshot; record mode
  writes the harvested log back to the scenario fixture; an orphan-fixture guard
  fails on an unregistered scenario dir.
- handshake scenario: initialize + session/new (no model call; a header-only
  session.jsonl, since session/new persists no events).
- vitest.snapshot.config.ts, test:snapshot / test:snapshot:record scripts, a
  pre-push snapshot job, and the knip entry.

Incorporates Codex review: record-fixture writeback, failure-safe teardown,
seq-not-scrubbed, harvest-after-flush. Per docs/rfc/implemented/2026-06-19.
2026-06-19 03:36:12 +08:00
Tianyi Cui 1a1ce734ba feat(acp-example): add record/replay llm/stream plugin for snapshot tests
Introduces examples/acp-agent/src/llm-replay.ts, a function/namespace plugin
that installs a single llm/stream waterfall listener. In record mode it tees
the real model's StreamChunks into a per-scenario llm.json (flushed atomically
after EACH stream, since the snapshot subprocess is SIGKILLed and start.ts has
no disposal path). In replay mode it short-circuits the waterfall and serves
recorded streams back positionally — the Nth stream() call gets the Nth entry —
so a snapshot test can drive the real agent with no API key.

Each fixture entry is a discriminated record {chunks|throw|hang} so it can
replay BOTH branches of the LLM failure contract (throw from stream() vs a
finish-error chunk) plus cancellation. A throw entry carries the prefix chunks
emitted before the throw, replayed before the error, so a mid-stream failure
(partial output then STREAM_CLOSED) reproduces what the loop saw live.

Fail-loud on a missing or exhausted fixture (never a silent skip). Unit tests
drive the real LlmService waterfall (record tee, ordered replay, the three
entry kinds, partial-then-throw, fail-loud, event-driven abort, HMR-safety).
Broadens the unit vitest include to examples/*/tests and registers the plugin
+ snapshot tests as knip entries. Per docs/rfc/implemented/2026-06-19.
2026-06-19 01:10:30 +08:00
Tianyi Cui fb9636db44 feat(acp): ACP bridge — drive the coding agent from an editor over JSON-RPC stdio
Implements the RFC 010 MVP: a new `@deepseek-ai/dsh-acp` package bridges the
harness agent to the Agent Client Protocol (JSON-RPC 2.0 over newline-delimited
stdio), so Zed and other ACP editors can drive the coding agent — streaming
render, tool-call display, and resumable sessions via `session/load`.

- packages/acp: AgentSideConnection wiring; initialize/newSession/loadSession/
  prompt/cancel; a total TurnEndReason→StopReason codec; settle-once with a
  fallback chain (agent/turn-end → logged turn/end → idle); single-session
  guard; cwd-must-equal-launch-dir validation; load replays from the persisted
  event log (assistant/chunk→agent_message_chunk, tool/call/result→tool_call*).
- agent: add Agent.whenIdle() quiescence signal to the interface; LoopAgent
  implements it (resolves on the first running→idle/disposed transition). The
  bridge awaits it on disposal so teardown reaches quiescence, not just abort.
- examples: extract the shared provider/tool core into examples/base.yml;
  coding-agent nest-includes it; new examples/acp-agent serves the agent over
  ACP with JSONL persistence and no stdout logger (stdout is the protocol).
- Permission gate deferred (TODO(rfc010-permission-gate)): tools run with the
  executor's full authority; only the Agent→sessionId ownership seam is laid
  down. Cancel is best-effort for a not-yet-started queued turn
  (TODO(rfc010-cancel-prestep)). RFC 010 stays `proposed`.
- Docs: package README + Zed snippet; client-driver cookbook section; root and
  packages layout/commands; RFC 010 implementation-status note.

48 bridge tests + whenIdle coverage; 100% per-file coverage; e2e boots the
example as a subprocess and verifies a written file on disk (key-gated, with a
no-key stdout-purity check).
2026-06-16 18:44:31 +08:00
Tianyi Cui e98c1c5d42 Add examples/coding-agent and the docs cookbook
The first real agent wiring: DeepSeek V4 + the bash tool suite + stdio
chat + JSONL persistence, runnable via yarn demo:coding (reads the
gitignored repo-root .env through process.loadEnvFile).

- examples/coding-agent: cordis.yml wiring both real plugin families
  (llm-deepseek with !!js env secrets; bash-local + tool-bash), a
  bash-only coding system prompt, a max-steps-guard plugin (bounds
  runaway turns via the agent/turn-continuation waterfall — abort()
  from step-end is a no-op by then), and a stdio UI with dimmed
  reasoning and exit-on-idle for piped stdin.
- e2e (yarn test:e2e, key-gated): full-loop.e2e.ts runs a real model
  against the real bash tool; coding-task.e2e.ts is the swebench-style
  smoke — the model fixes a buggy add.js in a temp dir and the test
  re-runs node add.test.js itself rather than trusting the agent.
- docs/cookbook: adding-a-package (the verified checklist),
  adding-a-tool (execute() contract, background pattern, seams),
  adding-an-llm-adapter (protocol obligations, mock-server testing,
  e2e policy). AGENTS.md layout/commands/secrets sections updated;
  architecture.md points at both examples and the cookbook.
- vitest.e2e.config.ts: serialize test files + retry twice — parallel
  e2e files trip the shared internal key's concurrency quota.
- fix: the !js YAML tag spelling in docs/JSDoc is actually !!js
  (js-yaml resolves custom tags under tag:yaml.org,2002:js).
2026-06-13 18:30:50 +08:00
Tianyi Cui ab19fed77c Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai
The first real LlmAdapter implementations, shipped as a deliberate pair:
same models and wire protocol, completely different internals, so the
StreamChunk protocol is verified across independent implementations.

- dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation
  state machine against the official chat-completions format (thinking
  mode via top-level thinking/reasoning_effort; the empty-string
  reasoning_content first chunk; usage attached to the finish chunk or
  trailing; reasoning_content passback on tool-call turns; disjoint
  cache-token accounting).
- dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai,
  mapping its event vocabulary (parsed tool arguments, in-stream error
  events, folded reasoning tokens) onto the same chunks.

The agent loop now honors the in-band error path: an adapter that ends
its stream with finish {kind:error|aborted} (the only option for
adapters that can't throw mid-stream, like pi-ai) is translated into a
step error, so the turn ends error/aborted with a logged error event
instead of a normal completed assistant message. This makes the
StreamChunk error contract real for both adapters; docs/architecture.md
and the StreamChunk doc are updated accordingly.

New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated
real-API matrices for both adapters across V4 Flash/Pro and all
thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit
suites run against local node:http mock SSE servers at 100% per-file
coverage.
2026-06-13 18:30:03 +08:00
Tianyi Cui 6796a3922d Add repo hygiene gates: knip, publint, yarn constraints
knip (workspace-aware config) fails on unused files, exports, and
dependencies in our code — vendor/ excluded, duplicate default+named
exports allowed (intentional API shape). publint checks every
packages/* package.json for publishing correctness (scripts/
publint-all.ts). Yarn constraints (yarn.config.cjs) mechanize the
AGENTS.md package rules: everything private, dsh-* packages declare
cordis as matching peer+dev dependency, uniform 0.0.1 version, ESM.

yarn hygiene runs all three.
2026-06-11 15:04:50 +08:00