Document the architecture and rewrite AGENTS.md

docs/architecture.md: layering, service map, event taxonomy, the session/turn/step lifecycle, Cordis waterfall semantics, an extension cookbook, the plugin sanity checklist mapping every MVP feature to its extension mechanism, and the deferred-work TODO list (sub-agents, persistence backends, compaction, DeepSeek V4 adapter, parallel tool execution, streaming-protocol review). AGENTS.md: repo layout, commands, conventions (dsh-* naming, ESM, effect-based registrations, declaration merging, waterfall semantics), and the vendoring policy pointer.
2026-06-11 10:55:05 +08:00
parent 53d1ef4a74
commit cacfae3cae
3 changed files with 401 additions and 2 deletions
@@ -4,9 +4,80 @@ This is the monorepo for the DeepSeek Harness group. It currently hosts the code

 ## Architecture

-This codebase is based on the **Cordis** framework. All necessary Cordis dependencies are copied into this monorepo as vendored source instead of being depended on via npm.
+This codebase is based on the **Cordis** framework, built microkernel-style: **everything is a plugin**. All necessary Cordis dependencies are copied into this monorepo as vendored source (under `vendor/`) instead of being depended on via npm.
+
+Read [docs/architecture.md](docs/architecture.md) before changing anything under `packages/` — it defines the service map, the event taxonomy, the session/turn/step lifecycle, and the plugin cookbook.

 ## Design Documents

 - [Coding Harness MVP 需求分析](https://trtgsjkv6r.feishu.cn/wiki/ZwK6wfBE9i91V6kzMGYcgRGanxg) — requirement analysis for the initial MVP.
 - [微内核Harness实现思路](https://trtgsjkv6r.feishu.cn/wiki/VS9Lw1kQki6mDJk2UHocyuphnsc) — discussion of the microkernel plugin-style architecture ("everything is a plugin").
+
+## Repository Layout
+
+```
+vendor/      Vendored Cordis framework source (original npm names, private).
+             See vendor/README.md for the manifest, local-modification log,
+             and the upstream sync procedure. Do NOT edit casually — every
+             divergence must be logged there.
+packages/    Harness packages, all named @deepseek-ai/dsh-<name>:
+  llm/            abstract LLM service + content-block vocabulary (no real adapter yet)
+  session/        event-sourced session log + in-memory store
+  system-prompt/  prompt-section + tool-schema assembly registry
+  tools/          tool registry + tools/execute waterfall
+  agent/          Agent interface, registry, agent/* event vocabulary
+  agent-loop/     THE concrete plugin: LoopAgent + the loop driver
+examples/    Runnable demos (not workspaces). echo-agent = mock model + echo
+             tool + stdio UI + JSONL persistence, wired via cordis.yml.
+docs/        architecture.md — the design doc.
+scripts/     build.ts — dumble JS bundling for all packages.
+```
+
+## Commands
+
+```sh
+yarn install        # Yarn 4 workspaces (node-modules linker), node >= 24
+yarn test           # vitest run (packages/*/tests/**/*.spec.ts)
+yarn typecheck      # tsc -b tsconfig.build.json (declarations only)
+yarn build          # typecheck + dumble JS bundles into each package's lib/
+yarn demo           # run examples/echo-agent (needs --expose-internals, the
+                    # script passes it; type "echo hi" to see a tool call)
+```
+
+Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root
+`tsconfig.json` (`vitest` resolves through `tsconfig.test.json`). Building is
+only needed for publishing/consumption outside the repo.
+
+## Conventions
+
+- **Package naming**: every npm package in this repo is `@deepseek-ai/dsh-<name>`
+  (vendored packages keep their upstream names and are `private: true`).
+- **ESM everywhere** (`"type": "module"`); imports between workspace packages
+  use package names, never relative paths across package boundaries.
+  In-package imports use explicit `.ts` extensions (allowImportingTsExtensions).
+- **`cordis` is a peerDependency** (+ devDependency) of every harness package,
+  mirroring upstream convention.
+- **Registrations are effects**: anything a plugin contributes (adapter, tool,
+  section, agent, event listener) goes through `ctx.effect()` / `ctx.on()` so
+  disposal and HMR work. If you write a registry, `register()` must return the
+  disposer.
+- **Typed events via declaration merging**: services declare their events in
+  `declare module 'cordis' { interface Events { … } }`, and their ctx key in
+  `interface Context`. Extensible unions use the merge-extensible-map pattern
+  (see `ContentBlockMap`, `MessageSourceMap`).
+- **Waterfall semantics**: `ctx.waterfall` listeners receive `(...args, next)`
+  and MUST call `next()` to delegate; returning without it short-circuits.
+  This is the veto mechanism — use deliberately.
+- **Plugins, not loop changes**: new behavior goes into a plugin on the
+  documented extension seams (see the plugin sanity checklist in
+  docs/architecture.md). Changing `agent-loop` requires updating that doc.
+- **Tests**: vitest, colocated under `packages/<name>/tests/*.spec.ts`. Every
+  registry needs an HMR-safety test (dispose the contributing fiber, assert
+  cleanup).
+
+## Vendoring Policy
+
+`vendor/` packages are pinned source copies (manifest with upstream commit
+SHAs in [vendor/README.md](vendor/README.md)). To update one, follow the sync
+procedure there; re-apply (or retire) the logged local modifications and rerun
+`yarn test && yarn build`.
@@ -8,4 +8,12 @@ Monorepo for the DeepSeek Harness group.

 ## Development

-This monorepo is built on the [Cordis](https://github.com/cordiverse/cordis) framework. See [AGENTS.md](AGENTS.md) for codebase conventions and agent instructions.
+This monorepo is built on the [Cordis](https://github.com/cordiverse/cordis) framework (vendored as source under `vendor/`), microkernel-style: everything is a plugin.
+
+```sh
+yarn install
+yarn test        # vitest
+yarn demo        # runnable echo-agent example
+```
+
+See [AGENTS.md](AGENTS.md) for layout, commands, and conventions, and [docs/architecture.md](docs/architecture.md) for the design.
@@ -0,0 +1,320 @@
+# DeepSeek Harness Architecture
+
+This document describes the phase-1 architecture of the DeepSeek Harness — the
+foundation of **DeepSeek Code**. The governing principle, from the
+[microkernel design discussion][microkernel-doc], is:
+
+> **Microkernel approach. Everything is a plugin.**
+
+The harness core is deliberately tiny: a handful of abstract services plus one
+concrete plugin (the agent loop). Every product feature — tools, hooks,
+compaction, sandboxing, UI, persistence, sub-agents, MCP, skills — is meant to
+be written as a plugin against the extension surface described here, without
+modifying the loop.
+
+Requirement context: [Coding Harness MVP 需求分析][mvp-doc].
+
+[microkernel-doc]: https://trtgsjkv6r.feishu.cn/wiki/VS9Lw1kQki6mDJk2UHocyuphnsc
+[mvp-doc]: https://trtgsjkv6r.feishu.cn/wiki/ZwK6wfBE9i91V6kzMGYcgRGanxg
+
+## Layering
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│  future plugins: tools, hooks, compaction, sandbox, UI, MCP… │
+├─────────────────────────────────────────────────────────────┤
+│  @deepseek-ai/dsh-agent-loop      (the ONE concrete plugin)  │
+├─────────────────────────────────────────────────────────────┤
+│  @deepseek-ai/dsh-agent           (vocabulary + registry)    │
+│  @deepseek-ai/dsh-tools           (registry + exec waterfall)│
+│  @deepseek-ai/dsh-system-prompt   (assembly registry)        │
+│  @deepseek-ai/dsh-session         (event-sourced log)        │
+│  @deepseek-ai/dsh-llm             (abstract model service)   │
+├─────────────────────────────────────────────────────────────┤
+│  vendor/: cordis, loader, include, group, timer, hmr,        │
+│           logger-console, cosmokit, schemastery              │
+└─────────────────────────────────────────────────────────────┘
+```
+
+Dependency rule: plugins depend on interface packages, never on
+`dsh-agent-loop`. The loop itself is swappable — UI/hook/tool plugins keep
+working against the `dsh-agent` vocabulary if the loop is replaced.
+
+## Service map
+
+| ctx key | Class | Package | Role |
+|---|---|---|---|
+| `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` / `streamBlocks()` / `generate()` |
+| `ctx.sessions` | `SessionStore` | dsh-session | creates/holds event-sourced `Session`s |
+| `ctx.systemPrompt` | `SystemPrompt` | dsh-system-prompt | ordered sections + tool schemas → `assemble()` |
+| `ctx.tools` | `ToolRegistry` | dsh-tools | tool definitions; `execute()` through waterfall |
+| `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles |
+| `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `LoopAgent`s and drives their loops |
+
+All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go
+through `ctx.effect()` and return disposers, so plugin hot-reload (vendored
+HMR) and fiber disposal clean up automatically.
+
+## The vocabulary (dsh-llm)
+
+Messages are arrays of typed **content blocks** (`text`, `reasoning`,
+`tool-call`, `tool-result`, `image`); the union is derived from the
+merge-extensible `ContentBlockMap`, so plugins can add block types via
+declaration merging. The same merge-extensible-map pattern is used for
+`MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason` — typed
+sum types instead of strings.
+
+Streaming is a raw chunk protocol (`block-start`, `text-delta`,
+`reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`).
+`BlockAssembler` is the single shared implementation that assembles chunks
+into blocks/messages; the loop logs raw chunks (replay fidelity) while feeding
+the same chunks through an assembler.
+
+`LlmAdapter` is the provider seam: subclass, implement `stream()`, call
+`ctx.llm.registerAdapter(models, adapter)`.
+**TODO**: the DeepSeek V4 adapter is the first real adapter (next phase); the
+streaming protocol gets a careful review then.
+
+## Event-sourced sessions (dsh-session)
+
+A `Session` is an append-only log of typed `SessionEvent`s — the single source
+of truth. The LLM message history is *derived* from the log
+(`deriveMessages()`):
+
+- `user/message` → user message
+- `assistant/message` → assistant message (raw `assistant/chunk` events are
+  replay/UI data and are skipped in derivation)
+- `tool/result` → user message carrying a `tool-result` block
+- `context/message`, `steering/message` → user-role messages wrapped in a
+  tagged envelope (`<context source="…">…</context>`) at their chronological
+  position — the "system-reminder" pattern; models distinguish them from real
+  user prompts by the envelope. **TODO(review)**: revisit the envelope once a
+  real adapter exists.
+
+Replay/fork = `ctx.sessions.create(id, seedEvents)`. Trace/telemetry = listen
+to `session/event`.
+
+**Durability seam**: `session/event` is a synchronous notification;
+persistence plugins buffer (write-behind) and drain at the awaited
+`session/flush` checkpoint the loop fires at every turn end (see
+`examples/echo-agent/src/session-jsonl.ts` for the pattern).
+**TODO**: real persistence backends (JSONL per session dir, sqlite) are a
+future phase.
+
+## Prompt assembly (dsh-system-prompt)
+
+Plugins contribute `PromptSection`s (named, ordered, static or computed) and
+tool-schema providers. `assemble()` returns a `PromptAssembly { sections,
+tools }` through the `system-prompt/assemble` waterfall.
+
+Tool schemas are deliberately **part of the assembly**: "what the model is
+told it can do" is one coherent thing managed here, even though adapters
+transmit schemas as the wire-level `tools` field rather than prompt text.
+
+## Tool pipeline (dsh-tools)
+
+`ToolRegistry.register()` takes schema + `execute()`. The registry feeds its
+schemas into the system-prompt assembly automatically.
+
+`execute()` runs through the **`tools/execute` waterfall** — the single seam
+where sandbox, permission, hooks, and plan-mode plugins wrap or veto a call.
+This collapses Claude Code's validate → PreToolUse → permission → execute →
+PostToolUse pipeline into ordered waterfall listeners.
+
+**TODO**: tool shapes get revisited when real tools land (e.g. a
+concurrency-safety hint for parallel execution; phase 1 executes tool calls
+sequentially).
+
+## Agents (dsh-agent) and the loop (dsh-agent-loop)
+
+`Agent` is the handle every plugin programs against:
+
+- `send(content)` — queued message; starts a turn when idle, else next turn
+- `steer(content)` — mid-turn injection, drained **between steps**; behaves
+  like `send` when idle
+- `inject(content)` — in-session context (`context/message` event) without
+  triggering a turn; the next request sees it (Claude Code attachment /
+  system-reminder analog)
+- `abort(reason)` — aborts the in-flight step via `AbortSignal`
+- `session`, `status`, `options`
+
+**TODO(sub-agents)**: `spawn`/`fork` land on `AgentLoop.create()` — fork seeds
+the child Session with the parent's event log, spawn starts fresh; children
+are ordinary `Agent` handles so `steer()` and event subscription work
+uniformly. Inter-agent channels beyond these primitives are deliberately
+deferred.
+
+### Loop lifecycle (session / turn / step)
+
+- **Session**: the whole event log of one agent.
+- **Turn**: triggered by ≥1 queued message; runs steps until the model stops
+  requesting tools and no plugin requests continuation.
+- **Step**: one model request + its tool executions.
+
+```
+forever:
+  wait for queued messages (idle)
+  TURN: drain queued → session('user/message'…) → 'turn/start' → emit agent/turn-start
+    STEP loop:
+      emit agent/step-start
+      assembly = ctx.systemPrompt.assemble()          ⟵ waterfall system-prompt/assemble
+      req = {model, system, tools, messages: session.deriveMessages(), signal}
+      req = waterfall agent/request                   ⟵ hooks, compaction, model switch
+      stream ctx.llm.stream(req)                      ⟵ waterfall llm/stream (raw chunks)
+        session('assistant/chunk'); emit agent/stream-chunk
+      session('assistant/message', 'usage')
+      msg = waterfall agent/step-result               ⟵ post-process before tool dispatch
+      each tool-call (sequential):
+        session('tool/call'); ctx.tools.execute()     ⟵ waterfall tools/execute
+        session('tool/result')
+      drain steering → session('steering/message'); emit agent/steering
+      emit agent/step-end
+      cont = waterfall agent/turn-continuation(default = hadToolCalls || steered)
+      if !cont: break
+  session('turn/end'); emit agent/turn-end
+  await ctx.parallel('session/flush', session)        ⟵ durability checkpoint
+  idle (emit agent/status) unless more queued
+```
+
+### Event taxonomy
+
+Declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package).
+
+| Event | Mode | Purpose |
+|---|---|---|
+| `agent/created` / `agent/disposed` / `agent/status` / `agent/queued` | emit | lifecycle + inbox notifications |
+| `agent/turn-start` / `agent/turn-end` / `agent/step-start` / `agent/step-end` | emit | boundaries |
+| `agent/request` | **waterfall** | mutate the final `GenerateOptions` before the model call |
+| `agent/stream-chunk` | emit | token-level UI/log feed |
+| `agent/step-result` | **waterfall** | post-process the assistant message before tool dispatch |
+| `agent/steering` | emit | steering content injected |
+| `agent/turn-continuation` | **waterfall** | override the continue/stop decision |
+| `agent/error` | emit | step/turn errors |
+| `tools/execute` (dsh-tools) | **waterfall** | wrap/veto/sandbox tool execution |
+| `llm/stream` / `llm/generate` (dsh-llm) | **waterfall** | model-call interception |
+| `system-prompt/assemble` (dsh-system-prompt) | **waterfall** | mutate the assembly |
+| `session/created` / `session/event` (dsh-session) | emit | session lifecycle + log feed |
+| `session/flush` (dsh-session) | parallel (awaited) | durability checkpoint |
+
+### Cordis waterfall semantics (important)
+
+`ctx.waterfall` is **around-middleware**, not a value reducer. Each listener
+receives `(...args, next)`:
+
+- call `next()` to delegate to later listeners (and ultimately the core
+  behavior), possibly wrapping it;
+- return a value **without** calling `next()` to short-circuit (veto);
+- listeners run in registration order; `prepend: true` jumps the queue.
+
+Mutate the passed-in object (e.g. `options.model = '…'`) or return a
+replacement value — both compose.
+
+## Plugin sanity checklist
+
+Every MVP feature (including the TODO-marked ones), with the mechanism that
+implements it **without modifying the loop**:
+
+| MVP feature | Plugin mechanism |
+|---|---|
+| Hook system (user + project level) | listeners on `agent/request`, `agent/step-result`, `tools/execute`, `agent/turn-continuation`; a hooks plugin bridges config files to shell commands |
+| `/goal` | force-continue via `agent/turn-continuation` + `steer()` reminders |
+| `/loop` | on `agent/turn-end`, `send()` the next iteration; or force-continue |
+| Dynamic workflow | orchestrator plugin on `agent/turn-end` / `agent/step-end` driving `send`/`steer` (+ sub-agents later) |
+| Queued + steering messages | core `Agent.send()` / `Agent.steer()` |
+| Context compaction (auto + manual) | wrap `agent/request`: measure tokens, rewrite `req.messages`, append merged `compaction/*` session events; manual = a command plugin invoking the same routine |
+| System prompt configurability | `ctx.systemPrompt.section()` with ordering |
+| AGENTS.md (root) | a section provider reading the file |
+| AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener |
+| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically |
+| ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` |
+| Tool sandbox (landlock / sandbox-exec) | wrap `tools/execute` |
+| Permission system / AskUserQuestion | wrap `tools/execute` (veto or ask); register an ask tool |
+| Plan mode | wrap `tools/execute` (deny writes) + `agent/request` (inject mode prompt) |
+| Sub-agents (spawn / fork / steer) | TODO seam on `AgentLoop.create()`; fork = seed Session with parent events; `steer()` on the child handle |
+| MCP | one plugin per server: discover tools → `ctx.tools.register()` |
+| Skills | section + tool registration; `inject()` skill content on invocation |
+| Memory | section provider + tool |
+| Scheduled tasks (cron) | plugin registers model-callable scheduling tools; timer fires → `send(…, {source: {kind: 'cron', …}})` when idle / `inject()` notification when busy |
+| UI (GUI; CLI emits JSONL) | listen `agent/stream-chunk` + `session/event`; input → `send()` |
+| Telemetry / replayable trace | `session/event` → JSONL; replay = `sessions.create(id, seed)` |
+| DeepSeek V4 (and other) models | `LlmAdapter` subclass via `registerAdapter` |
+| Plugin hot-reload | every registration is a `ctx.effect` → vendored HMR just works |
+
+## Extension cookbook
+
+### A tool plugin
+
+```ts
+import type { Context } from 'cordis'
+
+export const name = 'my-tool'
+export const inject = ['tools']
+
+export function apply(ctx: Context) {
+  ctx.tools.register({
+    name: 'read_file',
+    description: 'Read a file from disk.',
+    parameters: { type: 'object', properties: { path: { type: 'string' } }, required: ['path'] },
+    async execute(args: any) {
+      const text = await readFile(args.path, 'utf8')
+      return [{ type: 'text', text }]
+    },
+  })
+}
+```
+
+### A hook plugin (permission gate)
+
+```ts
+export const name = 'permission-gate'
+
+export function apply(ctx: Context) {
+  ctx.on('tools/execute', async (exec, next) => {
+    if (!(await isAllowed(exec))) {
+      return {
+        callId: exec.callId,
+        content: [{ type: 'text', text: 'Denied by policy.' }],
+        isError: true,
+      }
+    }
+    return next()
+  })
+}
+```
+
+### A UI plugin
+
+```ts
+export const name = 'my-ui'
+export const inject = ['agents']
+
+export function apply(ctx: Context) {
+  ctx.on('agent/stream-chunk', (agent, turn, step, chunk) => {
+    if (chunk.type === 'text-delta') render(chunk.text)
+  })
+  onUserInput(text => ctx.agents.get('main')?.send([{ type: 'text', text }]))
+}
+```
+
+A complete runnable wiring lives in [`examples/echo-agent`](../examples/echo-agent)
+(mock model + echo tool + stdio UI + JSONL persistence, loaded from
+`cordis.yml` with HMR).
+
+## Deferred work (TODO)
+
+Tracked here deliberately — each is designed-for but not implemented:
+
+- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent
+  channels beyond `send`/`steer`/events.
+- **Persistence backends** (JSONL session dirs, sqlite) on the
+  `session/event` + `session/flush` seam.
+- **Compaction implementation** (auto thresholds, summarization prompts) on
+  the `agent/request` seam, with its session-event types added by declaration
+  merging.
+- **DeepSeek V4 adapter** — first real `LlmAdapter`; triggers the
+  streaming-protocol review (`TODO(review)` markers in dsh-llm).
+- **Parallel tool execution** (concurrency-safety hints on ToolDefinition).
+- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based
+  forking.
+- **Session event vocabulary review** once the loop and a persistence plugin
+  coexist (`TODO(review)` in dsh-session).