diff --git a/AGENTS.md b/AGENTS.md index 85bec46560..c8a1cd226d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -21,7 +21,7 @@ packages/ Harness packages at packages///, all named @deepseek-ai todo/ the todo_write tool hooks/ Claude Code / Codex hook bridges + shared wire-protocol library session-persistence/ persistence seam + JSONL/SQLite backends - ui/ ACP bridge + the stdio/ACP app packages (each with a bin) + ui/ ACP bridge + app-boot glue + the stdio/ACP app bins support/ dev/test infrastructure: invariants, llm-replay, subagent-mock util/ zero-dependency utilities (Branded) examples/ Runnable demos: thin cordis.yml leaves over the app packages (see examples/AGENTS.md) diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 7b210df89f..b1df4a02d9 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -17,7 +17,7 @@ Every fact has exactly one home — the tier whose job it is — and every other | [cookbook/](cookbook/adding-a-package.md) | Step-by-step how-tos with numbered verify steps | Design rationale (→ the RFC each guide links) | | Package README | The per-package contract: config, semantics, limitations, extension points | JSDoc restatement, generated-catalog restatement (event/tool tables), other packages' concerns | | [development.md](development.md) | Human-facing setup and daily workflow; a bilingual pair under the [i18n contract](i18n/README.md) | Gate-by-gate enumerations that drift from `package.json` scripts | -| Generated catalogs: [cordis-catalog](cordis-catalog/events-and-services.md), [tool-catalog](tool-catalog/tools.md), [module-graph.md](module-graph.md) | Exhaustive enumerations regenerated from source, freshness-gated | Hand edits of any kind | +| Generated catalogs: [cordis-catalog](cordis-catalog/events-and-services.md), [tool-catalog](tool-catalog/tools.md), [persistence-catalog](persistence-catalog/log-events.md), [module-graph.md](module-graph.md) | Exhaustive enumerations regenerated from source, freshness-gated | Hand edits of any kind | | Skills (`.agents/skills/`) | Workflows: how to carry out a recurring task against the contracts | The contracts themselves (→ docs) | Placement test: a story about a bug → postmortem. Why we chose X → RFC. How to do task Y → cookbook. What type Z looks like → core-data-structures. What package P promises → its README. A rule every agent must always obey → root AGENTS.md, one line, linking the home that holds the why. diff --git a/docs/architecture.md b/docs/architecture.md index 800a1a2099..f7013b29d0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -59,7 +59,7 @@ Two seams bend the template deliberately: ## The vocabulary (dsh-llm) -Messages are arrays of typed **content blocks** (`text`, `reasoning`, `tool-call`, `tool-result`, `image`); the union derives from the merge-extensible `ContentBlockMap`; the same pattern types `MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`. Streaming is a raw chunk protocol (`block-start` … `finish`) with `BlockAssembler` as the single shared chunk→block assembler; the loop logs raw chunks (replay fidelity) while assembling them. `LlmAdapter` is the provider seam: subclass, implement `stream()`, register via `ctx.llm.registerAdapter(models, adapter)`; `dsh-llm-deepseek` and `dsh-llm-pi-ai` implement the one contract as deliberate design twins ([twin RFC](rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md)). The StreamChunk conventions (usage/finish ordering, raw-string tool arguments, the two sanctioned error paths) are pinned in `dsh-llm/src/types.ts` and [llm-streaming.md](core-data-structures/llm-streaming.md). +Messages are arrays of typed **content blocks** (`text`, `reasoning`, `tool-call`, `tool-result`); the union derives from the merge-extensible `ContentBlockMap`; the same pattern types `MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`. The core set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map in the same coordinated change that maps it in the adapters, surfaces it in the UI bridges, and prices it in compaction ([the drop-image RFC](rfc/implemented/simplification/2026-07-04-drop-image-content-block.md)). Streaming is a raw chunk protocol (`block-start` … `finish`) with `BlockAssembler` as the single shared chunk→block assembler; the loop logs raw chunks (replay fidelity) while assembling them. `LlmAdapter` is the provider seam: subclass, implement `stream()`, register via `ctx.llm.registerAdapter(models, adapter)`; `dsh-llm-deepseek` and `dsh-llm-pi-ai` implement the one contract as deliberate design twins ([twin RFC](rfc/implemented/architecture/2026-06-13-twin-llm-adapters.md)). The StreamChunk conventions (usage/finish ordering, raw-string tool arguments, the two sanctioned error paths) are pinned in `dsh-llm/src/types.ts` and [llm-streaming.md](core-data-structures/llm-streaming.md). ## Event-sourced sessions (dsh-session) @@ -118,7 +118,7 @@ forever: session('tool/result') append buffered post-execute additionalContext → session('context/message')(s) ⟵ after ALL tool/results (adjacency) - drain steering → session('steering/message'); emit agent/steering + drain steering → session('steering/message') session('step/end') ⟵ durable step boundary (no agent/* mirror) cont = waterfall agent/turn-continuation(default = {action: hadToolCalls||steered ? 'continue' : 'stop'}) → ContinuationDecision diff --git a/docs/cookbook/adding-an-llm-adapter.md b/docs/cookbook/adding-an-llm-adapter.md index aae6b0bb8b..fb59969e52 100644 --- a/docs/cookbook/adding-an-llm-adapter.md +++ b/docs/cookbook/adding-an-llm-adapter.md @@ -27,7 +27,7 @@ Registration is effect-based (HMR-safe); one adapter per model name — duplicat - Allocate block `index`es in first-seen stream order; reuse the index for every delta of the same block. - Errors have exactly two sanctioned paths: THROW from `stream()` (transport and protocol failures — use `LlmError` with a stable code), or end the stream with `finish {kind: 'error' | 'aborted'}` (provider in-band failures). Consumers handle both; pick per failure class and document it. - Honor `options.signal` (pass it to fetch / your SDK). -- `prefill` and other unsupported `GenerateOptions` fields: throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping. +- A `GenerateOptions` field your provider cannot honor (e.g. a `stop` list on a provider without stop sequences): throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping it. Provider-specific request knobs (thinking modes, effort levels) belong in the ADAPTER's Config, not in `GenerateOptions` — the core vocabulary stays provider-neutral. diff --git a/docs/cordis-catalog/events-and-services.md b/docs/cordis-catalog/events-and-services.md index 875a4e52fa..922a0ea018 100644 --- a/docs/cordis-catalog/events-and-services.md +++ b/docs/cordis-catalog/events-and-services.md @@ -49,7 +49,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w Types: [Agent](../core-data-structures/core.md) -Source: [`packages/core/agent/src/types.ts:389`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:380`](../../packages/core/agent/src/types.ts) #### `agent/pre-step` — serial @@ -125,18 +125,6 @@ Types: [Agent](../core-data-structures/core.md) Source: [`packages/core/agent/src/types.ts:250`](../../packages/core/agent/src/types.ts) -#### `agent/steering` — emit - -Steering content was injected into a running turn. - -```ts cordis-catalog -'agent/steering'(agent: Agent, turn: number, content: ContentBlock[], source: MessageSource): void -``` - -Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) - -Source: [`packages/core/agent/src/types.ts:379`](../../packages/core/agent/src/types.ts) - #### `agent/step-result` — waterfall Waterfall: post-process the assembled assistant Message before tool dispatch (validation, content rewriting, …). @@ -211,7 +199,7 @@ Waterfall around every streaming model call (retry, caching, routing). Bound to Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:32`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:33`](../../packages/llm/llm/src/index.ts) ### `session/*` @@ -327,18 +315,6 @@ Types: [ToolExecution](../core-data-structures/tools.md) Source: [`packages/core/tools/src/index.ts:66`](../../packages/core/tools/src/index.ts) -### `web/*` - -#### `web/providers-change` — emit - -Fired after the provider registry changes — a search or fetch provider was registered or disposed. Carries no payload and no capability graph: it means only "the provider registry changed; observers may recompute status from `ctx.web`". `searchStatus()` / `fetchStatus()` stay derived, not stored. - -```ts cordis-catalog -'web/providers-change'(this: WebService): void -``` - -Source: [`packages/web/web/src/index.ts:65`](../../packages/web/web/src/index.ts) - ## Services The `ctx.` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against. @@ -458,7 +434,7 @@ stream(options: GenerateOptions): AsyncIterable Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:70`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:78`](../../packages/llm/llm/src/index.ts) ### `ctx.sessionPersistence` — `SessionPersistence` (abstract seam) @@ -543,25 +519,23 @@ Source: [`packages/core/tools/src/index.ts:268`](../../packages/core/tools/src/i The web access service. Registered as `ctx.web` (one instance per context). -Selection semantics (identical for status and execution, never order- dependent): +Selection semantics (resolved at execution time, never order-dependent): - A configured id that is registered and `status().available` → that provider. -- A configured id not registered → `configured-missing` / `WEB_PROVIDER_CONFIGURED_MISSING`. -- A configured id registered but unavailable → `configured-unavailable` / `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`. +- A configured id not registered → `WEB_PROVIDER_CONFIGURED_MISSING`. +- A configured id registered but unavailable → `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`. - No id configured, exactly one registered usable provider → that provider. -- No id configured, multiple usable providers → `ambiguous` / `WEB_PROVIDER_AMBIGUOUS`. -- No id configured, no usable provider → `none` / `WEB_PROVIDER_UNAVAILABLE`. +- No id configured, multiple usable providers → `WEB_PROVIDER_AMBIGUOUS`. +- No id configured, no usable provider → `WEB_PROVIDER_UNAVAILABLE`. ```ts cordis-catalog registerSearchProvider(provider: WebSearchProvider): () => void registerFetchProvider(provider: WebFetchProvider): () => void -searchStatus(): WebCapabilityStatus -fetchStatus(): WebCapabilityStatus async search(request: WebSearchRequest, exec?: WebExecContext): Promise async fetch(request: WebFetchRequest, exec?: WebExecContext): Promise ``` -Source: [`packages/web/web/src/index.ts:105`](../../packages/web/web/src/index.ts) +Source: [`packages/web/web/src/index.ts:87`](../../packages/web/web/src/index.ts) ## Inherited tier (cordis core + loader/hmr/timer) diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 9a282e305c..7f38abe985 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -87,11 +87,10 @@ interface ContentBlockMap { 'reasoning': ReasoningBlock 'tool-call': ToolCallBlock 'tool-result': ToolResultBlock - 'image': ImageBlock } ``` -The block interfaces (full fields in source): `TextBlock` (`text`), `ReasoningBlock` (thinking, distinct from visible text), `ToolCallBlock` (`id: CallId`, `name`, raw-JSON `arguments`), `ToolResultBlock` (`toolCallId`, nested `content: ContentBlock[]`, `isError?`), `ImageBlock` (`url`, `mimeType?`). `ContentBlock = ContentBlockMap[ContentBlockType]`. +The block interfaces (full fields in source): `TextBlock` (`text`), `ReasoningBlock` (thinking, distinct from visible text), `ToolCallBlock` (`id: CallId`, `name`, raw-JSON `arguments`), `ToolResultBlock` (`toolCallId`, nested `content: ContentBlock[]`, `isError?`). `ContentBlock = ContentBlockMap[ContentBlockType]`. The core set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the merge-extensible map together with the adapter/UI/compaction support that honors it. A `Message` is a role plus blocks: @@ -108,7 +107,6 @@ Where a message came from is itself a merge-extensible sum type: interface MessageSourceMap { user: { kind: 'user' } plugin: { kind: 'plugin'; plugin: string } - agent: { kind: 'agent'; agentId: string } } ``` @@ -132,8 +130,6 @@ interface GenerateOptions { system?: string /** Tool schemas (adapters map to the provider's `tools` field). */ tools?: ToolSchema[] - /** Assistant prefix continuation (prefill). */ - prefill?: ContentBlock[] temperature?: number maxTokens?: number /** @@ -182,7 +178,6 @@ interface ToolSchema { description: string /** JSON Schema object for the arguments. */ parameters: Record - strict?: boolean } ``` diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index e384ffcc92..e13322dc5a 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -12,7 +12,6 @@ Every operation resolves a user-supplied path to an opaque backend target first. ```ts type-equiv interface FsTarget { - inputPath: string targetKey: FsTargetKey displayPath: string } @@ -81,8 +80,6 @@ interface FsEditRequest { ```ts type-equiv interface FsEditOutcome { - replacements: number - replaceAll: boolean version: FsVersion before: string after: string @@ -109,16 +106,14 @@ interface FsPolicyExec { ## Read outcome (consumer / read rendering) -A text read is bounded by line window, byte cap, and backend limits. The outcome the model-facing `read` tool renders carries the file's version at read time; there is no `full`/`partial` view — authorization is freshness-based, so any windowed read can authorize a later write/edit when the file is unchanged. Read windowing and this outcome shape live in `dsh-tool-fs` (the executor that owns the read), not in the policy plugin. +A text read is bounded by line window, byte cap, and backend limits. The outcome the model-facing `read` tool renders is purely presentational; there is no `full`/`partial` view — authorization is freshness-based (the tool emits `fs/observed` with the stat's version directly), so any windowed read can authorize a later write/edit when the file is unchanged. Read windowing and this outcome shape live in `dsh-tool-fs` (the executor that owns the read), not in the policy plugin. ```ts type-equiv interface FileReadOutcome { offset: number - limit: number lines: FileTextLine[] totalLines: number truncatedByBytes?: true - version: FsVersion } ``` diff --git a/docs/core-data-structures/llm-streaming.md b/docs/core-data-structures/llm-streaming.md index 7439eb14a6..54b804e95d 100644 --- a/docs/core-data-structures/llm-streaming.md +++ b/docs/core-data-structures/llm-streaming.md @@ -26,9 +26,22 @@ Every adapter MUST obey these, and every consumer may rely on them: - **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering. - **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`. - **Two sanctioned error paths.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted'}` (provider in-band errors, for adapters that can't throw mid-stream). Consumers must handle *both*. The agent loop translates a finish-error/aborted into a turn error — it never logs a normal completed assistant message for a failed step. +- **Every provider HTTP request carries the app-attribution header.** Adapters send `attributionHeaders()` (below) - the `User-Agent` baseline - and prove it with a wire-level test (mock server asserting the received header, or the library's header hook for a library-backed adapter). This contract is why two adapters exist as a deliberate pair: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (the same endpoint through `@earendil-works/pi-ai`). Two independent internals over one contract is what pinned the protocol down — the library-backed adapter can't throw mid-stream, so it exercises the finish-chunk error path the hand-rolled one might not. +## `AppIdentity` — app attribution + +The static public application identity every adapter sends to providers ([`packages/llm/llm/src/attribution.ts`](../../packages/llm/llm/src/attribution.ts)). `attributionHeaders(identity?)` maps it to the standard `User-Agent` header only; OpenRouter-specific app attribution headers are intentionally not supported by this contract. The default `APP_IDENTITY` sources its version from the package manifest; every field is a public product fact - no secrets, paths, session ids, or per-user identifiers, and nothing per-request may influence the values. Rationale: [Mandatory `User-Agent` attribution](../rfc/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md). + +```ts type-equiv +interface AppIdentity { + product: string + version: string + url: string +} +``` + ## `TokenUsage` Per-call token accounting. Counts are **disjoint**: `inputTokens` is uncached input only; cached input is reported separately, and billed input is the sum of the three. Adapters whose providers fold cache hits into a single prompt total (DeepSeek's `prompt_tokens`) subtract them back out. @@ -59,7 +72,6 @@ interface ContentBlockMap { 'reasoning': ReasoningBlock 'tool-call': ToolCallBlock 'tool-result': ToolResultBlock - 'image': ImageBlock } ``` diff --git a/docs/core-data-structures/persistence.md b/docs/core-data-structures/persistence.md index 327162792a..ec5589dd48 100644 --- a/docs/core-data-structures/persistence.md +++ b/docs/core-data-structures/persistence.md @@ -1,6 +1,6 @@ # Session Persistence -The **durability seam** for the event log. [session.md](session.md) describes the in-memory `Session` — the append-only `SessionEvent` log that is the source of truth. This page describes how that log is made durable: the abstract `SessionPersistence` service, its backends, the flush checkpoint, crash recovery, and the metadata header that travels alongside the log. +The **durability seam** for the event log. [session.md](session.md) describes the in-memory `Session` — the append-only `SessionEvent` log that is the source of truth. This page describes how that log is made durable: the abstract `SessionPersistence` service, its backends, the flush checkpoint, crash recovery, and the metadata header that travels alongside the log. The event vocabulary the log carries is enumerated, member by member, in the generated [persistence log event catalog](../persistence-catalog/log-events.md). The seam is a textbook [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md): one abstract service ([dsh-session-persistence](../../packages/session-persistence/session-persistence), `ctx.sessionPersistence`) defining create/append/load/list over the existing `SessionEvent` — **no parallel persisted type** — and two interchangeable backends that pass the same `runPersistenceContract` suite. See the [session-persistence RFC](../rfc/implemented/architecture/2026-06-14-session-persistence.md). diff --git a/docs/core-data-structures/session.md b/docs/core-data-structures/session.md index 4d2b47c05c..92a9d3421f 100644 --- a/docs/core-data-structures/session.md +++ b/docs/core-data-structures/session.md @@ -6,7 +6,7 @@ Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/t ## `SessionEventMap` — the event vocabulary -The append-only event types. Merge-extensible: a plugin declares extra event types via declaration merging — e.g. the [compaction seam](compaction.md) adds `compact/start` / `compact/summary` / `compact/end`, and `@deepseek-ai/dsh-hook-protocol` adds log-only `hook/invoked` / `hook/result` provenance for a hook bridge. Like `compact/*`, these are NOT `SurfaceEventType`s (no `surfaceOp`). +The append-only event types. Merge-extensible: a plugin declares extra event types via declaration merging — e.g. the [compaction seam](compaction.md) adds `compact/start` / `compact/summary` / `compact/end`, and `@deepseek-ai/dsh-hook-protocol` adds log-only `hook/invoked` / `hook/result` provenance for a hook bridge. Like `compact/*`, these are NOT `SurfaceEventType`s (no `surfaceOp`). The generated [persistence log event catalog](../persistence-catalog/log-events.md) enumerates every member — core and merged — with its payload, surface badge, and declaration site. ```ts type-equiv interface SessionEventMap { @@ -164,7 +164,6 @@ Everything else (`turn/*`, `step/*`) is structural and does not project into a m ```ts type-equiv interface TurnTriggerMap { message: { kind: 'message'; source: MessageSource } - continuation: { kind: 'continuation' } /** * An out-of-band context injection (`agent.inject()`) made while the agent * was idle. The loop wraps the injected `context/message` in a one-shot turn @@ -224,14 +223,9 @@ Every session event lives **inside** a turn (between a `turn/start` and its `tur ## Plugin-contributed log-only events -A plugin may declaration-merge extra `SessionEventMap` types. These are **log-only**: NOT `SurfaceEventType`s (they carry no `surfaceOp` and contribute nothing to derived history), but, like every event, they must sit inside an open turn. The compaction seam's `compact/*` are documented on [compaction.md](compaction.md); the hook bridges' `hook/*` provenance (from `@deepseek-ai/dsh-hook-protocol`) are: +A plugin may declaration-merge extra `SessionEventMap` types. These are **log-only**: NOT `SurfaceEventType`s (they carry no `surfaceOp` and contribute nothing to derived history), but, like every event, they must sit inside an open turn. The full per-event enumeration — core and plugin-contributed alike, with payloads and provenance — is the generated [persistence log event catalog](../persistence-catalog/log-events.md); the compaction seam's `compact/*` semantics are discussed on [compaction.md](compaction.md). -| Event | Payload | Role | -|---|---|---| -| `hook/invoked` | `{ turn, point, dialect, matcher?, handlerId }` | A hook command was invoked at a hook `point` (`PreToolUse`, `Stop`, …). `dialect` is the bridge (`claude`/`codex`/`native`); `matcher` the matcher-group pattern that selected it (absent for match-all); `handlerId` correlates with the result. | -| `hook/result` | `{ turn, point, handlerId, decision, exitCode?, stderrSummary?, durationMs }` | The decided outcome, paired by `handlerId`. `decision` is the resolved neutral outcome (`deny`/`allow`/`block`/`stop`/`pass`/…); `exitCode` absent when the hook could not run; `stderrSummary` the truncated block-reason source. | - -The mid-turn hook points (`PreToolUse`/`PostToolUse`/`UserPromptSubmit`/`Stop`) fire inside the loop's open turn, so their `hook/*` records are turn-enclosed by construction. `SessionStart` gets no `hook/*` record — its injected `context/message` is the durable evidence — because it has no open turn to enclose one (see the hooks RFC). +The hook bridges' `hook/invoked` / `hook/result` provenance pairs (from `@deepseek-ai/dsh-hook-protocol`) correlate by `handlerId`. The mid-turn hook points (`PreToolUse`/`PostToolUse`/`UserPromptSubmit`/`Stop`) fire inside the loop's open turn, so their `hook/*` records are turn-enclosed by construction. `SessionStart` gets no `hook/*` record — its injected `context/message` is the durable evidence — because it has no open turn to enclose one (see [the hook-bridges RFC](../rfc/implemented/feature/2026-06-30-hook-bridges.md)). ## Durability contract diff --git a/docs/core-data-structures/web.md b/docs/core-data-structures/web.md index 42797a8d99..05c3f648c0 100644 --- a/docs/core-data-structures/web.md +++ b/docs/core-data-structures/web.md @@ -73,9 +73,9 @@ type WebFetchBody = | { readonly kind: 'text'; readonly content: string } ``` -## Provider and capability status +## Provider status -A provider's `status()` is a cheap LOCAL check (credential presence, parseable config) and **must not make network calls**. It is an input to selection, not a health system. +A provider's `status()` is a cheap LOCAL check (credential presence, parseable config) and **must not make network calls**. It is an input to execution-time selection, not a health system: `search()`/`fetch()` read it to pick a usable provider, and a selection failure surfaces as the structured `WebError` the caller routes on — which carries the branchable detail (the missing id, the ambiguous candidate set) in its code and message. ```ts type-equiv type WebProviderStatus = @@ -83,15 +83,7 @@ type WebProviderStatus = | { readonly available: false; readonly reason: 'missing-credential' | 'misconfigured' } ``` -The service aggregates provider status into a `WebCapabilityStatus`: whether the capability has a selected usable provider, or the broad category in which selection fails. It carries the winning `providerId` on the available branch but NOT the per-reason payload (the missing id, the ambiguous set) — that branchable detail lives in the thrown `WebError`, the surface callers route on, so the same fact never gets two homes that can disagree. - -```ts type-equiv -type WebCapabilityStatus = - | { readonly available: true; readonly providerId: string } - | { readonly available: false; readonly reason: 'none' | 'configured-missing' | 'configured-unavailable' | 'ambiguous' } -``` - -Selection never depends on registration, config, or HMR order: a capability has an explicit provider id (config `searchProvider`/`fetchProvider`, or the matching env var feeding the same field), or auto-selects when exactly one usable provider is registered; multiple usable providers with no configured id is `ambiguous`, not first-wins. +Selection never depends on registration, config, or HMR order: a capability has an explicit provider id (config `searchProvider`/`fetchProvider`, or the matching env var feeding the same field), or auto-selects when exactly one usable provider is registered; multiple usable providers with no configured id is `WEB_PROVIDER_AMBIGUOUS`, not first-wins. ## Errors @@ -99,4 +91,4 @@ Selection never depends on registration, config, or HMR order: a capability has ## The service -`WebService` (`ctx.web`, defined in [`packages/web/web/src/index.ts`](../../packages/web/web/src/index.ts)) is a provider registry plus a provider-selecting execution surface, close to `LlmService`'s shape: `registerSearchProvider`/`registerFetchProvider` (duplicate ids throw `WEB_DUPLICATE_PROVIDER`, return disposers, emit `web/providers-change`), `searchStatus`/`fetchStatus` (derived, never stored), and `search`/`fetch` (resolve the provider at call time, throw a structured `WebError` when the capability cannot run). Providers issue requests with the platform-native `fetch` (Node 24), mirroring `dsh-llm-deepseek`; the `dsh-web-fetch-local` provider owns safe retrieval (http/https-only, credential rejection, byte/char/timeout/redirect caps, same-origin-only redirects with per-hop re-validation, charset decoding) while `dsh-tool-web` owns presentation (HTML→markdown). SSRF / private-network blocking is deferred (see the RFC) — until it lands, `web_fetch` must not be enabled where it can reach sensitive internal targets. +`WebService` (`ctx.web`, defined in [`packages/web/web/src/index.ts`](../../packages/web/web/src/index.ts)) is a provider registry plus a provider-selecting execution surface, close to `LlmService`'s shape: `registerSearchProvider`/`registerFetchProvider` (duplicate ids throw `WEB_DUPLICATE_PROVIDER`, return disposers) and `search`/`fetch` (resolve the provider at call time, throw a structured `WebError` when the capability cannot run). Providers issue requests with the platform-native `fetch` (Node 24), mirroring `dsh-llm-deepseek`; the `dsh-web-fetch-local` provider owns safe retrieval (http/https-only, credential rejection, byte/char/timeout/redirect caps, same-origin-only redirects with per-hop re-validation, charset decoding) while `dsh-tool-web` owns presentation (HTML→markdown). SSRF / private-network blocking is deferred (see the RFC) — until it lands, `web_fetch` must not be enabled where it can reach sensitive internal targets. diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 63cedd7510..1ac69ffd90 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -11,20 +11,19 @@ This matrix shows which packages dispatch each harness-owned event and which pac | --- | --- | --- | --- | --- | | `agent/created` | `emit` | [`packages/core/agent/src/types.ts:234`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) | | `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:241`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) | -| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:389`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | +| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:380`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | | `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:319`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | | `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:332`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | | `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:259`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | | `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:345`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`), [`compact-basic`](../packages/compact/compact-basic) (`waterfall`) | - | | `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:274`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | | `agent/status` | `emit` | [`packages/core/agent/src/types.ts:250`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`stdio-agent`](../packages/ui/stdio-agent) | -| `agent/steering` | `emit` | [`packages/core/agent/src/types.ts:379`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | | `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:355`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | | `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:368`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:123`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:138`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:109`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:32`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`llm-replay`](../packages/support/llm-replay) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:33`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`llm-replay`](../packages/support/llm-replay) | | `session/created` | `emit` | [`packages/core/session/src/index.ts:36`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/event` | `emit` | [`packages/core/session/src/index.ts:44`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio-agent`](../packages/ui/stdio-agent) | | `session/flush` | `parallel` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`parallel`) | [`session-persistence`](../packages/session-persistence/session-persistence) | @@ -35,4 +34,3 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `tools/change` | `emit` | [`packages/core/tools/src/index.ts:87`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - | | `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:82`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | | `tools/pre-execute` | `waterfall` | [`packages/core/tools/src/index.ts:66`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `web/providers-change` | `emit` | [`packages/web/web/src/index.ts:65`](../packages/web/web/src/index.ts) | [`web`](../packages/web/web) (`emit`) | - | diff --git a/docs/module-graph.md b/docs/module-graph.md index c38baf0c1a..2f89e8d1c9 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -75,6 +75,7 @@ flowchart TD subgraph group_ui["packages/ui"] pkg_acp["acp"] pkg_acp_agent["acp-agent"] + pkg_app_boot["app-boot"] pkg_stdio_agent["stdio-agent"] end pkg_llm --> pkg_brand @@ -189,9 +190,11 @@ flowchart TD pkg_subagent_spawn --> pkg_subagent_inprocess pkg_acp_agent --> pkg_acp pkg_acp_agent --> pkg_agent_core + pkg_acp_agent --> pkg_app_boot pkg_acp_agent --> pkg_session_persistence_jsonl pkg_stdio_agent --> pkg_agent pkg_stdio_agent --> pkg_agent_core + pkg_stdio_agent --> pkg_app_boot pkg_stdio_agent --> pkg_llm pkg_stdio_agent --> pkg_session pkg_stdio_agent --> pkg_session_persistence_jsonl @@ -200,6 +203,7 @@ flowchart TD | Package | Group | Depends on | | --- | --- | --- | | [`brand`](../packages/util/brand) | `util` | — | +| [`app-boot`](../packages/ui/app-boot) | `ui` | — | | [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand) | | [`bash`](../packages/bash/bash) | `bash` | [`brand`](../packages/util/brand) | | [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) | @@ -241,5 +245,5 @@ flowchart TD | [`subagent-mock`](../packages/support/subagent-mock) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent) | | [`subagent-fork`](../packages/subagent/subagent-fork) | `subagent` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | | [`subagent-spawn`](../packages/subagent/subagent-spawn) | `subagent` | [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | -| [`acp-agent`](../packages/ui/acp-agent) | `ui` | [`acp`](../packages/ui/acp), [`agent-core`](../packages/core/agent-core), [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | -| [`stdio-agent`](../packages/ui/stdio-agent) | `ui` | [`agent`](../packages/core/agent), [`agent-core`](../packages/core/agent-core), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | +| [`acp-agent`](../packages/ui/acp-agent) | `ui` | [`acp`](../packages/ui/acp), [`agent-core`](../packages/core/agent-core), [`app-boot`](../packages/ui/app-boot), [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | +| [`stdio-agent`](../packages/ui/stdio-agent) | `ui` | [`agent`](../packages/core/agent), [`agent-core`](../packages/core/agent-core), [`app-boot`](../packages/ui/app-boot), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | diff --git a/docs/persistence-catalog/log-events.md b/docs/persistence-catalog/log-events.md new file mode 100644 index 0000000000..5bca109a76 --- /dev/null +++ b/docs/persistence-catalog/log-events.md @@ -0,0 +1,240 @@ + + +# Persistence Log Event Catalog + +Every event type that can appear in a session's durable event log: each member of the merge-extensible `SessionEventMap` — the owning vocabulary in `@deepseek-ai/dsh-session` plus every plugin declaration merge in this repo — with the payload it carries, its surface badge, and the declaration it comes from. It complements [session.md](../core-data-structures/session.md) (the `SessionEvent` envelope, surface list, and `deriveMessages()` projection), [persistence.md](../core-data-structures/persistence.md) (how the log is made durable), and the [cordis catalog](../cordis-catalog/events-and-services.md) (the live bus wiring — a log event is NOT a cordis event; it reaches listeners via the single `session/event` emit). + +This file is GENERATED from source (`scripts/gen-persistence-catalog.ts`) and verified fresh by `pnpm run verify-persistence-catalog` (part of `doc-sync`) — do not edit it by hand. Payload blocks use a `ts persistence-catalog` fence (skipped by doc-typecheck, since a bare payload fragment is not standalone-compilable). Type names in a payload link to the page that documents them. See [the persistence-log-catalog RFC](../rfc/implemented/process/2026-07-04-persistence-log-catalog.md). + +The on-disk envelope around every payload is `SessionEvent` — `type`, monotonic `seq`, epoch-ms `time`, the `data` documented here, plus `surfaceOp`/`sourceEventSeqs` on **surface** events only ([envelope](../core-data-structures/session.md#sessioneventt--one-log-entry)). **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`), and the whole format is pinned at `SESSION_FORMAT_VERSION = 0` — pre-release, no compatibility implied ([the version stance](../core-data-structures/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further event types, which are outside this catalog by construction. + +## Events + +### `assistant/*` + +#### `assistant/chunk` — log-only + +Raw stream chunk — token-level replay fidelity. + +```ts persistence-catalog +'assistant/chunk': { turn: number; step: number; chunk: StreamChunk } +``` + +Types: [StreamChunk](../core-data-structures/llm-streaming.md) + +Source: [`packages/core/session/src/types.ts:237`](../../packages/core/session/src/types.ts) + +#### `assistant/message` — surface + +Assembled assistant message for one step (derived history uses this). Carries the step's `usage` when the adapter reported token accounting, so the model output and its accounting travel together (there is no separate usage record). `usage` is absent when the adapter reported none. + +```ts persistence-catalog +'assistant/message': { turn: number; step: number; content: ContentBlock[]; usage?: TokenUsage } +``` + +Types: [ContentBlock](../core-data-structures/core.md) · [TokenUsage](../core-data-structures/llm-streaming.md) + +Source: [`packages/core/session/src/types.ts:244`](../../packages/core/session/src/types.ts) + +### `compact/*` + +#### `compact/end` — log-only + +Marks the end of a compaction — log-only, releases the lock. `error` set if summarization failed. + +```ts persistence-catalog +'compact/end': { turn: number; error?: string } +``` + +Source: [`packages/compact/compact/src/types.ts:37`](../../packages/compact/compact/src/types.ts) + +#### `compact/start` — log-only + +Marks the start of a compaction — log-only, holds the lock until `compact/end`. + +```ts persistence-catalog +'compact/start': { turn: number } +``` + +Source: [`packages/compact/compact/src/types.ts:23`](../../packages/compact/compact/src/types.ts) + +#### `compact/summary` — log-only + +Provenance record of a completed summarization — log-only, no surfaceOp. The summary content is in `data.summary`; the actual surface replacement is performed by a subsequent `user/message` event that shadows the compacted range. + +```ts persistence-catalog +'compact/summary': { summary: ContentBlock[]; shadowedRange: { start: number; end: number }; shadowedSeqs: number[]; shadowedTokenCount: number } +``` + +Types: [ContentBlock](../core-data-structures/core.md) + +Source: [`packages/compact/compact/src/types.ts:30`](../../packages/compact/compact/src/types.ts) + +### `context/*` + +#### `context/message` — surface + +In-session context injection (file-change notices, subdir AGENTS.md, skill content, cron notifications, …). Rendered into the derived history as tagged synthetic context — NOT a user prompt. + +```ts persistence-catalog +'context/message': { content: ContentBlock[]; source: MessageSource } +``` + +Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) + +Source: [`packages/core/session/src/types.ts:235`](../../packages/core/session/src/types.ts) + +### `hook/*` + +#### `hook/invoked` — log-only + +A hook command was invoked at a hook point — log-only provenance (like `compact/*`; NOT a SurfaceEventType, carries no `surfaceOp`). `dialect` is the bridge that ran it (`claude`/`codex`), `point` the hook point (`PreToolUse`, `Stop`, …), `matcher` the matcher-group pattern that selected it (absent for match-all), `handlerId` a stable id for the command (so an invoked/result pair correlates). `turn` is the open turn the invocation lives inside. + +```ts persistence-catalog +'hook/invoked': { turn: number; point: string; dialect: HookDialect; matcher?: string; handlerId: string } +``` + +Source: [`packages/hooks/hook-protocol/src/types.ts:27`](../../packages/hooks/hook-protocol/src/types.ts) + +#### `hook/result` — log-only + +A hook command's outcome — log-only, paired with a prior `hook/invoked` (same `handlerId`). `decision` is the dialect-neutral outcome derived by `appendHookResult` (which owns the rule): the hook's parsed decision (`approve`/`allow`/`block`/`deny`/`ask`), else `'stop'` when it asked to halt via `continue:false`, else `'pass'`. `exitCode` is the process exit (absent if it never ran), `stderrSummary` the trimmed stderr truncated to the bridge's configured cap (the block reason source on exit 2), `durationMs` the wall-clock runtime (audit timing; snapshot replay normalizes it). `turn` matches the `hook/invoked`. + +```ts persistence-catalog +'hook/result': { turn: number; point: string; handlerId: string; decision: string; exitCode?: number; stderrSummary?: string; durationMs: number } +``` + +Source: [`packages/hooks/hook-protocol/src/types.ts:45`](../../packages/hooks/hook-protocol/src/types.ts) + +### `prompt/*` + +#### `prompt/blocked` — log-only + +A queued prompt an `agent/prompt-submit` listener VETOED — the durable record of a blocked prompt and why. Appended in place of the `user/message` the prompt would have become, so the block survives replay even in a MIXED batch where another queued prompt is allowed (there the turn does not end `rejected`, so the boundary reason alone would not preserve it). `content` is the original prompt the listener rejected; `reason` is the veto text (PromptDecision `block.reason`). NOT a SurfaceEventType: a blocked prompt produces no LLM message and never reaches `deriveMessages()`. + +```ts persistence-catalog +'prompt/blocked': { content: ContentBlock[]; source: MessageSource; reason: string } +``` + +Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) + +Source: [`packages/core/session/src/types.ts:229`](../../packages/core/session/src/types.ts) + +### `steering/*` + +#### `steering/message` — surface + +Steering content injected between steps of a running turn. + +```ts persistence-catalog +'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource } +``` + +Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) + +Source: [`packages/core/session/src/types.ts:262`](../../packages/core/session/src/types.ts) + +### `step/*` + +#### `step/end` — log-only + +Closes step `step` of turn `turn`. + +```ts persistence-catalog +'step/end': { turn: number; step: number } +``` + +Source: [`packages/core/session/src/types.ts:216`](../../packages/core/session/src/types.ts) + +#### `step/start` — log-only + +Opens step `step` of turn `turn` — one model call plus the tool executions it requested. + +```ts persistence-catalog +'step/start': { turn: number; step: number } +``` + +Source: [`packages/core/session/src/types.ts:214`](../../packages/core/session/src/types.ts) + +### `todo/*` + +#### `todo/write` — log-only + +The agent's whole todo list, carried as a full snapshot and replaced wholesale on each write — the current list is the most recent `todo/write` (last-write-wins on replay, no fold). Appended by an owning agent via `session.append('todo/write', { todos })`. + +NOT a SurfaceEventType: it produces no LLM message and never reaches `deriveMessages()`, so it carries no `surfaceOp` and stays off the surface — it is durable, replayable UI state, distinct from the conversation history. It is a `SessionEventMap` member riding the existing `session/event` emit, not a first-class Cordis `interface Events` notification, so it has no cordis-catalog row. + +```ts persistence-catalog +'todo/write': { todos: TodoItem[] } +``` + +Types: [TodoItem](../core-data-structures/session.md) + +Source: [`packages/core/session/src/types.ts:276`](../../packages/core/session/src/types.ts) + +### `tool/*` + +#### `tool/call` — log-only + +The model requested one tool invocation: `name` with the raw `arguments` JSON string exactly as the model produced it (unparsed). `callId` pairs the call with its `tool/result`. + +```ts persistence-catalog +'tool/call': { turn: number; step: number; callId: CallId; name: string; arguments: string } +``` + +Types: [CallId](../core-data-structures/core.md) + +Source: [`packages/core/session/src/types.ts:250`](../../packages/core/session/src/types.ts) + +#### `tool/result` — surface + +A completed tool call's model-facing result, plus an optional tool-private `meta` presentation payload. `meta` is opaque to the core (`unknown` — the producing tool owns its shape and reads it back in `presentResult`) but MUST be JSON-serializable: `Session.append` runtime-validates all event data with `isJsonValue`, so a non-serializable `meta` is rejected at the source, and the durable log reproduces the identical card on replay. Absent unless the tool attaches one (e.g. `dsh-tool-fs` carries its result-time contextual diff here). + +```ts persistence-catalog +'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string }; meta?: unknown } +``` + +Types: [CallId](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) + +Source: [`packages/core/session/src/types.ts:260`](../../packages/core/session/src/types.ts) + +### `turn/*` + +#### `turn/end` — log-only + +Closes turn `turn` with the TurnEndReason that ended it. The loop fires the awaited `session/flush` checkpoint at every turn end, so the turn boundary is also the durable-commit boundary. + +```ts persistence-catalog +'turn/end': { turn: number; reason: TurnEndReason } +``` + +Types: [TurnEndReason](../core-data-structures/session.md) + +Source: [`packages/core/session/src/types.ts:212`](../../packages/core/session/src/types.ts) + +#### `turn/start` — log-only + +Opens turn `turn`. `trigger` records what started it — a drained message batch or an idle-time injection. The turn is the durability/replay boundary: every event sits between a `turn/start` and its matching `turn/end` (the turn-enclosure invariant). + +```ts persistence-catalog +'turn/start': { turn: number; trigger: TurnTrigger } +``` + +Types: [TurnTrigger](../core-data-structures/session.md) + +Source: [`packages/core/session/src/types.ts:206`](../../packages/core/session/src/types.ts) + +### `user/*` + +#### `user/message` — surface + +A user-visible prompt (queued message drained at turn start). + +```ts persistence-catalog +'user/message': { content: ContentBlock[]; source: MessageSource } +``` + +Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md) + +Source: [`packages/core/session/src/types.ts:218`](../../packages/core/session/src/types.ts) diff --git a/docs/rfc/README.md b/docs/rfc/README.md index 1b5a3d350e..9633691cf7 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -52,16 +52,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | Title | First proposed | |---|---| | [Unify the agent id and the session id](proposed/simplification/2026-06-20-unify-agent-and-session-id.md) | 2026-06-20 | -| [Drop the `image` content block until a path can honor it](proposed/simplification/2026-07-04-drop-image-content-block.md) | 2026-07-04 | -| [Drop `GenerateOptions.prefill` and `ToolSchema.strict` — request knobs with no working end-to-end path](proposed/simplification/2026-07-04-drop-inert-request-knobs.md) | 2026-07-04 | -| [Drop the unconsumed web observation surface — the `providers-change` event and the status methods](proposed/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md) | 2026-07-04 | | [Prune dead core-spine surface — `SurfaceManager.invalidate()`, the loop-internal exports, `ToolExecutionResult.callId`](proposed/simplification/2026-07-04-prune-dead-core-spine-surface.md) | 2026-07-04 | -| [Prune producer-less vocabulary variants (block cache hints, the `agent` message source, the `continuation` turn trigger)](proposed/simplification/2026-07-04-prune-producerless-vocabulary-variants.md) | 2026-07-04 | -| [Prune write-only fields and a dead routing knob from the fs seam](proposed/simplification/2026-07-04-prune-write-only-fs-surface.md) | 2026-07-04 | -| [Remove the `agent/steering` mirror emit](proposed/simplification/2026-07-04-remove-agent-steering-mirror.md) | 2026-07-04 | -| [Share the app bins' boot glue instead of maintaining twin copies](proposed/simplification/2026-07-04-share-app-bin-boot-glue.md) | 2026-07-04 | -| [Tighten the hook-protocol contract — dialect, discarded fields, double defaults, and lib-owned `hook/result` semantics](proposed/simplification/2026-07-04-tighten-hook-protocol-contract.md) | 2026-07-04 | -| [Trim unreachable ACP bridge surface — the branding knobs and the kind-sniffing fallback](proposed/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md) | 2026-07-04 | ### Architecture @@ -85,7 +76,6 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r |---|---| | [Deterministic tests, the replay invariant fixture, and race stress](proposed/testing/2026-06-11-deterministic-and-stress-testing.md) | 2026-06-11 | | [Mutation testing as the coverage counterweight](proposed/testing/2026-06-11-mutation-testing.md) | 2026-06-11 | -| [Single-source the acp-agent replay config](proposed/testing/2026-07-04-single-source-acp-replay-config.md) | 2026-07-04 | ## Implemented @@ -119,7 +109,16 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Stop mirroring durable boundaries as agent events](implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md) | 2026-06-20 | | [Split the filesystem seam — provider text mutations plus the `dsh-fs-policy` plugin](implemented/simplification/2026-06-26-fsspec-style-fs-seam.md) | 2026-06-26 | | [Stop mirroring the token stream as an agent event](implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md) | 2026-07-02 | +| [Drop the `image` content block until a path can honor it](implemented/simplification/2026-07-04-drop-image-content-block.md) | 2026-07-04 | +| [Drop `GenerateOptions.prefill` and `ToolSchema.strict` — request knobs with no working end-to-end path](implemented/simplification/2026-07-04-drop-inert-request-knobs.md) | 2026-07-04 | +| [Drop the unconsumed web observation surface — the `providers-change` event and the status methods](implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md) | 2026-07-04 | | [Fold the stdio UI helper into the stdio app](implemented/simplification/2026-07-04-fold-stdio-ui-helper.md) | 2026-07-04 | +| [Prune producer-less vocabulary variants (block cache hints, the `agent` message source, the `continuation` turn trigger)](implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md) | 2026-07-04 | +| [Prune write-only fields and a dead routing knob from the fs seam](implemented/simplification/2026-07-04-prune-write-only-fs-surface.md) | 2026-07-04 | +| [Remove the `agent/steering` mirror emit](implemented/simplification/2026-07-04-remove-agent-steering-mirror.md) | 2026-07-04 | +| [Share the app bins' boot glue instead of maintaining twin copies](implemented/simplification/2026-07-04-share-app-bin-boot-glue.md) | 2026-07-04 | +| [Tighten the hook-protocol contract — dialect, discarded fields, double defaults, and lib-owned `hook/result` semantics](implemented/simplification/2026-07-04-tighten-hook-protocol-contract.md) | 2026-07-04 | +| [Trim unreachable ACP bridge surface — the branding knobs and the kind-sniffing fallback](implemented/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md) | 2026-07-04 | ### Architecture @@ -144,6 +143,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Branded IDs everywhere they belong](implemented/architecture/2026-06-20-branded-ids.md) | 2026-06-20 | | [Extract example apps into packages](implemented/architecture/2026-06-20-extract-example-app-packages.md) | 2026-06-20 | | [Reorganize packages into a modular hierarchy](implemented/architecture/2026-06-20-package-hierarchy.md) | 2026-06-20 | +| [Mandatory `User-Agent` attribution for provider requests](implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md) | 2026-06-21 | | [Web capability seam - stable tools over multiple providers](implemented/architecture/2026-06-24-web-capability-seam.md) | 2026-06-24 | | [Make `dsh-fs-policy` an event-gate plugin, not a method interface](implemented/architecture/2026-06-26-file-context-as-event-gate.md) | 2026-06-26 | | [stdin + extra env on the bash seam](implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) | 2026-06-30 | @@ -173,6 +173,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [JSDoc completeness gate for the cordis surface](implemented/process/2026-07-04-cordis-jsdoc-completeness-gate.md) | 2026-07-04 | | [Documentation tiers, budgets, and the ceiling gate](implemented/process/2026-07-04-doc-tiers-and-budgets.md) | 2026-07-04 | | [Generate the RFC index tables](implemented/process/2026-07-04-generate-rfc-index-tables.md) | 2026-07-04 | +| [Generated persistence log event catalog](implemented/process/2026-07-04-persistence-log-catalog.md) | 2026-07-04 | ### Testing @@ -186,6 +187,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Record fork and mixed spawn+fork snapshot scenarios](implemented/testing/2026-06-22-fork-snapshot-scenarios.md) | 2026-06-22 | | [Per-session snapshot replay for nested agents](implemented/testing/2026-06-22-subagent-snapshot-replay.md) | 2026-06-22 | | [Hook snapshot matrix — end-to-end goldens for both bridges](implemented/testing/2026-07-04-hook-snapshot-matrix.md) | 2026-07-04 | +| [Single-source the acp-agent replay config](implemented/testing/2026-07-04-single-source-acp-replay-config.md) | 2026-07-04 | ## Rejected diff --git a/docs/rfc/implemented/architecture/2026-06-11-content-block-vocabulary.md b/docs/rfc/implemented/architecture/2026-06-11-content-block-vocabulary.md index 49d55badac..bc6d71e6e9 100644 --- a/docs/rfc/implemented/architecture/2026-06-11-content-block-vocabulary.md +++ b/docs/rfc/implemented/architecture/2026-06-11-content-block-vocabulary.md @@ -10,12 +10,12 @@ The harness needs one internal language for messages that the loop, session log, ## Decision -Own it: messages are arrays of typed content blocks (`text`, `reasoning`, `tool-call`, `tool-result`, `image`), with the union derived from the merge-extensible `ContentBlockMap` so plugins add block types via declaration merging. The same merge-extensible-map pattern types every "stringly" field (`MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`). Streaming is a raw chunk protocol; `BlockAssembler` is the single shared assembly implementation. Adapters translate to provider wire formats — mapping cost lives in adapters, where it belongs. +Own it: messages are arrays of typed content blocks (`text`, `reasoning`, `tool-call`, `tool-result`), with the union derived from the merge-extensible `ContentBlockMap` so plugins add block types via declaration merging. The same merge-extensible-map pattern types every "stringly" field (`MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`). Streaming is a raw chunk protocol; `BlockAssembler` is the single shared assembly implementation. Adapters translate to provider wire formats — mapping cost lives in adapters, where it belongs. In-session context injection (`context/message`, `steering/message`) renders as tagged user-role envelopes (the system-reminder pattern) rather than a new role, so adapters carry zero burden. Live-adapter review has since validated the tagged-envelope rendering against current DeepSeek behavior; a future provider-specific mismatch should be handled in that adapter rather than by adding a new role to the canonical content vocabulary. ## Consequences -- Reasoning, prefill, cache hints, and multimodal content all have a home without provider contortions. +- Reasoning has a home without provider contortions. Multimodal content deliberately has NO core block type: the core set is limited to blocks every shipping path honors, and a multimodal feature adds its block type through the merge-extensible map in the same coordinated change that maps it in the adapters, surfaces it in the UI bridges, and prices it in compaction — see [the drop-image RFC](../simplification/2026-07-04-drop-image-content-block.md). Block cache hints likewise have no core field: DeepSeek prompt caching is automatic, so no shipping adapter can transmit a hint; a caching feature adds a `cache` field together with the adapter that honors it — see [the producer-less-variants RFC](../simplification/2026-07-04-prune-producerless-vocabulary-variants.md). Assistant-prefix continuation (prefill) likewise has no request field: DeepSeek's chat-prefix completion is a Beta feature on a base URL neither shipping adapter targets, so a prefill feature adds `GenerateOptions.prefill` together with the adapter that honors it — see [the inert-request-knobs RFC](../simplification/2026-07-04-drop-inert-request-knobs.md). - Every adapter pays a translation cost; the first real adapters have since validated the streaming protocol, and new adapters should continue proving their provider-specific mapping in adapter-local tests. - IDs that cross package boundaries are branded (`CallId`, `SessionId`, `AgentId`) — nominal typing at zero runtime cost. diff --git a/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md b/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md index 02517d7322..380586699b 100644 --- a/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md +++ b/docs/rfc/implemented/architecture/2026-06-20-extract-example-app-packages.md @@ -14,7 +14,7 @@ Each example is now **mostly an invocation of an app package**, splitting the wi - **`@deepseek-ai/dsh-agent-core`** ([packages/core/agent-core](../../../../packages/core/agent-core)) — a Cordis bundle plugin for the providerless, executor-less, UI-less spine: `timer` + `llm` + sessions + system-prompt + tools + agents + invariants + `tool-bash` + `agent-loop`, mounted as child plugins inside its `apply(ctx)` via `ctx.plugin(...)`. This is the old `base-core.yml` **minus** `bash-local`, **plus** `timer` and the loop, as code instead of a YAML include. The bundle **forwards** `agent-loop`'s `agents` list as its own config (`export const Config = AgentLoop.Config`, default `[]`, the existing `AgentLoop.Config` shape in [packages/core/agent-loop/src/index.ts](../../../../packages/core/agent-loop/src/index.ts)) — so each app supplies its own pre-created agents. This is precisely the reason the old `base-core.yml` gave for keeping `agent-loop` *out* of the shared core ("the examples disagree — stdio needs a pre-created `main`, acp needs none"); forwarding the config dissolves that objection — the loop is shared, the agents list is per-app. The bundle children register into the root service store, so a leaf-mounted sibling (the adapter, the executor) sees them exactly as a nested `plugin-include` subtree's services were seen before. Depending on the CONCRETE `dsh-agent-loop` (not just the `dsh-agent` interface) is deliberate and is the sanctioned exception to the "extension plugins depend on interfaces, never on the concrete loop" rule (packages/README.md, docs/architecture.md § Layering): the rule constrains plugins that EXTEND the system, whereas this bundle's whole job is to COMPOSE the concrete spine. Swapping the loop means publishing a different bundle, not rewiring every extension. - **`@deepseek-ai/dsh-stdio-agent`** ([packages/ui/stdio-agent](../../../../packages/ui/stdio-agent)) and **`@deepseek-ai/dsh-acp-agent`** ([packages/ui/acp-agent](../../../../packages/ui/acp-agent)) — app packages, each consuming `dsh-agent-core` and **baking in its coupled front-door cluster**: stdio = `ui-stdio` + console logger + a pre-created `main`; acp = the `acp` bridge + JSONL persistence + **no stdout logger** + no pre-created agents. The leaf no longer carries the cluster, so it has no logger entry to copy wrong by default — the common stdout-purity mistake loses its foothold. (A leaf can still *add* a sibling logger entry — a package cannot forbid what a leaf author writes — so the rule "never add a stdout logger to an ACP leaf" stays documented at the leaf; what changed is that the default leaf has nothing to get wrong.) They land under the existing `ui` group alongside `acp`, so no new package group (and no `tsconfig`/`packages/README` group plumbing) was needed. -- **`start.ts` is gone.** Each app package exposes a `bin` (`dsh-stdio-agent` / `dsh-acp-agent`); the `demo:*` scripts invoke it (e.g. `dsh-stdio-agent ./cordis.yml`). The Loader-boot tail, `.env` loading, snapshot-mode selection, and stdin-dispose lifecycle moved into that bin, owned by the app. The `bin.ts` files are coverage-excluded (a self-executing CLI entry, like the old `start.ts`) and driven by the keyless Loader-path tests. +- **`start.ts` is gone.** Each app package exposes a `bin` (`dsh-stdio-agent` / `dsh-acp-agent`); the `demo:*` scripts invoke it (e.g. `dsh-stdio-agent ./cordis.yml`). The Loader-boot tail, `.env` loading, and fail-loud guards live in the shared [`@deepseek-ai/dsh-app-boot`](../../../../packages/ui/app-boot) package (unit-tested under the per-file coverage gate — see [share the app bins' boot glue](../simplification/2026-07-04-share-app-bin-boot-glue.md)); each bin is a thin self-executing composition over those helpers plus its app-specific lifecycle (the ACP bin: snapshot-mode selection and stdin-dispose). The `bin.ts` files themselves stay coverage-excluded (self-executing CLI entries, like the old `start.ts`) and are driven by the keyless Loader-path tests. - **Each leaf `cordis.yml` collapses** to backends + config: the LLM adapter (`llm-deepseek` with apiKey/models, or `llm-replay`), the bash executor (`bash-local`), `hmr` for the stdio demos (see the amendment below), and one app entry carrying the app's config (model, system prompt, persistence root — surfaced as the app package's own `Config`, which routes each value to wherever the app wires it: stdio onto its pre-created agent, acp onto the bridge plugin). - **echo-agent folds onto `dsh-stdio-agent`**, swapping the LLM backend to the local `mock-llm` and adding the local `echo-tool` (plus `bash-local`, which the spine's `tool-bash` injects) at the leaf — the clean demonstration of "swap the backend, keep the app". `mock-llm.ts` / `echo-tool.ts` stay as example-local teaching plugins. - **`base.yml`, `base-core.yml`, and `acp-agent/acp-tail.yml` are retired** — the spine they shared now lives in `dsh-agent-core`. diff --git a/docs/rfc/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md b/docs/rfc/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md new file mode 100644 index 0000000000..26ca40c647 --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md @@ -0,0 +1,80 @@ +# RFC: Mandatory `User-Agent` attribution for provider requests + +Status: implemented + +## Problem + +LLM provider requests should identify the product making them. That is useful for provider-side support, abuse investigation, compatibility debugging, and traffic analytics. Before this RFC the harness only partially did this: the hand-rolled DeepSeek adapter sent a hand-copied `User-Agent` constant (`packages/llm/llm-deepseek/src/adapter.ts`), while the pi-ai-backed twin sent no harness-owned headers at all (`packages/llm/llm-pi-ai/src/adapter.ts`). New adapters could therefore omit attribution silently, and a library-backed adapter could drift from the hand-rolled adapter even though [the twin-adapter RFC](2026-06-13-twin-llm-adapters.md) exists to keep the provider seam honest across both implementations. + +The immediate prompt came from OpenRouter's [App Attribution](https://openrouter.ai/docs/app-attribution) docs. OpenRouter creates app pages and rankings from `HTTP-Referer` plus display/category headers. That is valuable, but it is not the HTTP standard for application identity. The risk is adopting OpenRouter's exact header set as if it were universal, then leaking provider-specific headers to direct DeepSeek requests, future OpenAI/Anthropic/Vertex adapters, test servers, or proxies that log unknown fields indefinitely. + +## Investigation + +- **OpenRouter's mechanism is provider-specific.** Their current docs say app attribution is tracked through `HTTP-Referer` (required), `X-OpenRouter-Title`, and `X-OpenRouter-Categories`; `X-Title` is only accepted for backward compatibility. Their API reference calls the headers optional and says they make the app discoverable on OpenRouter. This is a concrete OpenRouter contract, not an IETF or OpenAI-compatible API standard. +- **In agent tooling, `HTTP-Referer` is an OpenRouter-aware convention, not a general agent convention.** It is common enough that OpenRouter SDKs and OpenRouter examples expose it directly, and frameworks that target OpenRouter usually need a way to pass it through. But agent protocols such as ACP negotiate names, versions, and capabilities in their own initialize messages, while model-provider requests still need HTTP-level identity. "Accepted in the agent world" therefore means "recognized by OpenRouter integrations," not "portable across agent runtimes or providers." +- **Observed coding agents use product/version `User-Agent` strings, sometimes with environment context.** A non-exhaustive public-code survey found OpenAI Codex building `{originator}/{version} ({os} {os_version}; {arch}) ...` and carrying an `originator` header; Google Gemini CLI sending `GeminiCLI[-clientName]/{version}/{model} ({platform}; {arch}; {surface})` or a Cloud Code VS Code variant; Cline's Codex backend client sending `cline/{version} ({platform} {release}; {arch}) node/{nodeVersion}` plus `originator: cline`; SWE-agent setting `swe-agent/{version}` unless the user already supplied a header; Continue setting `Continue/{version}` for its ClawRouter provider plus `X-Continue-Provider`. Aider also appends `Aider/{version} +{website}` to browser-like user agents for web scraping, but that is not a model-provider request path. The pattern is not one exact format; it is product identity in `User-Agent`, with provider-specific side headers only where a provider/backend asks for them. +- **The standards-track general client identity header is `User-Agent`.** RFC 9110 section 10.1.5 defines `User-Agent` as the user-agent software identity, says it is used for interoperability reports and analytics, and says a user agent SHOULD send it on each request unless configured not to. This is the only standard header that directly matches "what product is making this HTTP request." +- **`Referer` is standard, but OpenRouter's `HTTP-Referer` is not the standard field.** RFC 9110 section 10.1.3 defines `Referer` as the URI from which the target URI was obtained and spends significant text on privacy restrictions. OpenRouter instead asks for `HTTP-Referer`, using it as an app URL identifier. That name and meaning are OpenRouter-specific even though it resembles the CGI environment variable form of the standard `Referer` header. +- **`From` is standard but not suitable as a mandatory default.** RFC 9110 section 10.1.2 defines `From` as an email address for the human responsible for a user agent. Robotic agents SHOULD send it so servers can contact an operator, but non-robotic agents should not send it without explicit user configuration because of privacy and security policy concerns. The harness can support an operator contact later, but must not invent one or require it globally. +- **Request-body `user` or `metadata` fields are not app attribution.** Some model APIs expose a stable end-user identifier, request metadata, labels, or project/account headers. Those are useful for abuse monitoring, internal billing, dashboards, or trace correlation, but they either identify the end user rather than the product, are provider-specific body schema, or are not guaranteed to be forwarded through OpenAI-compatible gateways. They are not a substitute for a static application identity header. +- **SDK telemetry headers identify the SDK, not the app.** Official and third-party SDKs often send library/version headers. Those help the SDK maintainer debug their client, but they do not identify the harness as the application unless the application explicitly supplies a product attribution layer. +- **pi-ai has a first-class header hook.** `@earendil-works/pi-ai`'s `StreamOptions.headers` merges caller headers last over provider defaults, so a library-backed adapter can satisfy the same wire contract as the hand-rolled one without wrapping or upstream work. The mock-server suites assert arrival on the wire for both adapters. + +## Decision + +Provider request attribution is mandatory at the LLM adapter boundary, using the standard `User-Agent` header only. The rule: every product LLM adapter sends a static, non-secret application identity on every provider HTTP request, and every adapter has tests proving that `User-Agent` reaches the wire (a mock server asserting received headers; for a library-backed adapter, the library's header hook feeding the same mock-server assertion). + +Do **not** implement OpenRouter app attribution in this RFC. `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, and `X-OpenRouter-Categories` are OpenRouter-specific product-surface headers, not provider-neutral model-request attribution. They can be proposed later by an OpenRouter adapter or explicit OpenRouter mode, with its own privacy/product decision, tests, and docs. Until then, even requests pointed at OpenRouter send only the shared `User-Agent` attribution from this RFC. + +The provider-neutral identity is owned by `dsh-llm` (`packages/llm/llm/src/attribution.ts`), not by individual adapters. `AppIdentity` contains only public product facts needed to build `User-Agent`, and the default `APP_IDENTITY` settles the values the proposal left open: + +- product token for `User-Agent`: `deepseek-harness` (continuity with the pre-RFC wire value and the repo/org identity) +- version: read from the owning package's manifest via `createRequire`, never a hand-copied constant +- app URL: `https://github.com/deepseek-ai/deepseek-harness-sdk` - the planned public home; a `FIXME` in `attribution.ts` blocks release until that repository actually exists + +The default is mandatory and non-empty. White-label deployments pass their own `AppIdentity` to `attributionHeaders(identity)` - the override seam is the function parameter, with no deployment config plumbing until a consumer needs it - and omission falls back to the harness default rather than suppressing attribution. There is no per-request API for the model, user prompt, session id, cwd, user email, API key owner, or local machine identity to influence these fields. + +Wire mapping (`attributionHeaders`; header names lowercase in code - HTTP field names are case-insensitive on the wire): + +| Target | Mapping | +|---|---| +| All HTTP-based adapters | `User-Agent: {product}/{version} (+{url})` - the parenthesized `+url` comment stays within RFC 9110's conservative product/comment syntax. | +| Direct DeepSeek endpoint | `User-Agent`; do not send OpenRouter-only headers unless DeepSeek documents an equivalent contract. | +| OpenRouter endpoints | `User-Agent` only for now. Do not send `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, or `X-OpenRouter-Categories` under this RFC. | +| Future providers | `User-Agent` only unless a later provider-specific RFC accepts additional headers. Do not reuse `HTTP-Referer` by analogy. | + +Endpoint detection is not part of this RFC because no endpoint-specific mapping is accepted here. If OpenRouter support lands later, detection must be explicit: either a dedicated OpenRouter provider package or an explicit `provider: 'openrouter'` / `attributionTarget: 'openrouter'` config, not arbitrary path fragments or model names. + +## Acceptance criteria (all landed) + +- `dsh-llm` documents the mandatory `User-Agent` attribution contract for `LlmAdapter` authors (`LlmAdapter` JSDoc, package README, and the adapter-contract section of `docs/core-data-structures/llm-streaming.md`). +- A shared helper (`attributionHeaders` / `userAgent`) constructs the app identity and the standard `User-Agent` value from package metadata, so adapters do not hand-copy version constants. +- `dsh-llm-deepseek` sends the shared `User-Agent` on every request and its mock-server suite asserts the exact value. +- `dsh-llm-pi-ai` sends the same `User-Agent` through pi-ai's `StreamOptions.headers` hook and its mock-server suite asserts the exact value. +- No adapter sends OpenRouter-specific attribution headers (`HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, `X-OpenRouter-Categories`) as part of this RFC. +- No app-attribution field carries secrets, local paths, session ids, prompt text, model output, user email, or per-user stable identifiers. +- The adapter READMEs state the `User-Agent` attribution policy and explicitly avoid documenting OpenRouter app attribution as implemented behavior. + +## Alternatives considered + +**OpenRouter app attribution now.** Rejected for this RFC. Sending `HTTP-Referer` plus `X-OpenRouter-Title` would satisfy OpenRouter rankings, but those headers are a provider-specific product feature, not the provider-neutral model-request attribution this RFC is trying to standardize. Supporting them should be an explicit OpenRouter adapter/mode decision later, not hidden inside the first shared attribution helper. + +**OpenRouter headers everywhere.** Rejected. It would treat a custom OpenRouter contract as a universal standard and send fields with misleading semantics to providers that did not ask for them. It also risks using `HTTP-Referer` as a generic app URL field even though standard HTTP already has `User-Agent` for product identity and `Referer` for a different browsing-context concept. + +**Only provider account/project identity.** Rejected. Organization/project headers, API keys, cloud accounts, and billing projects identify who pays or owns the request, not which application is sending traffic. They also expose no public app title/category and do not help gateways like OpenRouter build app rankings. + +**End-user `user`/`metadata` fields.** Rejected for this RFC. Those are valuable for abuse monitoring and customer support but describe the human or tenant behind a request. App attribution must be static product identity and safe to send on every request. + +**Config-only opt-in attribution.** Rejected. A default-off setting is exactly how adapters keep drifting. The policy is mandatory default attribution with overrideable public values, not optional attribution. + +**Product-named token (`deepseek-code`).** Considered for the `User-Agent` token, since the product's name is DeepSeek Code. `deepseek-harness` won on continuity: it is the identity providers already see from this codebase, it matches the org/repo and planned SDK-repo naming, and a public rename can change the product token deliberately later. + +## Risks / what we give up + +**Providers see that traffic comes from the harness.** That is the point, but it means deployments that previously blended into generic SDK traffic become identifiable. Mitigation: send only static public product data and let forks/white-label deployments pass their own `AppIdentity`. + +**The app URL points at a repository that does not exist yet.** `deepseek-ai/deepseek-harness-sdk` is the planned public home; until it is created the URL is a dangling promise. The `FIXME` marker on the constant blocks a release from shipping with it unresolved (see `docs/development.md` marker semantics). + +**Header support differs by client library.** The hand-rolled adapter sets headers directly; the pi-ai-backed adapter depends on pi-ai continuing to honor `StreamOptions.headers` (merged last over provider defaults). The wire-level mock-server tests are the guard: if a pi-ai upgrade stops delivering the header, the suite goes red. This is useful pressure on the abstraction: a provider adapter that cannot set mandatory headers cannot fully implement the harness LLM contract. + +**OpenRouter rankings do not benefit yet.** `User-Agent` is the correct baseline for provider-neutral HTTP identity, but it will not create OpenRouter app pages or rankings because OpenRouter requires `HTTP-Referer` for that product feature. That is deliberate: public app marketplace participation is a separate product decision, not a prerequisite for mandatory request attribution. diff --git a/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md b/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md index 833cbeebb5..6dc07221fa 100644 --- a/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md +++ b/docs/rfc/implemented/architecture/2026-06-24-web-capability-seam.md @@ -22,7 +22,7 @@ Introduce web access as a first-class capability seam following [the capability- Providers do not register tools. Providers register capabilities. `dsh-tool-web` is the only owner of model-facing names, descriptions, prompt guidance, JSON schemas, and presentation. -Search and fetch are separate capabilities and separate model-facing tools, but they are deliberately one seam. `ctx.web` is a single web-access middle layer between provider packages on one side and the tool consumer on the other: one service to inject, one provider-selection policy owner, one abort/error vocabulary, one place a product configures "how this harness reaches the web." The two halves do not share a request schema and have no shared business logic — search normalizes provider-backed discovery into a portable result with optional answer text and citeable sources, while fetch retrieves a concrete public HTTP(S) URL and returns a status code plus bounded decoded content — but they are parallel registries on one capability surface, not two surfaces. The cost is a `WebService` whose registry/status/exec methods come in `Search`/`Fetch` pairs; that parallelism is intentional, not a missed extraction. Splitting into `dsh-search` and `dsh-fetch` is the rejected alternative below. +Search and fetch are separate capabilities and separate model-facing tools, but they are deliberately one seam. `ctx.web` is a single web-access middle layer between provider packages on one side and the tool consumer on the other: one service to inject, one provider-selection policy owner, one abort/error vocabulary, one place a product configures "how this harness reaches the web." The two halves do not share a request schema and have no shared business logic — search normalizes provider-backed discovery into a portable result with optional answer text and citeable sources, while fetch retrieves a concrete public HTTP(S) URL and returns a status code plus bounded decoded content — but they are parallel registries on one capability surface, not two surfaces. The cost is a `WebService` whose registry/exec methods come in `Search`/`Fetch` pairs; that parallelism is intentional, not a missed extraction. Splitting into `dsh-search` and `dsh-fetch` is the rejected alternative below. `dsh-tool-web` should register model-facing web tools when the product has enabled those tools and the `ctx.web` seam is present. Backend availability is an execution-time concern, not a schema-registration concern: @@ -33,11 +33,11 @@ Search and fetch are separate capabilities and separate model-facing tools, but This keeps the model schema stable without making plugin load order, credential state, or HMR timing part of the model-facing contract. If web search is enabled but no usable search provider exists, `web_search` remains visible and execution fails with a structured `WebError` such as `WEB_PROVIDER_UNAVAILABLE` or `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`. If a provider appears after `dsh-tool-web`, the next execution can use it without changing the schema. If a provider disappears mid-call, execution fails with a structured `WebError` instead of silently choosing another provider or falling through to `UNKNOWN_TOOL`. -The first version's provider-change signal is intentionally small. `web/providers-change` has no payload, carries no capability graph, and does not expose provider metadata. It means only "the provider registry changed; observers may recompute status from `ctx.web`." `searchStatus()` and `fetchStatus()` remain derived, not stored, and they are diagnostics plus execution-resolution inputs rather than tool-schema visibility switches. +The seam deliberately exposes no observation surface — no registry-change event and no aggregated capability-status query. Unavailability is a fact a caller observes by executing: `search()`/`fetch()` resolve the provider at call time and throw the structured `WebError` that names what failed. [The observation-surface RFC](../simplification/2026-07-04-drop-unconsumed-web-observation-surface.md) records that judgment: derived-on-call selection and enablement-based registration leave no consumer that needs a change signal or an availability probe distinct from executing and routing the error, and a future provider-status panel reintroduces the smallest signal or query it actually consumes. ## Package topology -The three-package interface/implementation/consumer split follows bash and filesystem, but the *interface* package is closer to the LLM seam. `LlmService` (`packages/llm/llm/src/index.ts`) is a name-keyed provider registry: `registerAdapter(models, adapter)` stores adapters in a `Map`, returns a disposer, throws `DUPLICATE_ADAPTER` on duplicate keys, and throws `NO_ADAPTER` at resolution time. `ctx.web` follows that registry shape, but has two capability kinds and one small selection-status layer so diagnostics and execution can explain why a search or fetch capability can or cannot run. +The three-package interface/implementation/consumer split follows bash and filesystem, but the *interface* package is closer to the LLM seam. `LlmService` (`packages/llm/llm/src/index.ts`) is a name-keyed provider registry: `registerAdapter(models, adapter)` stores adapters in a `Map`, returns a disposer, throws `DUPLICATE_ADAPTER` on duplicate keys, and throws `NO_ADAPTER` at resolution time. `ctx.web` follows that registry shape, but has two capability kinds and a richer selection policy (a configured provider id, or auto-select when exactly one usable provider is registered), so the `WebError` an execution throws can explain why a search or fetch capability cannot run. The dependency direction mirrors bash and filesystem: @@ -52,7 +52,7 @@ The dependency direction mirrors bash and filesystem: implementation ``` -At runtime, provider packages register capabilities with `ctx.web`; `tool-web` reads capability status and registers stable tools with `ctx.tools`: +At runtime, provider packages register capabilities with `ctx.web`; `tool-web` registers stable tools with `ctx.tools` and executes through the seam: ```mermaid flowchart LR @@ -60,12 +60,12 @@ flowchart LR perplexity["@deepseek-ai/dsh-web-search-perplexity"] -->|registerSearchProvider| web deepseek["@deepseek-ai/dsh-web-search-deepseek"] -->|registerSearchProvider| web fetchLocal["@deepseek-ai/dsh-web-fetch-local"] -->|registerFetchProvider| web - toolWeb["@deepseek-ai/dsh-tool-web"] -->|searchStatus/fetchStatus| web + toolWeb["@deepseek-ai/dsh-tool-web"] -->|search/fetch| web toolWeb -->|ctx.tools.register| webSearch["tool: web_search"] toolWeb -->|ctx.tools.register| webFetch["tool: web_fetch"] ``` -`@deepseek-ai/dsh-web` depends only on Cordis and low-level harness support. It declares `ctx.web`, provider interfaces, request/result types, status types, and error codes. It does not import tool, agent, session, LLM, or provider packages. +`@deepseek-ai/dsh-web` depends only on Cordis and low-level harness support. It declares `ctx.web`, provider interfaces, request/result types, the provider status type, and error codes. It does not import tool, agent, session, LLM, or provider packages. Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credentials, endpoint config, provider-specific request mapping, provider-specific response parsing, and provider-specific error translation into `WebError`. They issue network requests with the platform-native `fetch` (Node 24), mirroring `@deepseek-ai/dsh-llm-deepseek`'s adapter, NOT a cordis HTTP-client service (`ctx.http`/`@cordisjs/plugin-http`) — even where a Perplexity provider's request is shaped like an OpenAI-compatible chat completion, that wire shape is a provider-private detail and does not make the provider depend on `ctx.llm`. A provider does NOT own the `ctx.web` key (two search providers cannot both own it): like `dsh-llm-deepseek`, each provider package is a function/namespace plugin (`inject: ['web']`) whose `apply` constructs the backend and calls `ctx.web.registerSearchProvider` / `registerFetchProvider`. `@deepseek-ai/dsh-web` is the `export default` service that owns the key. @@ -92,9 +92,6 @@ interface WebService { registerSearchProvider(provider: WebSearchProvider): () => void registerFetchProvider(provider: WebFetchProvider): () => void - searchStatus(): WebCapabilityStatus - fetchStatus(): WebCapabilityStatus - search(request: WebSearchRequest, exec?: WebExecContext): Promise fetch(request: WebFetchRequest, exec?: WebExecContext): Promise } @@ -106,39 +103,33 @@ interface WebExecContext { `WebExecContext` is execution control, not business input. The first version should carry only `signal` so `tool-web` can propagate turn cancellation, tool timeout, and agent disposal into provider network requests, SSE readers, and expensive decoding. It should not pass `ToolExecution` through the seam, because that would make `dsh-web` depend on `dsh-tools`. -`@deepseek-ai/dsh-web` should also declare a Cordis event named `web/providers-change`. Provider ids are stable strings and unique within their capability kind. Registering a duplicate search provider id or duplicate fetch provider id should fail rather than silently replace the old provider. Provider registration returns a disposer, emits `web/providers-change` after successful registration, and emits it again when the provider is disposed. The registry should follow the existing `ctx.tools.register()` / `ctx.systemPrompt.section()` pattern: wrap the mutation in `ctx.effect()`, install the rollback disposer before emitting `web/providers-change`, and let a throwing registration-time change listener roll back the just-added provider instead of leaking it into the registry. +Provider ids are stable strings and unique within their capability kind. Registering a duplicate search provider id or duplicate fetch provider id should fail rather than silently replace the old provider. Provider registration returns a disposer and follows the existing `ctx.tools.register()` / `ctx.systemPrompt.section()` pattern: wrap the mutation in `ctx.effect()` so the registration is torn down with the contributing fiber. ## Provider status and selection -Provider status and capability selection are separate concepts, but both stay minimal. A provider reports only whether that concrete implementation is usable by cheap local checks such as credential presence or parseable endpoint config. A provider `status()` must not make network calls. The service reports whether the capability has a selected usable provider, or why execution would fail. +Provider status and capability selection are separate concepts, but both stay minimal. A provider reports only whether that concrete implementation is usable by cheap local checks such as credential presence or parseable endpoint config. A provider `status()` must not make network calls. -`LlmService` has no status type at all: availability is expressed as registry membership plus a resolution-time throw. `ctx.web` needs a small status answer because product apps, diagnostics, tests, and execution can report precise provider-selection failures without probing individual providers from the tool layer. Status must be derived from the configured provider id, registered providers, and each provider's cheap local `status()` on each call; it must not be stored as mutable service state. +`LlmService` has no status type at all: availability is expressed as registry membership plus a resolution-time throw. `ctx.web` follows the same discipline. The seam exposes no aggregated capability-status query — `search()` / `fetch()` derive the selection on each call from the configured provider id, the registered providers, and each provider's cheap local `status()`, and a selection failure is the structured `WebError` thrown at execution time, whose code answers "in which broad category does this capability fail" and whose message answers "exactly which provider/ids/reason." A caller that needs to know whether a capability can run executes and routes that error; nothing is stored as mutable service state. -`WebCapabilityStatus` stays intentionally small: `available` plus a `reason` discriminant, and the selected `providerId` on the available branch so diagnostics can report which provider won. It does NOT carry the per-reason payload (the unavailable provider id, the ambiguous candidate set, the underlying provider-unavailable reason). That branchable detail lives in the structured `WebError` thrown at execution time, which is the surface callers route on; duplicating it into the status union would give the same fact two homes that can disagree. `searchStatus()` / `fetchStatus()` answer "can this capability run, and if not, in which broad category does it fail" — enough for startup diagnostics and the execution-resolution decision — and the thrown error answers "exactly which provider/ids/reason." - -`WebProviderStatus` is an input to selection, not a health system. `tool-web` reads only the aggregated `searchStatus()` / `fetchStatus()`, never each provider's `status()` directly, so selection policy has one owner. +`WebProviderStatus` is an input to selection, not a health system. `tool-web` never calls a provider's `status()` directly — its only path into the seam is `search()` / `fetch()` — so selection policy has one owner. ```ts type WebProviderStatus = | { readonly available: true } | { readonly available: false; readonly reason: 'missing-credential' | 'misconfigured' } - -type WebCapabilityStatus = - | { readonly available: true; readonly providerId: string } - | { readonly available: false; readonly reason: 'none' | 'configured-missing' | 'configured-unavailable' | 'ambiguous' } ``` Selection must not depend on registration order. Cordis load order, config ordering, and HMR timing are not product semantics. -| Situation | Status / behavior | +| Situation | Execution behavior | |---|---| -| A configured provider id is registered and `status().available === true` | `available: true` for that provider | -| A configured provider id is not registered | `configured-missing`; execution fails with `WEB_PROVIDER_CONFIGURED_MISSING` | -| A configured provider id is registered but unavailable | `configured-unavailable`; execution fails with `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` | -| No provider id is configured and exactly one provider for that kind is registered and available | `available: true` for that single provider | -| No provider id is configured and no provider for that kind is registered | `none`; execution fails with `WEB_PROVIDER_UNAVAILABLE` | -| No provider id is configured and multiple usable providers for that kind are registered | `ambiguous`; execution fails with `WEB_PROVIDER_AMBIGUOUS` rather than choosing by registration order | -| No provider id is configured and providers exist but none are usable | `none`; execution fails with `WEB_PROVIDER_UNAVAILABLE` | +| A configured provider id is registered and `status().available === true` | runs that provider | +| A configured provider id is not registered | fails with `WEB_PROVIDER_CONFIGURED_MISSING` | +| A configured provider id is registered but unavailable | fails with `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` | +| No provider id is configured and exactly one provider for that kind is registered and available | runs that single provider | +| No provider id is configured and no provider for that kind is registered | fails with `WEB_PROVIDER_UNAVAILABLE` | +| No provider id is configured and multiple usable providers for that kind are registered | fails with `WEB_PROVIDER_AMBIGUOUS` rather than choosing by registration order | +| No provider id is configured and providers exist but none are usable | fails with `WEB_PROVIDER_UNAVAILABLE` | The "single provider auto-selects" rule is for tests, demos, and simple deployments. Product configs should set explicit provider ids: @@ -167,7 +158,7 @@ The "single provider auto-selects" rule is for tests, demos, and simple deployme Operational overrides such as environment variables may exist, but they must feed the same explicit selection path. For example, `DSH_WEB_SEARCH_PROVIDER=perplexity` is equivalent to config `searchProvider: perplexity`; it is not a hidden priority chain inside `dsh-tool-web`. -`ctx.web.search()` and `ctx.web.fetch()` resolve the provider at execution time using the same rules as the status query. If the selected capability is unavailable, they throw `WebError` with a structured code such as `WEB_PROVIDER_UNAVAILABLE`, `WEB_PROVIDER_CONFIGURED_MISSING`, `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`, or `WEB_PROVIDER_AMBIGUOUS`. If no provider is explicitly configured and no usable provider exists, the status and execution error are both the generic `none` / `WEB_PROVIDER_UNAVAILABLE` case; the first version should not add a diagnostic summary of every unavailable provider. +`ctx.web.search()` and `ctx.web.fetch()` resolve the provider at execution time using the selection rules above. If the selected capability is unavailable, they throw `WebError` with a structured code such as `WEB_PROVIDER_UNAVAILABLE`, `WEB_PROVIDER_CONFIGURED_MISSING`, `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`, or `WEB_PROVIDER_AMBIGUOUS`. If no provider is explicitly configured and no usable provider exists, the execution error is the generic `WEB_PROVIDER_UNAVAILABLE` case; the first version should not add a diagnostic summary of every unavailable provider. ## Search request and result schema @@ -269,14 +260,14 @@ SSRF / private-network protection (blocking private, loopback, link-local, multi `dsh-tool-web` owns two `ToolDefinition`s: `web_search` and `web_fetch`. It owns model-facing JSON schemas, snake_case argument names, prompt sections, result rendering to `ContentBlock[]`, `presentCall`, and `presentResult`. -`dsh-tool-web` must not enumerate providers or call provider `status()` directly. Its execution path is `ctx.web.search()` / `ctx.web.fetch()`, and any optional startup diagnostics should read only `ctx.web.searchStatus()` / `ctx.web.fetchStatus()`. That keeps provider selection in one layer; otherwise the tool package could decide one provider is usable while execution resolves a different state. +`dsh-tool-web` must not enumerate providers or call provider `status()` directly. Its only path into the seam is `ctx.web.search()` / `ctx.web.fetch()`. That keeps provider selection in one layer; otherwise the tool package could decide one provider is usable while execution resolves a different state. Tool registration in the first version is a minimal stable sync: 1. On plugin startup, read the `dsh-tool-web` `Config` (`search?: boolean`, `fetch?: boolean`, both default `true`) that enables or disables each web tool. 2. If web search is enabled, register `web_search` (its disposer is fiber-scoped via the effect-based registry). 3. If web fetch is enabled, register `web_fetch` (likewise fiber-scoped). -4. Do not dispose either tool merely because `ctx.web.searchStatus()` or `ctx.web.fetchStatus()` is unavailable. +4. Do not dispose either tool merely because its selected provider is missing, unusable, or ambiguous. 5. Disposing the `tool-web` fiber tears down its registrations automatically. Provider status changes affect execution results and diagnostics, not whether the model-facing schema exists. If a product wants no web tools at all, it disables `dsh-tool-web` or the individual web tool in config; if it wants web tools but the backend is misconfigured, the model sees a structured tool error at execution time. @@ -311,7 +302,7 @@ Tool execution should let these errors flow through `ToolRegistry.execute()`, wh Tests should prove the seam contract without turning this RFC into an implementation checklist. -`dsh-web` tests cover provider registration and disposal, duplicate provider ids, `web/providers-change` emission, rollback when a registration-time `web/providers-change` listener throws, `searchStatus()` and `fetchStatus()` for the selection table above, execution-time provider resolution, `maxResults` truncation of `sources[]` with `truncated` set when a provider over-returns, abort propagation through `WebExecContext.signal`, and structured `WebError` codes. +`dsh-web` tests cover provider registration and disposal (proved through execution behavior — a registered provider serves `search()`/`fetch()`, a disposed one no longer resolves), duplicate provider ids, the selection table above exercised through execution-time provider resolution, `maxResults` truncation of `sources[]` with `truncated` set when a provider over-returns, abort propagation through `WebExecContext.signal`, and structured `WebError` codes. Search provider tests cover request mapping, response parsing into `content` plus `sources[]`, missing credentials, provider errors, timeout/abort, truncation, and a self-skipping with-key smoke test for each real provider. Perplexity fixtures must include URL-only citations so the optional source fields stay honest. @@ -329,7 +320,7 @@ This is new capability work, so no compatibility migration is required while the Land the work in seam order: -1. Add `packages/web/web` with `ctx.web`, provider registration, provider status, capability status, selection, request/result/error types, and contract tests. +1. Add `packages/web/web` with `ctx.web`, provider registration, provider status, selection, request/result/error types, and contract tests. 2. Add `packages/web/web-search-exa` with parser/unit tests and a self-skipping real-provider smoke test. 3. Add `packages/web/web-search-perplexity` with parser/unit tests and a self-skipping real-provider smoke test. 4. Add `packages/web/web-search-deepseek` with parser/unit tests and a self-skipping real-provider smoke test. @@ -370,7 +361,7 @@ Rejected for the seam. `prompt` turns fetch into LLM summarization and couples p **Perplexity citations may be sparse.** A citation may be only a URL. Making `title` and `snippet` optional keeps the seam truthful but means `tool-web` must render useful fallback labels. -**Stable tool registration can defer misconfiguration to execution.** Keeping the tool visible is correct when the product enabled web access, but product apps that expect web search should surface `configured-missing`, `configured-unavailable`, and `ambiguous` loudly during startup diagnostics so users do not discover setup problems only after the model calls the tool. +**Stable tool registration can defer misconfiguration to execution.** Keeping the tool visible is correct when the product enabled web access, but product apps that expect web search should surface the structured `WEB_PROVIDER_CONFIGURED_MISSING` / `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` / `WEB_PROVIDER_AMBIGUOUS` failures loudly so users do not discover setup problems only after the model calls the tool. **Provider state can change after startup.** A tool can be visible in the request assembled at step start and lose its provider before execution. The execution path must resolve again and fail with a structured error. @@ -388,5 +379,5 @@ Rejected for the seam. `prompt` turns fetch into LLM summarization and couples p ## Open questions -- Should product app packages treat `configured-missing`, `configured-unavailable`, and `ambiguous` as fatal startup errors when web is explicitly configured, or should `dsh-web` only report status and let apps decide? +- Should product app packages probe web configuration at startup (treating `WEB_PROVIDER_CONFIGURED_MISSING`, `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`, and `WEB_PROVIDER_AMBIGUOUS` as fatal when web is explicitly configured), or leave misconfiguration to surface at the first execution? - Where should permission policy for public web access live once the deferred permission system lands: a dedicated web permission plugin on `tools/execute`, provider config, or both? diff --git a/docs/rfc/implemented/architecture/2026-06-30-event-domain-semantics.md b/docs/rfc/implemented/architecture/2026-06-30-event-domain-semantics.md index 8f5be09b2b..9a9db6359f 100644 --- a/docs/rfc/implemented/architecture/2026-06-30-event-domain-semantics.md +++ b/docs/rfc/implemented/architecture/2026-06-30-event-domain-semantics.md @@ -19,7 +19,7 @@ This is the foundational change in a stack that adds a Hooks subsystem; it estab **Three domains, one job each, with a single boundary rule.** - **`session/*` — the durable, replayable FACT log.** Owns `SessionEventMap`; every entry is JSON-only (no live objects). One `session/event` emit per append, plus the `session/flush` parallel durability checkpoint. It is also the live transcript feed: a consumer that wants to render or react to what happened subscribes here, so live rendering and `session/load` replay share one path. -- **`agent/*` — the LIVE runtime surface.** Always carries the live `Agent`. Two shapes: INTERCEPTION waterfalls (`agent/request`, `agent/step-result`, `agent/turn-continuation`) that mutate or veto, and TRANSIENT emits (`agent/status`, `agent/error`, `agent/created`/`agent/disposed`, `agent/queued`, `agent/steering`) that notify with the `Agent` in hand. Turn and step BOUNDARIES are NOT here — they are durable session events read off `session/event`, and so is the token stream (`assistant/chunk`). +- **`agent/*` — the LIVE runtime surface.** Always carries the live `Agent`. Two shapes: INTERCEPTION waterfalls (`agent/request`, `agent/step-result`, `agent/turn-continuation`) that mutate or veto, and TRANSIENT emits (`agent/status`, `agent/error`, `agent/created`/`agent/disposed`, `agent/queued`) that notify with the `Agent` in hand. Turn and step BOUNDARIES are NOT here — they are durable session events read off `session/event`, and so are the token stream (`assistant/chunk`) and mid-turn steering (`steering/message`). - **`tools/*` — the tool registry + execution seam.** **The boundary rule:** a durable, replayable fact is a `SessionEvent`; a live interception or a transient/live-object signal is an `agent`/`tools` Cordis event. A turn or step boundary is a durable fact, so it lives in the session log and is read off the `session/event` feed — it is NOT mirrored as an `agent/*` emit. @@ -31,5 +31,5 @@ This is the foundational change in a stack that adds a Hooks subsystem; it estab - The loop no longer emits any boundary mirror; `closeStep` appends `step/end` only and `closeTurn` appends `turn/end` only. A throwing `step/end`/`turn/end` session-event listener is the surviving boundary-listener failure path (contained inside `closeStep`/`closeTurn` — `Session.append` pushes the event before notifying listeners, so the boundary is durable and the turn closes balanced regardless). - Tests that observed boundaries via the removed emits now observe the durable `turn/start`/`turn/end`/`step/start`/`step/end` session events — the behavior they pin (boundary ordering, step counting) is unchanged; only the feed they read moved to the canonical one. The tests that exercised a *throwing turn-boundary emit listener* were deleted, because that code path no longer exists (there is no emit to throw from). Per [AGENTS.md "tests document behavior, not golden truth"](../../../../AGENTS.md), the behavior and its test moved (or died) together. - The loop marks the step open (`stepOpen = true`) BEFORE appending `step/start`, because `Session.append` pushes the event to the log before notifying `session/event` listeners (validation throws happen earlier, before the push — see [the session append contract](../../../core-data-structures/session.md)). So a throwing `step/start` session-event listener runs with the step already open and the event already in the log: the loop's outer catch then calls `closeStep()`, which appends the balancing `step/end`, and the turn closes balanced with an error (`turn/start → step/start → step/end → turn/end` — verified by the invariants oracle in the regression test). Closing the open step is owed precisely because the marker is set first. -- The full realization of this is [the simplification RFC "Stop mirroring durable boundaries as agent events"](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md): all four boundary mirrors are removed and every consumer reads boundaries off `session/event`. `agent/steering` (a live control signal, not a boundary mirror) is retained; see that RFC's scope section. +- The full realization of this is [the simplification RFC "Stop mirroring durable boundaries as agent events"](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md): all four boundary mirrors are removed and every consumer reads boundaries off `session/event`. `agent/steering` (not a boundary mirror) stayed outside that RFC's scope and was removed by its own follow-up, [Remove the `agent/steering` mirror emit](../simplification/2026-07-04-remove-agent-steering-mirror.md) — it mirrored the durable `steering/message`. - The cordis catalog (`docs/cordis-catalog/events-and-services.md`) is regenerated to drop the mirror events. diff --git a/docs/rfc/implemented/feature/2026-06-30-hook-protocol-lib.md b/docs/rfc/implemented/feature/2026-06-30-hook-protocol-lib.md index 848eeec219..12a47ddcf1 100644 --- a/docs/rfc/implemented/feature/2026-06-30-hook-protocol-lib.md +++ b/docs/rfc/implemented/feature/2026-06-30-hook-protocol-lib.md @@ -16,10 +16,10 @@ A new `packages/hooks/` group with `hook-protocol` as a pure library. It owns fo **Shared (here):** - **Matcher** — `matchesMatcher(pattern, query, mode)`. The ONE axis the dialects differ on is collapsed to the `mode` parameter: `claude` treats a pure `[A-Za-z0-9_|]+` pattern as a literal (pipe = exact-match alternation) and anything else as a regex; `codex` is always an unanchored regex. Match-all on absent/`''`/`'*'`; an invalid regex matches nothing (never throws into the loop). -- **Execution** — `runHook(bash, hook, options, now)`. Runs a command hook through the `ctx.bash` seam rather than a bespoke `spawn`: the executor already provides the scrubbed-but-overridable env, process-group kills, and timeout the protocol needs, and `dsh-bash`'s `stdin`/`env` fields (added in the bash-seam PR for exactly this) are the trusted-plugin surface an in-process bridge is allowed to use. It serializes the bridge-built payload to stdin (trailing newline iff CC), honors the hook's `timeoutSec`, and never throws (an executor rejection becomes a non-blocking-error `HookOutput`). -- **Decode** — `parseHookOutput(exit, stdout, stderr)`, the exit-code + structured-stdout codec, producing a dialect-neutral `HookOutput`. Exit `0` → lenient JSON parse of stdout; exit `2` → blocking error with `stderr` as the reason (surfaced as `decision: 'block'` so no caller needs a separate exit-code branch); other → non-blocking error. Parses the full CC superset (`continue`/`stopReason`/`suppressOutput`/`decision`/`hookSpecificOutput.{permissionDecision,additionalContext,updatedInput}`/`systemMessage`); the bridge honors only the subset meaningful for its dialect. +- **Execution** — `runHook(bash, hook, options)`. Runs a command hook through the `ctx.bash` seam rather than a bespoke `spawn`: the executor already provides the scrubbed-but-overridable env, process-group kills, and timeout the protocol needs, and `dsh-bash`'s `stdin`/`env` fields (added for exactly this) are the trusted-plugin surface an in-process bridge is allowed to use. It serializes the bridge-built payload to stdin (trailing newline iff CC), honors the hook's `timeoutSec` (else `DEFAULT_HOOK_TIMEOUT_MS`, the 10-minute reference default both dialects share), and never throws (an executor rejection becomes a non-blocking-error `HookOutput`). +- **Decode** — `parseHookOutput(exit, stdout, stderr)`, the exit-code + structured-stdout codec, producing a dialect-neutral `HookOutput`. Exit `0` → lenient JSON parse of stdout; exit `2` → blocking error with `stderr` as the reason (surfaced as `decision: 'block'` so no caller needs a separate exit-code branch); other → non-blocking error. Parses the CC structured-stdout fields that have a consumer on some path (`continue`/`stopReason`/`decision`/`hookSpecificOutput.{permissionDecision,additionalContext,updatedInput}`/`systemMessage`); the bridge honors only the subset meaningful for its dialect. Fields with no consumer on any path are not parsed at all (CC's `suppressOutput` — hook stdout never enters a transcript here, so there is nothing to suppress; see [the tighten-hook-protocol-contract RFC](../simplification/2026-07-04-tighten-hook-protocol-contract.md)). - **Merge** — `mergeHookOutputs(outputs)`, folding multiple matched hooks into one most-restrictive `MergedHookOutcome`: permission precedence **deny > ask > allow**, halt sticky on the first `continue:false`, block reasons joined `\n\n`, context/system-messages accumulated in order. -- **`hook/*` session events** — `hook/invoked` / `hook/result`, declaration-merged into `SessionEventMap` (log-only, like `compact/*` — NOT `SurfaceEventType`s), with `appendHookInvoked`/`appendHookResult` helpers so the invoked/result pairing and turn-enclosure stay consistent across bridges. +- **`hook/*` session events** — `hook/invoked` / `hook/result`, declaration-merged into `SessionEventMap` (log-only, like `compact/*` — NOT `SurfaceEventType`s), with `appendHookInvoked`/`appendHookResult` helpers so the invoked/result pairing and turn-enclosure stay consistent across bridges. `appendHookResult` also owns the durable record's semantics — the decision string (the hook's parsed decision, else `'stop'` on `continue:false`, else `'pass'`) and the 500-character `stderrSummary` truncation derive from the `HookOutput` here, not per-bridge. **Per-dialect (the bridge plugins):** building each event's stdin payload (CC's base+per-event field sets vs Codex's snake_case with `turn_id`/`model` extras), the dialect's env + `${CLAUDE_PLUGIN_ROOT}` substitution (CC) vs none (Codex), and mapping the neutral `HookOutput`/`MergedHookOutcome` onto the harness's seam-specific typed Decisions (`PreToolDecision`, `PromptDecision`, `ContinuationDecision`, `PostToolDecision`). diff --git a/docs/rfc/implemented/process/2026-07-04-persistence-log-catalog.md b/docs/rfc/implemented/process/2026-07-04-persistence-log-catalog.md new file mode 100644 index 0000000000..b03c79e84a --- /dev/null +++ b/docs/rfc/implemented/process/2026-07-04-persistence-log-catalog.md @@ -0,0 +1,29 @@ +# RFC: Generated persistence log event catalog + +Status: implemented (accepted 2026-07-04) + +## Context + +The session event log is the harness's on-disk contract: every `SessionEventMap` member is a record a persistence backend writes verbatim and a replay reconstructs from, and adding one that breaks the durability rules is a breaking change to the on-disk format. Yet the vocabulary had no single reference. The declarations are split across three files — the owning interface in `@deepseek-ai/dsh-session` plus declaration merges in `@deepseek-ai/dsh-compact` and `@deepseek-ai/dsh-hook-protocol` — and the doc surfaces covered it with hand-copies: a `hook/*` payload table in [session.md](../../../core-data-structures/session.md), a `compact/*` payload table in the compact README, payload bullets in the hook-protocol README, and a name-list in the session README. The name-list's merge note had already drifted (it named the compaction merge and omitted the hook merge entirely), and nothing could catch the next merge going undocumented: a hand-copy only checks the names someone already wrote down. This is the same gap the [cordis catalog](2026-06-20-generated-cordis-catalog.md) closed for bus events and the [tool catalog](2026-07-02-tool-schema-catalog.md) closed for model-facing tools — and log events are covered by neither: a `SessionEventMap` member is not a cordis `Events` declaration (it reaches listeners via the single `session/event` emit), so it has no cordis-catalog row by design. + +## Decision + +Generate `docs/persistence-catalog/log-events.md` from source, with a freshness gate, as the fourth reference surface: the *records* a persisted session log can contain, complementing the cordis catalog (wiring), core-data-structures (vocabulary), and the tool catalog (tools). + +`scripts/gen-persistence-catalog.ts` is a pure TypeScript-AST pass, like `gen-cordis-catalog.ts` and unlike the boot-based tool catalog — the right technique because log events ARE statically knowable: every member is a string-literal-named property with a static type annotation, so the AST is the whole truth. The walk collects every `interface SessionEventMap` declaration under `packages/*/*/src` — the owning top-level interface and every `declare module '@deepseek-ai/dsh-session'` merge — so a brand-new event, core or merged, appears in the next regenerate and an un-regenerated file fails `--check` (`verify-persistence-catalog`, a `doc-sync` member, so pre-push and CI both run it). Each entry renders the member's JSDoc prose, its payload (printed through the TypeScript printer, so a newline-separated multi-line type literal still yields a valid one-line fragment), a surface badge, cross-links into core-data-structures, and the declaration's source pointer, grouped by scope. + +Specific choices: + +- **JSDoc completeness, enforced.** Every member must carry description prose — the JSDoc becomes the catalog entry, the same forcing function the cordis catalog applies to bus events. An `@mode` tag on a member is a hard error: dispatch modes belong to cordis bus events, and a log event has none — the tag would misread as "this fires on the bus with mode X". Violations aggregate into one error listing every offender. +- **The surface badge is derived, not hand-listed.** `SurfaceEventType` — the subset that produces LLM messages and may carry `surfaceOp` — is parsed from its union declaration in the owning package; a union member naming no declared event is a hard error (a stale union member would otherwise silently badge nothing). Everything else renders **log-only**. +- **A dedicated fence.** Payload blocks use a ` ```ts persistence-catalog ` info string that `doc-typecheck` recognizes and skips, excluded from the opt-out ratio — the same treatment as `ts cordis-catalog` (a bare payload fragment is not standalone-compilable). +- **Repo scope.** The catalog enumerates the packages in this repo, matching the siblings' packages-only scope; a downstream plugin can merge further event types, which are outside the catalog by construction. The walk defends its own assumptions with hard errors: the owning top-level `interface SessionEventMap` must be the single exported declaration in `@deepseek-ai/dsh-session` (an unrelated, local, or duplicate same-named interface cannot be catalogued as the on-disk vocabulary), no declaration may carry `extends` (inherited keys would join `keyof SessionEventMap` without a catalog row), every member must be a property signature with an explicit payload type (a method-form member would join `keyof` yet slip past a silent walk), and a duplicate member across declarations fails. + +This supersedes the hand-copies: the session.md `hook/*` table, the compact README's event table, the hook-protocol README's payload bullets, and the session README's name-list now link the catalog instead of restating payloads (the surrounding semantics prose stays where it was). The two stray `@mode emit` tags on the hook-protocol merge members are removed — the new gate rejects them as the category error they were. + +## Consequences + +- The catalog cannot drift: a vocabulary change the committed file doesn't reflect fails `verify-persistence-catalog` in the pre-push hook and CI, and a new merged event with no JSDoc fails the generator outright — a plugin can no longer add an undocumented on-disk record type. +- Event prose has a single home, the JSDoc at the declaration; thin JSDoc yields a thin catalog entry, pressuring authors to document at the source. +- The `SurfaceEventType` union is now structurally load-bearing for docs: renaming an event without updating the union (or vice versa) fails the generator, not just the compiler. +- The badge derivation assumes the union stays a closed set of string literals with exactly one owner; a refactor away from that shape must update the generator in the same change. diff --git a/docs/rfc/implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md b/docs/rfc/implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md index d8ff017d63..f198ff9ec6 100644 --- a/docs/rfc/implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md +++ b/docs/rfc/implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md @@ -5,10 +5,11 @@ Status: implemented (accepted 2026-07-01) + The original proposal bundled `agent/steering` into the removal; keeping it + out kept this RFC's scope to boundaries. Each retained event was later + removed by its own decision — see + [Stop mirroring the token stream as an agent event](2026-07-02-remove-stream-chunk-mirror.md) + and [Remove the `agent/steering` mirror emit](2026-07-04-remove-agent-steering-mirror.md). --> ## Problem @@ -30,7 +31,7 @@ Removed (durable-boundary mirrors — the session log is authoritative for each) RETAINED — NOT durable-boundary mirrors, so out of scope for this decision: -- `agent/steering` — a live control signal, not a boundary. (The original proposal bundled it into the removal; validating against the code, it is not a duplicate of a durable boundary, so removing it here would have been scope creep. Its fate is a separate future decision.) +- `agent/steering` — not a boundary, so out of scope for THIS decision (the original proposal bundled it into the removal; that would have been scope creep here). It mirrors the durable `steering/message` control record rather than a boundary, and was removed by its own follow-up: [Remove the `agent/steering` mirror emit](2026-07-04-remove-agent-steering-mirror.md). - `agent/stream-chunk` — the live token stream. Out of scope for THIS decision (a mirror of the durable `assistant/chunk`, not a boundary), it was removed by its own follow-up: [Stop mirroring the token stream as an agent event](2026-07-02-remove-stream-chunk-mirror.md). - `agent/created`, `agent/disposed`, `agent/status`, `agent/error`, `agent/queued` — lifecycle/control events that are not transcript data. `agent/queued` in particular is an inbox acknowledgement that fires before any durable event exists (cancelled queued work may never enter the log), so it is deliberately live-only. diff --git a/docs/rfc/implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md b/docs/rfc/implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md index b2ed1bc5d4..73c62a6864 100644 --- a/docs/rfc/implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md +++ b/docs/rfc/implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md @@ -33,7 +33,7 @@ Removed: `agent/stream-chunk`. Not touched: - `assistant/chunk` (the durable session event) — the authoritative token stream, kept exactly as-is. This RFC removes the LIVE MIRROR, not the persistence (the persistence-removal proposal was separately rejected — see above). -- `agent/steering` — a live control signal with no durable twin, retained (its fate remains a separate future decision, per the boundary RFC). +- `agent/steering` — not touched by THIS decision (a control signal, not the token stream). Its durable twin is `steering/message`, and the mirror emit was removed by its own follow-up: [Remove the `agent/steering` mirror emit](2026-07-04-remove-agent-steering-mirror.md). - `agent/status`, `agent/error`, `agent/created`/`agent/disposed`, `agent/queued`, `agent/session-start` — lifecycle/control events that are not transcript data and have no durable duplicate. ## What we give up diff --git a/docs/rfc/implemented/simplification/2026-07-04-drop-image-content-block.md b/docs/rfc/implemented/simplification/2026-07-04-drop-image-content-block.md new file mode 100644 index 0000000000..ccbf5d755d --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-drop-image-content-block.md @@ -0,0 +1,27 @@ +# RFC: Drop the `image` content block until a path can honor it + +Status: implemented (proposed and accepted 2026-07-04) + +## Problem + +`ImageBlock` (`packages/llm/llm/src/types.ts`) had no production producer, and every consumer on every path DROPPED it: the deepseek adapter's serializer skipped image blocks (a documented MVP limitation), the pi-ai converter skipped them as unrepresentable, the ACP codec neither advertises image prompt capability nor forwarded image blocks outbound and REJECTS image prompt content inbound, and the compaction estimator charged a flat token constant and rendered `[image]`. An `ImageBlock` constructed then would silently vanish from the wire — the vocabulary advertised a capability no path honored, which is the silent-data-loss shape AGENTS.md's defensive patterns warn against. The only constructors anywhere were tests pinning the skip/drop/estimate branches. + +## Decision + +Remove `ImageBlock`, its `ContentBlockMap` entry (and its `cache?: CacheHint` field with it), the explicit `image` estimate/placeholder arms in compact-basic, and the image-naming comments in the deepseek serializer's, pi-ai converter's, and ACP codec's default arms — those default arms absorb the case the way they absorb any unknown block type. Updated in the same change: the vocabulary line in [architecture.md](../../../architecture.md), the block list in `packages/llm/llm/README.md`, the deepseek README's image-skip row, the pi-ai README's images-not-representable row, the compact-basic README's image-estimation and `[image]`-placeholder rows, the pastes in [core.md](../../../core-data-structures/core.md) and [llm-streaming.md](../../../core-data-structures/llm-streaming.md), and the [content-block vocabulary RFC](../architecture/2026-06-11-content-block-vocabulary.md)'s block list and multimodal-home consequence per [implemented/AGENTS.md](../AGENTS.md); the tests that constructed image blocks to exercise the removed branches were dropped (the estimate pin) or retargeted onto the merge-extensible default arms (plugin-added block types). The ACP codec's inbound rejection of image PROMPT content is unaffected — that guard is about protocol content a client can send regardless of our vocabulary, and it stays. + +## Why not keep it? + +This was the most contested cut in the batch. Multimodal input (screenshots) is a plausible near-term coding-agent feature, and the [content-block vocabulary RFC](../architecture/2026-06-11-content-block-vocabulary.md) reserved the slot deliberately. Two responses. First, `ContentBlockMap` is merge-extensible by design: a real multimodal feature reintroduces `image` in core in the same coordinated change that maps it in the adapters, advertises and renders it in ACP, and prices it in compaction — the producer and its consumers arrive together, which is how the map is meant to grow. Second, the middle option — keep the type but make adapters throw UNSUPPORTED instead of silently dropping — converts this into exactly the shape the sibling request-knobs proposal (`2026-07-04-drop-inert-request-knobs`) argues against: surface whose only implementation is rejection. Absence (a compile error at the would-be producer) is strictly clearer than either silent loss or universal throw. + +The recorded fallback, had review landed on keeping the slot: keep `ImageBlock` but replace every silent skip with a loud rejection, and document that policy in the vocabulary — the silent drop was the one state with no defender. Review landed on removal; the fallback stands as the documented alternative should the slot ever return ahead of a full feature. + +## Acceptance criteria + +- No `ImageBlock` / harness `type: 'image'` block construction outside this RFC; the codec's inbound ACP-image rejection still passes its tests. +- Adapter/codec/compaction switches handle the case through their unknown-block default arms (pinned by the plugin-added-block tests). +- Doc pastes, the manifest, and the architecture vocabulary list updated; `pnpm run doc-sync` green. + +## Risks + +Re-adding a core vocabulary type later touches several packages at once — but that coordinated change is the shape a real multimodal feature needs anyway (adapter mapping, ACP advertisement, compaction pricing), and none of it existed to preserve. diff --git a/docs/rfc/implemented/simplification/2026-07-04-drop-inert-request-knobs.md b/docs/rfc/implemented/simplification/2026-07-04-drop-inert-request-knobs.md new file mode 100644 index 0000000000..3724d680f5 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-drop-inert-request-knobs.md @@ -0,0 +1,33 @@ +# RFC: Drop `GenerateOptions.prefill` and `ToolSchema.strict` — request knobs with no working end-to-end path + +Status: implemented (proposed and accepted 2026-07-04) + +## Problem + +Two request-contract knobs rode the whole request pipeline, yet neither could do anything: + +- **`prefill`** (`packages/llm/llm/src/types.ts`) had no production setter — the loop assembles `model`/`system`/`tools`/`messages` plus `sessionId`/`signal`, and the compaction backend adds only `maxTokens` — and BOTH adapters rejected it: `packages/llm/llm-deepseek/src/serialize.ts` and `packages/llm/llm-pi-ai/src/adapter.ts` each threw `LlmError('UNSUPPORTED')` on a non-undefined `prefill`. The field's entire observable behavior was two throws, each pinned by one adapter test. DeepSeek's chat-prefix completion is a Beta feature on a base URL neither adapter targets. +- **`strict`** (`ToolSchema`, same file) was threaded through `DefineToolOptions`/`defineTool` (`packages/core/tools/src/schema.ts`), the registry's `schemas()` allowlist (`packages/core/tools/src/index.ts`), the deepseek wire mapping (`packages/llm/llm-deepseek/src/serialize.ts`, whose wire-type note recorded that strict mode requires the `/beta` base URL the adapter does not use), a per-tool payload-patching pass in `packages/llm/llm-pi-ai/src/adapter.ts`, and a conditional `Strict:` row in the tool-catalog renderer (`scripts/gen-tool-catalog.ts`). No shipped tool set it — `rg` across every `tool-*` package src and `examples/` found zero `strict:` producers; the only setters were dsh-tools unit tests. + +Both knobs were adapter-symmetric, so removal shed them from both twins together — the [twin-adapter design](../architecture/2026-06-13-twin-llm-adapters.md) is untouched. + +## Decision + +- `prefill` is removed from `GenerateOptions`, along with both adapters' UNSUPPORTED guards, the tests pinning the throws, the paste line in [core.md](../../../core-data-structures/core.md), and the adapter README rows documenting the rejection. The cookbook's UNSUPPORTED guidance ([adding-an-llm-adapter.md](../../../cookbook/adding-an-llm-adapter.md)) states the rule generically — a `GenerateOptions` field your provider cannot honor throws `LlmError(..., 'UNSUPPORTED')` — instead of using prefill as the example. The [content-block vocabulary RFC](../architecture/2026-06-11-content-block-vocabulary.md)'s consequences record prefill as producer-gated rather than as having a home, per [implemented/AGENTS.md](../AGENTS.md). +- `strict` is removed from `ToolSchema`, `DefineToolOptions`, `defineTool`, the `schemas()` allowlist, the deepseek serializer branch and its wire-type field, and the tool-catalog renderer's `Strict:` row. The pi-ai payload fixup is simplified to the unconditional scrub of pi-ai's own per-tool strict default (pi-ai stamps `strict: false` on every serialized tool; the hand-rolled twin sends no such field, so the scrub survives for wire parity, pinned by its serializer test). The setter tests and the core.md paste line are gone; both `GenerateOptions` and `ToolSchema` keep their rows in `scripts/type-equiv.manifest.json`, since each type survives minus a field. + +This RFC deliberately does NOT touch `temperature`, `stop`, or `maxTokens`: those are honored end-to-end by both adapters and are the natural first targets of a request-mutating hook plugin on `agent/request`. + +## Why not keep them? + +"An explicit UNSUPPORTED throw is honest contract behavior" — but a knob whose only implementation across both twins is rejection promises nothing, and deleting it upgrades the failure mode: an accidental setter becomes a compile error instead of a runtime throw. "Strict schema adherence is an officially documented provider feature with complete plumbing" — but a knob is not product surface until a shipped tool sets it AND an endpoint honors it; today neither is true. Each returns with its first real producer: `prefill` together with an adapter that implements chat-prefix completion (and a stated policy for adapters that do not), `strict` together with a tool that wants it and a beta-endpoint story. + +## Acceptance criteria + +- `rg prefill` returns only RFC records (this one and the [content-block vocabulary RFC](../architecture/2026-06-11-content-block-vocabulary.md)'s producer-gated consequence); a tool-schema-scoped `rg strict` returns only this RFC, the surviving pi-ai scrub, and unrelated prose such as `strictEqual`. +- Both adapters compile and their contract tests pass without the guards; the pi-ai fixup still scrubs the library's strict default (wire parity pinned by its serializer tests). +- Doc pastes and the type-equiv manifest in sync; `pnpm run doc-sync` green. + +## Risks + +The shipped hook bridges set no request fields at all, and a request-mutating plugin (an `agent/request` waterfall listener) would reach for `temperature`/`stop` (kept, working), not a field adapters reject. If chat-prefix completion or strict mode become product features, the re-add lands with the adapter/endpoint work, where the contract can say what actually happens rather than "everyone throws". diff --git a/docs/rfc/proposed/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md b/docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md similarity index 79% rename from docs/rfc/proposed/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md rename to docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md index e3516d974f..7c5881b106 100644 --- a/docs/rfc/proposed/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md +++ b/docs/rfc/implemented/simplification/2026-07-04-drop-unconsumed-web-observation-surface.md @@ -1,13 +1,13 @@ # RFC: Drop the unconsumed web observation surface — the `providers-change` event and the status methods -Status: proposed +Status: implemented (proposed and accepted 2026-07-04) ## Problem `WebService` exposes an observation surface no production code observes: - **`web/providers-change`** (`packages/web/web/src/index.ts`) is declared and emitted on every provider registration and disposal, and each registration effect's rollback yield is ordered BEFORE the emit solely so a throwing change listener unwinds the registration. No listener exists outside the package's own two unit tests (one of which exists to pin that rollback ordering). -- **`searchStatus()` / `fetchStatus()` and the `WebCapabilityStatus` union** (same package) have zero production callers: `dsh-tool-web` executes directly through `ctx.web.search()`/`fetch()` and surfaces unavailability as the structured `WebError` codes the seam throws at execution time (`packages/web/tool-web/src/search.ts`, `packages/web/tool-web/src/fetch.ts`); the only status callers are the web packages' own tests. The prose in `packages/web/tool-web/README.md` and [architecture.md](../../../architecture.md) still claims the tool "reads only the aggregated `searchStatus()`/`fetchStatus()`" — drift that survives only because nothing checks prose against call sites. +- **`searchStatus()` / `fetchStatus()` and the `WebCapabilityStatus` union** (same package) have zero production callers: `dsh-tool-web` executes directly through `ctx.web.search()`/`fetch()` and surfaces unavailability as the structured `WebError` codes the seam throws at execution time (`packages/web/tool-web/src/search.ts`, `packages/web/tool-web/src/fetch.ts`); the only status callers are the web packages' own tests. The prose in `packages/web/tool-web/README.md` and [architecture.md](../../../architecture.md) claims the tool "reads only the aggregated `searchStatus()`/`fetchStatus()`" — drift that survives only because nothing checks prose against call sites. The seam's own design starves both surfaces of consumers: tool registration follows product ENABLEMENT, not provider availability (`packages/web/tool-web/src/index.ts`), and provider selection resolves at execution time, never cached — so there is no cache to invalidate, no registration set to recompute, and no caller that needs an availability probe distinct from executing and routing the structured error. HMR cleanup is carried by the effect disposers themselves. @@ -15,7 +15,7 @@ This mirrors [drop the unconsumed `llm/adapter-change` event](../../implemented/ ## Proposal -Delete the event declaration, both emits, and the rollback-before-emit ordering (the plain `ctx.effect` disposer keeps HMR cleanup). Delete `searchStatus()`/`fetchStatus()`/`WebCapabilityStatus` — the provider-private `status()` stays, since it feeds execution-time selection. Delete the two event tests and rewrite the status-based test assertions onto the behavior a real caller observes (a successful `search()`/`fetch()`, or the structured `WebError` codes for unavailable/ambiguous/misconfigured provider sets). Run `pnpm run gen-cordis-catalog`; update `packages/web/web/README.md`, `packages/web/tool-web/README.md` (the drifted reads-status sentence), [web.md](../../../core-data-structures/web.md), and the web paragraph in [architecture.md](../../../architecture.md). The implementing PR amends the [web capability seam RFC](../../implemented/architecture/2026-06-24-web-capability-seam.md)'s facts (it specifies the event and the status aggregation) per [implemented/AGENTS.md](../../implemented/AGENTS.md). +Delete the event declaration, both emits, and the rollback-before-emit ordering (the plain `ctx.effect` disposer keeps HMR cleanup). Delete `searchStatus()`/`fetchStatus()`/`WebCapabilityStatus` — the provider-private `status()` stays, since it feeds execution-time selection. Delete the listener-throw rollback test that exists solely for the removed event, and rewrite the emission assertions and every status-based assertion onto the behavior a real caller observes (a successful `search()`/`fetch()`, or the structured `WebError` codes for unavailable/ambiguous/misconfigured provider sets). Run `pnpm run gen-cordis-catalog`; update `packages/web/web/README.md`, `packages/web/tool-web/README.md` (the drifted reads-status sentence), [web.md](../../../core-data-structures/web.md), and the web paragraph in [architecture.md](../../../architecture.md). Amend the [web capability seam RFC](../../implemented/architecture/2026-06-24-web-capability-seam.md)'s facts (it specified the event and the status aggregation) per [implemented/AGENTS.md](../AGENTS.md). ## Why not keep it? diff --git a/docs/rfc/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md b/docs/rfc/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md new file mode 100644 index 0000000000..d938f3eafb --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md @@ -0,0 +1,31 @@ +# RFC: Prune producer-less vocabulary variants (block cache hints, the `agent` message source, the `continuation` turn trigger) + +Status: implemented (proposed and accepted 2026-07-04) + +## Problem + +The merge-extensible vocabulary maps are designed to grow by declaration merging, and the codebase already states the admission policy on `TurnEndReasonMap` (`packages/core/session/src/types.ts`): a variant like `refusal` is "deliberately omitted until" an adapter or loop first emits it. Three declared vocabulary items violated that policy — each had no producer and no consumer, and two had not even a test: + +- **`CacheHint` and its `cache?: CacheHint` block fields** on `TextBlock`/`ToolResultBlock` (`packages/llm/llm/src/types.ts`; the image block carried a third such field, which left with it — see [the drop-image RFC](2026-07-04-drop-image-content-block.md)). Nothing constructed a block with `cache:` anywhere — src, tests, and doc pastes all came up empty — and neither adapter read `.cache`: DeepSeek prompt caching is automatic, so the adapters map `prompt_cache_hit_tokens` OUT of responses without ever sending a hint IN. This was Anthropic-style `cache_control` surface with no provider that could honor it. +- **`MessageSourceMap.agent`** (`{ kind: 'agent'; agentId: string }`, same file). Zero constructors, tests included. Its intended producer shipped without it: the subagent backends send the parent's prompt to the child with no `source`, so it logs as `{ kind: 'user' }`, and the generic envelope renderer interpolates `source.kind` without ever routing on it. +- **`TurnTriggerMap.continuation`** (`packages/core/session/src/types.ts`). The loop structurally cannot emit it — continuation happens *within* a turn as further steps, never as a new turn — and it constructs only `message` and `injection` triggers. The only writer was one hand-built test fixture needing an arbitrary non-message trigger (`packages/support/llm-replay/tests/llm-replay.spec.ts`), which an `injection` trigger serves equally; the only production trigger reader, the ACP bridge, filters on `kind === 'message'`. + +## Decision + +`CacheHint`, its `cache?` block fields, the `agent` message-source variant, and the `continuation` turn-trigger variant are deleted: the shipped vocabulary carries none of them. The llm-replay fixture uses an `injection` trigger (any non-`message` trigger serves its purpose). The type-equiv pastes in [core.md](../../../core-data-structures/core.md) and [session.md](../../../core-data-structures/session.md) match the pruned maps — both symbols keep their rows in `scripts/type-equiv.manifest.json`, since each map survives minus a member — and the [content-block vocabulary RFC](../architecture/2026-06-11-content-block-vocabulary.md)'s consequences record cache hints as producer-gated rather than as having a home, per [implemented/AGENTS.md](../AGENTS.md). + +Each variant returns the day it gains a real producer, exactly as the maps are designed to grow: a caching feature re-adds `cache` together with the adapter that transmits it; subagent attribution re-adds `agent` together with the backend that stamps it and a consumer that routes on it; an auto-continue feature that genuinely starts new turns re-adds `continuation` with the plugin that emits it. + +## Why not keep them? + +The [content-block vocabulary RFC](../architecture/2026-06-11-content-block-vocabulary.md) listed "cache hints … have a home" as a design consequence, and reserved slots do advertise intent. But an empty slot is contract surface every implementation and consumer must consider (must my adapter honor `cache`? must my renderer route `agent` sources?), and the sibling map's own JSDoc already rejects reservation-without-emitter — `refusal` and `max_turn_requests` are named as variants to add *when something first emits them*, not declared in advance. Holding already-declared dead variants to the same standard makes the vocabulary mean something: if it is in the map, something produces it. + +## Acceptance criteria + +- `rg` for `CacheHint`, the `agent` message-source spelling, and the `continuation` trigger spelling returns only RFC records (this one, and [the drop-image RFC](2026-07-04-drop-image-content-block.md)'s account of the image block's own `cache` field). +- The core-data-structures pastes and the type-equiv manifest are in sync (`pnpm run doc-sync` green). +- The fixture asserts the same replay behavior with an `injection` trigger; the suite is green. + +## Risks + +None operational — nothing could construct these values. The mirror-event removals (recorded in [the boundary-mirror RFC](2026-06-20-remove-agent-boundary-mirror-events.md) and [the stream-chunk RFC](2026-07-02-remove-stream-chunk-mirror.md)) touch only transient `agent/*` events, never the durable vocabulary, so there is no collision. Elsewhere in the vocabulary the admission policy already holds: `rejected`, `prompt/blocked`, and `hook/invoked`/`hook/result` each have live producers — this RFC extends the same bar to the three variants that lacked one. The image block's own `cache?` field belongs to [the drop-image RFC](2026-07-04-drop-image-content-block.md), which removed it together with the block; this RFC covers the two fields on the block types that remain. diff --git a/docs/rfc/implemented/simplification/2026-07-04-prune-write-only-fs-surface.md b/docs/rfc/implemented/simplification/2026-07-04-prune-write-only-fs-surface.md new file mode 100644 index 0000000000..b80c22af4e --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-prune-write-only-fs-surface.md @@ -0,0 +1,29 @@ +# RFC: Prune write-only fields and a dead routing knob from the fs seam + +Status: implemented (proposed and accepted 2026-07-04) + +## Problem + +The [fs seam split](2026-06-26-fsspec-style-fs-seam.md) moved read routing and policy out of the backend into `dsh-tool-fs` and `dsh-fs-policy`. Four pieces of surface kept the pre-split shape — populated on every call, read by nobody: + +1. **`STREAM_MIN_SIZE` + `FsIoInternals.streamMinSize` in `dsh-fs-local`** — *removed ahead of this change by the no-hardcoded-tunables audit, which made the routing bound `dsh-tool-fs`'s `readStreamMinSize` config; recorded here as part of the full prune.* Originally (`packages/fs/fs-local/src/fsio.ts`, re-exported from `packages/fs/fs-local/src/index.ts`): zero readers anywhere, including fs-local's own source and tests. The backend has no read routing — `readWholeText`/`streamWholeText` are separate primitives the caller chooses between — and the real routing constant lives in the consumer (`packages/fs/tool-fs/src/read.ts`, compared against `info.size`). Two mirrors of the 10 MiB fact; the backend's was dead, and the knob's JSDoc claimed a "read routing" override that did not exist. +2. **`FsTarget.inputPath`** (`packages/fs/fs/src/types.ts`): every backend and every test fake had to fabricate a "diagnostics only" value with zero production readers — the policy plugin and every error message use `targetKey`/`displayPath`. The `listDir` producer exposed the semantic wobble: directory children got the bare entry name, which was nobody's "input". +3. **`FsEditOutcome.replacements` + `.replaceAll`** (`packages/fs/fs/src/types.ts`): `replacements` had zero production readers (the single-match policy itself stays — it is enforced by the `FS_AMBIGUOUS_EDIT`/`FS_EDIT_NOT_FOUND` throws inside the backend, whose error message keeps the internal count); `replaceAll` was read only by `formatEditOutput` in `packages/fs/tool-fs/src/edit.ts` — as an echo of the `replace_all` argument the tool already holds. Shrunk, `FsEditOutcome` is `{ version, before, after }`, parallel to `FsWriteOutcome`'s genuinely backend-discovered fields. +4. **`FileReadOutcome.limit` + `.version`** (`packages/fs/tool-fs/src/read-render.ts`): populated by the read tool, but `formatReadOutput` renders `offset`/`lines`/`totalLines`/`truncatedByBytes` only, and the `fs/observed` emit uses `info.version` directly rather than an outcome copy. + +## Decision + +Delete the fs-local constant, its re-export, and the `streamMinSize` knob (the remaining `FsIoInternals` knobs are genuinely used by the atomic-write tests); drop `inputPath` from `FsTarget`; shrink `FsEditOutcome` to `{ version, before, after }` and pass `replaceAll` to `formatEditOutput` from the parsed args; drop `limit`/`version` from `FileReadOutcome`. The [filesystem.md](../../../core-data-structures/filesystem.md) pastes, `packages/fs/fs/README.md`, and the test fakes that had to fabricate the removed fields shrink with the types. + +## Why not keep them? + +A future permission/containment layer might want the pre-resolution path for error text — but it would want the *request*, which every call site still holds. "N occurrences replaced" might become model-facing text — a behavior change to design when wanted, and the backend-internal count survives for its error message. A read footer might display `limit` — everything the footer shows already derives from `lines`/`totalLines`. Meanwhile every current and future backend (remote, native) would have to fabricate wire fields nobody consumes, and every test fake would have to satisfy them. + +## Acceptance criteria + +- The removed surfaces are gone — `STREAM_MIN_SIZE`/`streamMinSize` in `dsh-fs-local`, `FsTarget.inputPath`, `FsEditOutcome.replacements`/`.replaceAll`, and `FileReadOutcome.limit`/`.version` — while the request-side `replaceAll` (`FsEditRequest`) and the version fields on the other outcome types are untouched; doc pastes and the manifest in sync; the suite is green with the shrunk fakes. +- `formatEditOutput`'s emitted text is unchanged for both `replace_all` branches, so no snapshot golden churns. + +## Risks + +The in-flight fs discovery work (glob/grep tools) touches the same `dsh-fs` type files — a textual, not design, conflict; land in either order and reconcile mechanically. Backends gain no new obligations; they shed four. diff --git a/docs/rfc/implemented/simplification/2026-07-04-remove-agent-steering-mirror.md b/docs/rfc/implemented/simplification/2026-07-04-remove-agent-steering-mirror.md new file mode 100644 index 0000000000..a8a5edd3a2 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-remove-agent-steering-mirror.md @@ -0,0 +1,30 @@ +# RFC: Remove the `agent/steering` mirror emit + +Status: implemented (accepted 2026-07-04) + +## Problem + +`agent/steering` was the last remaining transient mirror of a durable session event. The loop's steering drain appends the durable `steering/message { turn, content, source }` and, on the very next line, emitted `agent/steering(agent, turn, content, source)` — the identical fact as a fire-and-forget event (`packages/core/agent-loop/src/loop.ts`, `drainSteering`). It had zero production listeners: the only subscriber anywhere was a loop regression test asserting the emit carried `source` — the same fact the durable event already records one line above. + +Both mirror-removal RFCs retained it while explicitly deferring the decision this RFC makes. The [boundary-mirror removal](2026-06-20-remove-agent-boundary-mirror-events.md) kept it as a live control signal rather than a boundary; the [stream-chunk removal](2026-07-02-remove-stream-chunk-mirror.md) retained it on the reading that it had no durable twin. The second rationale did not survive the code: the durable twin is `steering/message`, appended immediately before the emit with the same payload. The mirrored-vs-live-only line the taxonomy actually draws puts it on the mirror side: `agent/queued` is genuinely live-only (it fires at enqueue time, before any durable event exists, and already carries a `steering: boolean` flag — cancelled queued work never enters the log), while `agent/steering` fired at the exact moment its durable twin landed, carrying nothing the log does not. + +Steering carries real production traffic — the hook bridges' turn-continuation decisions inject their reasons through `inbox.steer()`, landing as durable `steering/message` events that the hook-matrix goldens pin — and every one of those consumers observes the durable event. Nothing observed the mirror. + +## Decision + +`agent/steering` is removed from the agent event taxonomy: the declaration in `packages/core/agent/src/types.ts` (and its mention in the live-events JSDoc list there), the emit in `drainSteering` (whose then-unused `ctx` parameter went with it), the row in `packages/core/agent/README.md`, and the emit line in the loop-pseudocode blocks (the `packages/core/agent-loop/src/loop.ts` module doc and [architecture.md](../../../architecture.md)); the cordis catalog is regenerated without it. The one regression test pins source preservation on the durable `steering/message` event — the fact it pins lives on the log. + +Three implemented RFCs stated the retention, and each is amended per [implemented/AGENTS.md](../AGENTS.md) to point here as the record of the removal: the [boundary RFC](2026-06-20-remove-agent-boundary-mirror-events.md)'s retained-list entry, the [stream-chunk RFC](2026-07-02-remove-stream-chunk-mirror.md)'s scope clause, and the [event-domain-semantics RFC](../architecture/2026-06-30-event-domain-semantics.md)'s transient-emit enumeration. + +## Why not keep it? + +"It is a control signal, not a boundary" — but the taxonomy's operative distinction is mirrored-vs-live-only, not control-vs-boundary, and this event mirrored. A consumer that wants enqueue-time notification has `agent/queued` (with its steering flag); a consumer that wants drain-time notification is by definition asking for the moment `steering/message` is appended, which `session/event` delivers with the same payload plus durability. The rejected [retire-mid-turn-steering RFC](../../rejected/simplification/2026-06-20-retire-mid-turn-steering.md) defended the steering *capability* — `steer()`, the durable event, continuation forcing — all of which this removal keeps untouched. + +## Acceptance criteria + +- The `agent/steering` spelling survives only in RFC prose (this RFC, the three amended RFCs above, and the frozen [rejected steering-capability RFC](../../rejected/simplification/2026-06-20-retire-mid-turn-steering.md), whose text records the proposal it declined); the catalog is regenerated and fresh. +- The retargeted test pins source preservation on `steering/message`; the suite is green. + +## Risks + +None known: zero production listeners existed to migrate, and both live-notification needs (enqueue, drain) have surviving homes (`agent/queued`, `session/event`). diff --git a/docs/rfc/implemented/simplification/2026-07-04-share-app-bin-boot-glue.md b/docs/rfc/implemented/simplification/2026-07-04-share-app-bin-boot-glue.md new file mode 100644 index 0000000000..faa7b3f106 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-share-app-bin-boot-glue.md @@ -0,0 +1,23 @@ +# RFC: Share the app bins' boot glue instead of maintaining twin copies + +Status: implemented + +## Problem + +`packages/ui/stdio-agent/src/bin.ts` and `packages/ui/acp-agent/src/bin.ts` carried four near-twin helpers — `loadEnv`, `installFailLoud`, `assertEntriesLoaded`, `boot` — whose bodies differed essentially in the diagnostic prefix, plus two copies of the hardest-won boot lore in the repo: the `Promise.allSettled` swallow inside `loader.await()`, the silent-exit-0 import-failure guard, and the `--expose-internals` resolution note. The copies had drifted (`boot(configPath)` resolved the path internally in one bin but required a pre-resolved absolute path in the other, with forked JSDoc prose), and all of it sat outside the per-file 100% gate — `vitest.config.ts` excludes `packages/*/*/src/bin.ts` because importing a self-executing bin runs it — which also made the helpers' `export` keywords decorative: no spec could import them, so the only exercisers were subprocess smokes. + +## Decision + +The helpers live once, in [`@deepseek-ai/dsh-app-boot`](../../../../packages/ui/app-boot) (`packages/ui/app-boot`, in the `ui` group because the bins are published artifacts whose runtime dependency must itself be published, not `support/`): `resolveConfigPath` (snapshot-aware, the single path resolver for both bins), `loadEnv`, `installFailLoud`, `assertEntriesLoaded`, and `boot`, each parameterized by the bin's diagnostic prefix and injectable at its side-effect seams (the warn sink, the process slice) so the unit suite covers every branch — including `boot()` driven in-process against the real Loader with relative-specifier configs, both the settled-tree happy path and the fiber-less-entry rejection. The package carries the per-file 100% coverage gate; the loader-failure lore has one home. + +Each `bin.ts` is a thin self-executing composition over the shared helpers plus its app-specific lifecycle (the ACP bin: replay-mode env skipping and the stdin-EOF dispose; the stdio bin: nothing extra). The bins stay coverage-excluded and export nothing; the published-artifact guards are unchanged — the built-bin smokes still run each bin under plain node in a node_modules-shaped temp dir (now symlinking `ui/app-boot` too) and still assert the missing-config non-zero exit, per the "real entry path means the published artifact" defensive pattern. The [extract-example-app-packages RFC](../architecture/2026-06-20-extract-example-app-packages.md)'s bin-ownership facts are amended accordingly. + +## Why not keep the duplication? + +The bins were framed as independently-owned published artifacts, and a new package carries fixed overhead (manifest, README, tsconfig reference, publint surface) comparable to the deduplicated line count. But app-vs-app sharing was never weighed by the RFC that created the bins — it consolidated three example `start.ts` copies INTO the bins and stopped there; the drift was observed fact; and the coverage-gap argument is independent of the dedup argument: this was the only nontrivial runtime logic in the repo exempt from the per-file 100% gate. The recorded fallback (extracting only the pure logic into per-app modules) would have ended the exemption but kept two homes for the lore. + +## Consequences + +- A boot-glue change (a new guard, a resolution fix) lands once and both published bins inherit it; the bins cannot drift apart again. +- `dsh-app-boot` stays dependency-light (cordis + the loader/include pair) — it is boot machinery, not app surface. +- The bins' own files are near-trivial compositions; everything with branches lives under the coverage gate. diff --git a/docs/rfc/implemented/simplification/2026-07-04-tighten-hook-protocol-contract.md b/docs/rfc/implemented/simplification/2026-07-04-tighten-hook-protocol-contract.md new file mode 100644 index 0000000000..9ee422b130 --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-tighten-hook-protocol-contract.md @@ -0,0 +1,31 @@ +# RFC: Tighten the hook-protocol contract — dialect, discarded fields, double defaults, and lib-owned `hook/result` semantics + +Status: implemented (proposed and accepted 2026-07-04) + +## Problem + +Four pieces of the `dsh-hook-protocol`/bridge contract missed the discipline the [subagent-observe-enrich RFC](../feature/2026-06-30-subagent-observe-enrich.md) records — it dropped an `agentType` lifecycle field for lacking a consumer, and these failed the same test: + +1. **`HookDialect`'s `'native'` variant** (`packages/hooks/hook-protocol/src/types.ts`) had zero producers — the bridges stamp `'claude'` and `'codex'`; the only `'native'` constructor anywhere was the lib's own unit test. The field's own JSDoc defines `dialect` as "the bridge that ran it", and native is not a bridge: the [interception-seams RFC](../feature/2026-06-30-interception-seams.md) records that native hooks are not a package and that "a native plugin can already use the typed Decisions" without the durable hook log, and the flagship native-plugin worked example asserts exactly that (no `hook/*` events at all). +2. **`HookOutput.suppressOutput`** (same file) was parsed by the codec and discarded on every path: no bridge branch, no merge fold, no warn, no deferred-list row — uniquely among its parsed-but-unhonored siblings, each of which carries a stated deferral (`updatedInput` → a logged warn plus the [pre-tool-input-rewrite proposal](../../proposed/feature/2026-06-30-pre-tool-input-rewrite.md); `systemMessage` → a logged warn plus a README deferred row; `continue`/`stopReason` → a `TODO(hook-continue-false)` anchor plus the `'stop'` decision record). Structurally there is nothing to suppress: hook stdout never enters any transcript (context flows only via `additionalContext`; the log records only `decision`/`stderrSummary`), so a hook author setting `suppressOutput: true` got silent nothing with no warn. +3. **`defaultTimeoutMs` was double-defaulted in both bridge configs with a floating literal** — a schema `.default(600_000)` AND a `?? 600_000` fallback (`packages/hooks/hooks-claude/src/index.ts`, `packages/hooks/hooks-codex/src/index.ts`), two homes per bridge for one protocol-level constant, so the bridges could silently drift apart on the shared default. *The proposal's original remedy — delete the knob outright — was overtaken by the no-hardcoded-tunables audit, which kept the knob as the explicit bridge-owned config (and added `stderrSummaryMaxChars` beside it); what remained to fix was the literal's home.* +4. **The `hook/result` semantics lived in the bridges, twice, not in the lib that owns the event.** `summarize()` — the stderr truncation rule — was byte-identical in `packages/hooks/hooks-claude/src/index.ts` and `packages/hooks/hooks-codex/src/index.ts`, and so was the decision-string rule `output.decision ?? (output.continue === false ? 'stop' : 'pass')`; yet `dsh-hook-protocol` declared `hook/result`, documented `stderrSummary` as "truncated" without owning the truncation, and documented the decision values without owning the mapping. If one bridge drifted (a different cap, a different fallback), the shared durable event's semantics would fork silently. + +## What shipped + +`HookDialect` is `'claude' | 'codex'`, its JSDoc names the two bridges, and the lib's unit test constructs a `'codex'` invocation. `suppressOutput` is gone from `HookOutput`, the codec's parse, the codec tests, and the parsed-superset lists in the lib README and the [hook-protocol-lib RFC](../feature/2026-06-30-hook-protocol-lib.md) (amended per [implemented/AGENTS.md](../AGENTS.md)). `hook/result.durationMs` stays: review judged wall-clock hook runtime worth its bytes as durable audit timing (which hook made a turn slow), so `runHook` keeps its injected `now` clock and `RunHookResult` wrapper, the bridges keep passing the measured duration through `HookResultRecord`, and the snapshot normalizer keeps scrubbing the one nondeterministic field to `0` for replay. On the tunables, the no-hardcoded-tunables audit set the shape this change keeps: `defaultTimeoutMs` and `stderrSummaryMaxChars` stay explicit bridge configs, and `RunHookOptions.defaultTimeoutMs` stays a required parameter the bridge passes in. What this change adds is one home per literal: the reference defaults live in the lib as `DEFAULT_HOOK_TIMEOUT_MS` (600 000 ms, exported from the runner) and `DEFAULT_STDERR_SUMMARY_MAX_CHARS` (500, exported from the events module), and both bridges' schema defaults and `??` fallbacks read those constants instead of restating the numbers. The `hook/result` semantics live in the lib: `HookResultRecord` carries the decoded `HookOutput` plus the bridge's `stderrSummaryMaxChars`, and `appendHookResult` derives `stderrSummary` (via the exported `summarizeStderr(stderr, maxChars)`) and the decision string from them; both bridges deleted their private copies, and the derived values are byte-identical to what the bridges wrote (the goldens prove it — their only diff is the dropped `durationMs`). Rider: `BLOCKING_EXIT_CODE` is a codec-internal const, no longer exported (it had zero importers; even the codec tests spell the literal `2`). + +## Why not keep them? + +The [hook-protocol-lib RFC](../feature/2026-06-30-hook-protocol-lib.md) deliberately recorded "parses the full CC superset" — the strongest counterargument was that this proposal re-litigates decisions that RFC records. But parsing a field whose value can never influence anything is not protocol faithfulness, it is a reader trap; a dialect variant that the design's own thesis says will never be stamped is vocabulary without an interpreter; Each returns trivially with its first real consumer (a transcript surface with hook stdout to suppress; a native-provenance feature that logs hook events). On `durationMs` the review reached the opposite verdict: a persistence log is written for future readers, and wall-clock hook timing is audit signal worth carrying before a reader exists — so it stays, with replay normalization as the accepted cost. On item 4, the lib RFC chose per-bridge explicitness over a parameterized engine — but that choice governed payload construction and Decision mapping; the semantics of the SHARED durable event are precisely the "primitives where duplication would actually be dangerous" that the same RFC assigns to the lib. + +## Acceptance criteria + +- `HookDialect` is two-valued; `rg "'native'"` in the hooks packages returns nothing. +- `suppressOutput` appears nowhere in source, parsed-field doc lists, or the normalizer; `durationMs` stays on `hook/result` (and in the fixtures), with the normalizer's replay scrub intact. +- Both bridge configs keep `defaultTimeoutMs`/`stderrSummaryMaxChars` (the audit's explicit-tunables shape), but the literals `600_000` and `500` each live once, in the lib's `DEFAULT_HOOK_TIMEOUT_MS`/`DEFAULT_STDERR_SUMMARY_MAX_CHARS`; per-hook `timeoutSec` still overrides the timeout. +- One definition each of the truncation rule and the decision-string rule, in `dsh-hook-protocol`'s `appendHookResult`, exercised by both bridges' suites. + +## Risks + +The `dialect`, `suppressOutput`, tunables, and semantics changes are invisible on the wire and in the goldens. The cost was churn in `dsh-hook-protocol` and both bridges — cheap under the pre-release stance, and cheaper than letting two copies of a durable event's semantics age apart. diff --git a/docs/rfc/implemented/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md b/docs/rfc/implemented/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md new file mode 100644 index 0000000000..3df0e0322a --- /dev/null +++ b/docs/rfc/implemented/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md @@ -0,0 +1,22 @@ +# RFC: Trim unreachable ACP bridge surface — the branding knobs and the kind-sniffing fallback + +Status: implemented (accepted 2026-07-04) + +## Problem + +Two pieces of `dsh-acp` surface were unreachable from any shipped configuration: + +1. **`AcpConfig.agentName` / `agentVersion`** (`packages/ui/acp/src/index.ts`). The shipped app package hands the bridge only `{ model, systemPrompt }` (`packages/ui/acp-agent/src/index.ts`), so no leaf `cordis.yml` — the only production config surface — could set the knobs at all; they were settable solely by direct-mounting the bridge, which only a unit test did. Every snapshot golden — the hook-matrix scenarios included — pins the schema defaults (`deepseek-harness-acp` / `0.0.1`). The pair also carried a live `TODO(double-default)`: the literals existed twice (schema `.default(...)` plus `??` fallbacks), with the TODO asking to pick one home. +2. **The `toolKindFor` name heuristic** (same file) special-cased `bash*`/`read*`/`write`/`edit*` tool names in the generic-fallback path. Since the [render-intent union](../architecture/2026-07-02-tool-render-intent-union.md), every first-party tool those arms matched ships its own `presentCall` carrying its kind, and the presenter-less production tools (`subagent`, `subagent_fork`) fell through to `other` anyway. The arms were production-reachable only when a tool declined to present its own call — a `presentCall` that THROWS (the containment fallback), or model arguments that fail the tool's schema so `defineTool`'s `presentCall` wrapper returns `undefined` (e.g. a `bash` call missing the required `description`) — and the bridge's own module doc states the design rule the heuristic violated: "the bridge never special-cases tool names". + +## Decision + +`agentInfo` is hardcoded at the `initialize` site (`{ name: 'deepseek-harness-acp', version: '0.0.1' }`); the two config fields, their schema defaults, the `??` fallbacks, and the `TODO(double-default)` (whose subject vanished with them) are gone, along with the knob half of the direct-mount config test, the two config rows in `packages/ui/acp/README.md`, and the `packages/ui/acp/acp-feature-support.md` cells that described the knobs and the name inference. The emitted handshake wire value is unchanged — zero golden churn on the branding half. `toolKindFor` is replaced by the constant `'other'` at both fallback sites (the presenter fallback and `nullToolPresenter`), and the heuristic is deleted with its test rows. The fixed handshake identity stays pinned by the bridge's initialize unit test and by every snapshot golden. On the fallback half the transcript delta shows up in exactly one committed golden: `hook-codex-posttool-block`, whose recorded model omits the required `description` on three `bash` calls, so those cards take the declined-to-present fallback and carry `kind: 'other'` — the honest neutral card for a call the tool would not vouch for. + +## Why not keep them? + +`agentInfo` is client-visible branding a deployment will eventually want configurable — but a knob no shipped config can reach is not configurability, it is drift surface (the double-default TODO was its symptom), and the honest re-add must include the `dsh-acp-agent` plumb-through that does not exist either; both arrive together with the deployment that needs them. For the heuristic: a hypothetical third-party presenter-less tool named `read_docs` loses an inferred `read` icon — but inferring kinds from unknown plugins' names is exactly the special-casing the render-intent design rejected. The only shipped paths the heuristic reached were the declined-to-present fallbacks (a throwing `presentCall`, or schema-invalid model args); rendering kind `other` there makes the client show the raw input instead of a masquerading first-party card — strictly better diagnostics for a broken presenter or a malformed call. + +## Risks + +None beyond the fallback rendering trade described above — degenerate paths whose neutral card is more diagnosable than an inferred first-party one. diff --git a/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md b/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md index 7f6cd47b48..cd7058487f 100644 --- a/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md +++ b/docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md @@ -46,7 +46,7 @@ Replay is positional: the Nth `stream()` call serves the Nth `ReplayEntry`. This Recording runs the scenario with the real `llm-deepseek` adapter and the JSONL persistence backend, then copies the produced `.jsonl` into the scenario dir. Per-event appends are durable, but the harness shuts the subprocess down gracefully (close stdin → `await ctx.dispose()`) before harvesting so the final events are flushed. `llm-replay` itself does no recording — it is replay-only. -The ACP server app loads `@deepseek-ai/dsh-llm-deepseek`, whose `apply` throws when no API key is present ([packages/llm/llm-deepseek/src/index.ts](../../../../packages/llm/llm-deepseek/src/index.ts)). So replay cannot reuse the normal config — it uses a dedicated `examples/acp-agent/cordis.snapshot.yml` that installs `llm-replay` in place of the adapter. The rest of the tree is not duplicated: both the normal `examples/acp-agent/cordis.yml` and the replay config load the same `@deepseek-ai/dsh-acp-agent` app entry (which bundles the agent-core spine + JSONL persistence + the ACP bridge), differing only in the LLM backend (`llm-deepseek` vs `llm-replay`) and the bash executor line. Recording reuses the normal `cordis.yml` (real adapter) — its persistence root reads `$DSH_SNAPSHOT_SESSIONS_ROOT` when the harness sets it — so there is no separate record config. The `dsh-acp-agent` bin selects `cordis.snapshot.yml` for `DSH_SNAPSHOT=replay` and skips `.env` loading in that mode so a stray key cannot trigger a live call. +The ACP server app loads `@deepseek-ai/dsh-llm-deepseek`, whose `apply` throws when no API key is present ([packages/llm/llm-deepseek/src/index.ts](../../../../packages/llm/llm-deepseek/src/index.ts)). So replay cannot boot the normal config as-is — `examples/acp-agent/cordis.snapshot.yml` is an include-overlay of `cordis.yml` that disables the `llm-deepseek` entry by id and inserts `llm-replay` (see [single-source the acp-agent replay config](2026-07-04-single-source-acp-replay-config.md)); every other entry IS the live tree, loaded through the include. Recording reuses the normal `cordis.yml` (real adapter) — its persistence root reads `$DSH_SNAPSHOT_SESSIONS_ROOT` when the harness sets it — so there is no separate record config. The `dsh-acp-agent` bin selects `cordis.snapshot.yml` for `DSH_SNAPSHOT=replay` and skips `.env` loading in that mode so a stray key cannot trigger a live call. ### Two surfaces: normalize, then compare diff --git a/docs/rfc/implemented/testing/2026-07-04-hook-snapshot-matrix.md b/docs/rfc/implemented/testing/2026-07-04-hook-snapshot-matrix.md index f8ec029d22..ad61bd3b84 100644 --- a/docs/rfc/implemented/testing/2026-07-04-hook-snapshot-matrix.md +++ b/docs/rfc/implemented/testing/2026-07-04-hook-snapshot-matrix.md @@ -18,7 +18,7 @@ Two coupled changes, in one PR: It is safe because a bridge whose config file is absent is a **silent no-op**: `apply()` catches the read failure, logs through `ctx.logger`, and registers nothing — zero listeners, zero session events. The `acp-agent` app ships no stdout logger, so the warning cannot reach the ACP JSON-RPC channel. A scenario (or a real project) that wants only Claude hooks ships only `hooks.json`; the Codex bridge sees no `codex-hooks.json` and vanishes. This was verified empirically: with both bridges loaded, all pre-existing snapshots (none of which ship a `codex-hooks.json`) are byte-identical. -Loading both is the minimum that lets the snapshot tier exercise each dialect against the same real app the product ships. Recording (which boots `cordis.yml`) must load both too, so a recorded Codex scenario captures the transcript with its hook genuinely active — hence the symmetric edit to both configs. +Loading both is the minimum that lets the snapshot tier exercise each dialect against the same real app the product ships. Recording (which boots `cordis.yml`) loads both by construction, and replay inherits them the same way: `cordis.snapshot.yml` is an include-overlay of `cordis.yml` that swaps only the llm entry (see [single-source the acp-agent replay config](2026-07-04-single-source-acp-replay-config.md)), so a bridge added to the live tree is in the replay tree with no second edit. ### 2. A snapshot scenario per hook point × its headline outcome, both dialects diff --git a/docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md b/docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md new file mode 100644 index 0000000000..c5f836b9f7 --- /dev/null +++ b/docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md @@ -0,0 +1,23 @@ +# RFC: Single-source the acp-agent replay config + +Status: implemented + +## Problem + +`examples/acp-agent` shipped two hand-maintained configs: `cordis.yml` (the live tree) and a `cordis.snapshot.yml` that mirrored it entry-for-entry with only the llm backend swapped — stripped of comments, the entire difference was the eight-line `llm-deepseek` stanza versus the two-line `llm-replay` stanza. Every app-shape change had to be made twice, and nothing gated the symmetry: if the copies drifted, the snapshot tier would silently exercise a different app than the one that ships — the ["green units, broken product" class of gap](../../../postmortem/0001-acp-default-export-drops-inject.md) the snapshot tier exists to close, reintroduced one level up, with reviewer vigilance as the only defense. + +## Decision + +`cordis.snapshot.yml` is a declarative overlay, not a copy: its single entry mounts `@cordisjs/plugin-include` on `./cordis.yml` with `patches` that disable the `llm-deepseek` entry (matched by id AND asserted by `name`, so a reused id can never disable the wrong plugin) and insert the `llm-replay` entry ([the vendored include plugin](../../../../vendor/include/src/index.ts)'s patch mechanism: by-id overrides with an optional name assertion, plus top-level inserts). Every other entry — the app, the bash executor, the fs/subagent/todo tools, both hook bridges, the system prompt — is the live tree itself, loaded through the include, so replay exercises exactly what ships and an app-shape change lands once. The `dsh-acp-agent` bin is untouched (it still just selects this file for `DSH_SNAPSHOT=replay`); recording still boots `cordis.yml` directly; the bin's `assertEntriesLoaded` guard tolerates the disabled entry by design (a disabled entry is the one legitimate fiber-less state). + +One vendored-plugin fact the overlay depends on, deliberately: the include applies `patches` when it loads the file — its `refresh()`/`internal/update` paths re-read without re-patching — which is exactly enough for a one-shot replay boot (the replay app loads no `hmr` and nothing rewrites the config mid-run). The snapshot suite is the proof: all scenarios pass unchanged on the overlay, byte-identical goldens included. + +## Why not the alternatives? + +Keeping the full twin with a symmetry verify-gate was the recorded fallback — it would have removed the silent-drift class but kept a 125-line near-copy whose only content was one entry's difference, growing with every plugin the app gains. A bin-side swap (parse the config, replace the entry, delete the file) would have put YAML surgery inside a published artifact and moved the replay delta out of sight; the overlay keeps the delta declarative, readable, and next to the base config — the teaching value the twin's defenders actually wanted. + +## Consequences + +- A plugin added to `cordis.yml` is in the replay tree with no second edit; the drift class is structurally gone rather than gated. +- The overlay depends on entries carrying stable `id:`s. The `name` assertion on the disable patch guards mis-targeting (a reused id skips the patch instead of disabling the wrong plugin). An id RENAME degrades the patch to a skip whose warning needs a logger the replay app deliberately lacks — the observable result is a futile keyless `llm-deepseek` entry alongside `llm-replay`, with replay output still correct (`llm-replay` owns the stream short-circuit); config rot for review to catch, not wrong snapshots. A top-level insert whose id collides with an existing entry resolves last-wins through the loader's id map — the current config has no collision, and a new patch line is where one would be introduced. +- If a future replay tree needs a second divergence (another backend swapped), it is one more patch line, not a second fork of the file. diff --git a/docs/rfc/proposed/simplification/2026-07-04-drop-image-content-block.md b/docs/rfc/proposed/simplification/2026-07-04-drop-image-content-block.md deleted file mode 100644 index e9144cc3aa..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-drop-image-content-block.md +++ /dev/null @@ -1,27 +0,0 @@ -# RFC: Drop the `image` content block until a path can honor it - -Status: proposed - -## Problem - -`ImageBlock` (`packages/llm/llm/src/types.ts`) has no production producer, and every consumer on every path DROPS it: the deepseek adapter's serializer skips image blocks (a documented MVP limitation), the pi-ai converter skips them as unrepresentable, the ACP codec neither advertises image prompt capability nor forwards image blocks outbound and REJECTS image prompt content inbound, and the compaction estimator charges a flat token constant and renders `[image]`. An `ImageBlock` constructed today would silently vanish from the wire — the vocabulary advertises a capability no path honors, which is the silent-data-loss shape AGENTS.md's defensive patterns warn against. The only constructors anywhere are tests pinning the skip/drop/estimate branches. - -## Proposal - -Remove `ImageBlock`, its `ContentBlockMap` entry, the explicit `image` estimate/placeholder arms in compact-basic, and the image-naming comments in the deepseek serializer's, pi-ai converter's, and ACP codec's default arms — those default arms already absorb the case the way they absorb any unknown block type. Update the vocabulary line in [architecture.md](../../../architecture.md), the block list in `packages/llm/llm/README.md`, the deepseek README's image-skip row, the pi-ai README's images-not-representable row, the compact-basic README's image-estimation and `[image]`-placeholder rows, the pastes in [core.md](../../../core-data-structures/core.md) and [llm-streaming.md](../../../core-data-structures/llm-streaming.md), and the type-equiv manifest; amend the [content-block vocabulary RFC](../../implemented/architecture/2026-06-11-content-block-vocabulary.md)'s block list and multimodal-home consequence per [implemented/AGENTS.md](../../implemented/AGENTS.md); drop or retarget the tests that construct image blocks to exercise the removed branches. The ACP codec's inbound rejection of image PROMPT content is unaffected — that guard is about protocol content a client can send regardless of our vocabulary, and it stays. - -## Why not keep it? - -This is the most contested cut in the batch. Multimodal input (screenshots) is a plausible near-term coding-agent feature, and the [content-block vocabulary RFC](../../implemented/architecture/2026-06-11-content-block-vocabulary.md) reserved the slot deliberately. Two responses. First, `ContentBlockMap` is merge-extensible by design: a real multimodal feature reintroduces `image` in core in the same coordinated change that maps it in the adapters, advertises and renders it in ACP, and prices it in compaction — the producer and its consumers arrive together, which is how the map is meant to grow. Second, the middle option — keep the type but make adapters throw UNSUPPORTED instead of silently dropping — converts this into exactly the shape the [request-knobs RFC](2026-07-04-drop-inert-request-knobs.md) argues against: surface whose only implementation is rejection. Absence (a compile error at the would-be producer) is strictly clearer than either silent loss or universal throw. - -If review lands on keeping the slot, the fallback this RFC records is: keep `ImageBlock` but replace every silent skip with a loud rejection, and document that policy in the vocabulary — the current silent drop is the one state with no defender. - -## Acceptance criteria - -- No `ImageBlock` / harness `type: 'image'` block construction outside this RFC; the codec's inbound ACP-image rejection still passes its tests. -- Adapter/codec/compaction switches handle the case through their unknown-block default arms (pinned by the existing plugin-added-block tests where present). -- Doc pastes, the manifest, and the architecture vocabulary list updated; `pnpm run doc-sync` green. - -## Risks - -Re-adding a core vocabulary type later touches several packages at once — but that coordinated change is the shape a real multimodal feature needs anyway (adapter mapping, ACP advertisement, compaction pricing), and none of it exists today to preserve. diff --git a/docs/rfc/proposed/simplification/2026-07-04-drop-inert-request-knobs.md b/docs/rfc/proposed/simplification/2026-07-04-drop-inert-request-knobs.md deleted file mode 100644 index 60375ecf8c..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-drop-inert-request-knobs.md +++ /dev/null @@ -1,33 +0,0 @@ -# RFC: Drop `GenerateOptions.prefill` and `ToolSchema.strict` — request knobs with no working end-to-end path - -Status: proposed - -## Problem - -Two request-contract knobs ride the whole request pipeline, yet neither can do anything today: - -- **`prefill`** (`packages/llm/llm/src/types.ts`) has no production setter — the loop assembles `model`/`system`/`tools`/`messages` plus `sessionId`/`signal`, and the compaction backend adds only `maxTokens` — and BOTH adapters reject it: `packages/llm/llm-deepseek/src/serialize.ts` and `packages/llm/llm-pi-ai/src/adapter.ts` each throw `LlmError('UNSUPPORTED')` on a non-undefined `prefill`. The field's entire observable behavior is two throws, each pinned by one adapter test. DeepSeek's chat-prefix completion is a Beta feature on a base URL neither adapter targets. -- **`strict`** (`ToolSchema`, same file) is threaded through `DefineToolOptions`/`defineTool` (`packages/core/tools/src/schema.ts`), the registry's `schemas()` allowlist (`packages/core/tools/src/index.ts`), the deepseek wire mapping (`packages/llm/llm-deepseek/src/serialize.ts`, whose wire-type note records that strict mode requires the `/beta` base URL the adapter does not use), and a per-tool payload-patching pass in `packages/llm/llm-pi-ai/src/adapter.ts`. No shipped tool sets it — `rg` across every `tool-*` package src and `examples/` finds zero `strict:` producers; the only setters are dsh-tools unit tests. - -Both knobs are adapter-symmetric, so removal sheds them from both twins together — the [twin-adapter design](../../implemented/architecture/2026-06-13-twin-llm-adapters.md) is untouched. - -## Proposal - -- Remove `prefill` from `GenerateOptions`, both adapters' UNSUPPORTED guards, the tests pinning the throws, the paste lines in [core.md](../../../core-data-structures/core.md), the adapter README rows documenting the rejection, and the cookbook line using prefill as the UNSUPPORTED example ([adding-an-llm-adapter.md](../../../cookbook/adding-an-llm-adapter.md)); amend the [content-block vocabulary RFC](../../implemented/architecture/2026-06-11-content-block-vocabulary.md)'s consequence line naming prefill as having a home, per [implemented/AGENTS.md](../../implemented/AGENTS.md). -- Remove `strict` from `ToolSchema`, `DefineToolOptions`, `defineTool`, and the `schemas()` allowlist; drop the deepseek serializer branch; simplify the pi-ai payload fixup to the unconditional scrub of pi-ai's own strict default (that half exists for wire parity with the hand-rolled twin and survives); drop the setter tests and the core.md paste line. - -This RFC deliberately does NOT touch `temperature`, `stop`, or `maxTokens`: those are honored end-to-end by both adapters and are the natural first targets of a request-mutating hook plugin on `agent/request`. - -## Why not keep them? - -"An explicit UNSUPPORTED throw is honest contract behavior" — but a knob whose only implementation across both twins is rejection promises nothing, and deleting it upgrades the failure mode: an accidental setter becomes a compile error instead of a runtime throw. "Strict schema adherence is an officially documented provider feature with complete plumbing" — but a knob is not product surface until a shipped tool sets it AND an endpoint honors it; today neither is true. Each returns with its first real producer: `prefill` together with an adapter that implements chat-prefix completion (and a stated policy for adapters that do not), `strict` together with a tool that wants it and a beta-endpoint story. - -## Acceptance criteria - -- `rg prefill` and a tool-schema-scoped `rg strict` return only this RFC (and unrelated prose such as `strictEqual`). -- Both adapters compile and their contract tests pass without the guards; the pi-ai fixup still scrubs the library's strict default (wire parity pinned by its serializer tests). -- Doc pastes and the type-equiv manifest in sync; `pnpm run doc-sync` green. - -## Risks - -The shipped hook bridges set no request fields at all, and a request-mutating plugin (an `agent/request` waterfall listener) would reach for `temperature`/`stop` (kept, working), not a field adapters reject. If chat-prefix completion or strict mode become product features, the re-add lands with the adapter/endpoint work, where the contract can say what actually happens rather than "everyone throws". diff --git a/docs/rfc/proposed/simplification/2026-07-04-prune-producerless-vocabulary-variants.md b/docs/rfc/proposed/simplification/2026-07-04-prune-producerless-vocabulary-variants.md deleted file mode 100644 index d967e1d3b2..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-prune-producerless-vocabulary-variants.md +++ /dev/null @@ -1,31 +0,0 @@ -# RFC: Prune producer-less vocabulary variants (block cache hints, the `agent` message source, the `continuation` turn trigger) - -Status: proposed - -## Problem - -The merge-extensible vocabulary maps are designed to grow by declaration merging, and the codebase already states the admission policy on `TurnEndReasonMap` (`packages/core/session/src/types.ts`): a variant like `refusal` is "deliberately omitted until" an adapter or loop first emits it. Three declared vocabulary items violate that policy — each has no producer and no consumer, and two have not even a test: - -- **`CacheHint` and the three `cache?: CacheHint` fields** on `TextBlock`/`ToolResultBlock`/`ImageBlock` (`packages/llm/llm/src/types.ts`). Nothing constructs a block with `cache:` anywhere — src, tests, and doc pastes all come up empty — and neither adapter reads `.cache`: DeepSeek prompt caching is automatic, so the adapters map `prompt_cache_hit_tokens` OUT of responses without ever sending a hint IN. This is Anthropic-style `cache_control` surface with no provider that can honor it. -- **`MessageSourceMap.agent`** (`{ kind: 'agent'; agentId: string }`, same file). Zero constructors, tests included. Its intended producer shipped without it: the subagent backends send the parent's prompt to the child with no `source`, so it logs as `{ kind: 'user' }`, and the generic envelope renderer interpolates `source.kind` without ever routing on it. The variant is pasted into [core.md](../../../core-data-structures/core.md). -- **`TurnTriggerMap.continuation`** (`packages/core/session/src/types.ts`). The loop structurally cannot emit it — continuation happens *within* a turn as further steps, never as a new turn — and it constructs only `message` and `injection` triggers. The only writer is one hand-built test fixture that needs an arbitrary non-message trigger (`packages/support/llm-replay/tests/llm-replay.spec.ts`); the only production trigger reader, the ACP bridge, filters on `kind === 'message'`. The variant is pasted into [session.md](../../../core-data-structures/session.md). - -## Proposal - -Delete `CacheHint` with its three `cache?` fields, the `agent` message-source variant, and the `continuation` turn-trigger variant. Switch the llm-replay fixture to an `injection` trigger (any non-`message` trigger serves its purpose). Update the type-equiv pastes in [core.md](../../../core-data-structures/core.md) and [session.md](../../../core-data-structures/session.md) (and `scripts/type-equiv.manifest.json` where block identity shifts) in the same change, and amend the [content-block vocabulary RFC](../../implemented/architecture/2026-06-11-content-block-vocabulary.md)'s consequence line naming cache hints as having a home, per [implemented/AGENTS.md](../../implemented/AGENTS.md). - -Each variant returns the day it gains a real producer, exactly as the maps are designed to grow: a caching feature re-adds `cache` together with the adapter that transmits it; subagent attribution re-adds `agent` together with the backend that stamps it and a consumer that routes on it; an auto-continue feature that genuinely starts new turns re-adds `continuation` with the plugin that emits it. - -## Why not keep them? - -The [content-block vocabulary RFC](../../implemented/architecture/2026-06-11-content-block-vocabulary.md) lists "cache hints … have a home" as a design consequence, and reserved slots do advertise intent. But an empty slot is contract surface every implementation and consumer must consider (must my adapter honor `cache`? must my renderer route `agent` sources?), and the sibling map's own JSDoc already rejects reservation-without-emitter — `refusal` and `max_turn_requests` are named as variants to add *when something first emits them*, not declared in advance. Holding already-declared dead variants to the same standard makes the vocabulary mean something: if it is in the map, something produces it. - -## Acceptance criteria - -- `rg` for `CacheHint`, the `agent` message-source spelling, and the `continuation` trigger spelling returns only this RFC. -- The core-data-structures pastes and the type-equiv manifest are in sync (`pnpm run doc-sync` green). -- The fixture asserts the same replay behavior with an `injection` trigger; the suite is green. - -## Risks - -None operational — nothing can construct these values today. The mirror-event removals (recorded in [the boundary-mirror RFC](../../implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md) and [the stream-chunk RFC](../../implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md)) touch only transient `agent/*` events, never the durable vocabulary, so there is no collision. Elsewhere in the vocabulary the admission policy already holds: `rejected`, `prompt/blocked`, and `hook/invoked`/`hook/result` each have live producers — this RFC extends the same bar to the three variants that lack one. If the [image-block RFC](2026-07-04-drop-image-content-block.md) ships first, one of the three `cache?` fields leaves with it; the two proposals are independent and compose in either order. diff --git a/docs/rfc/proposed/simplification/2026-07-04-prune-write-only-fs-surface.md b/docs/rfc/proposed/simplification/2026-07-04-prune-write-only-fs-surface.md deleted file mode 100644 index 604c1774d5..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-prune-write-only-fs-surface.md +++ /dev/null @@ -1,29 +0,0 @@ -# RFC: Prune write-only fields and a dead routing knob from the fs seam - -Status: proposed - -## Problem - -The [fs seam split](../../implemented/simplification/2026-06-26-fsspec-style-fs-seam.md) moved read routing and policy out of the backend into `dsh-tool-fs` and `dsh-fs-policy`. Four pieces of surface kept the pre-split shape — populated on every call, read by nobody: - -1. **`STREAM_MIN_SIZE` + `FsIoInternals.streamMinSize` in `dsh-fs-local`** — *already removed by the no-hardcoded-tunables audit (the routing bound became `dsh-tool-fs`'s `readStreamMinSize` config); listed here for the record of the full prune, no work remains.* Originally (`packages/fs/fs-local/src/fsio.ts`, re-exported from `packages/fs/fs-local/src/index.ts`): zero readers anywhere, including fs-local's own source and tests. The backend has no read routing — `readWholeText`/`streamWholeText` are separate primitives the caller chooses between — and the real routing constant lives in the consumer (`packages/fs/tool-fs/src/read.ts`, compared against `info.size`). Two mirrors of the 10 MiB fact; the backend's is dead, and the knob's JSDoc claims a "read routing" override that does not exist. -2. **`FsTarget.inputPath`** (`packages/fs/fs/src/types.ts`): every backend and every test fake must fabricate a "diagnostics only" value with zero production readers — the policy plugin and every error message use `targetKey`/`displayPath`. The `listDir` producer exposes the semantic wobble: directory children get the bare entry name, which was nobody's "input". -3. **`FsEditOutcome.replacements` + `.replaceAll`** (`packages/fs/fs/src/types.ts`): `replacements` has zero production readers (the single-match policy itself stays — it is enforced by the `FS_AMBIGUOUS_EDIT`/`FS_EDIT_NOT_FOUND` throws inside the backend, whose error message keeps the internal count); `replaceAll` is read only by `formatEditOutput` in `packages/fs/tool-fs/src/edit.ts` — as an echo of the `replace_all` argument the tool already holds. Shrunk, `FsEditOutcome` becomes `{ version, before, after }`, parallel to `FsWriteOutcome`'s genuinely backend-discovered fields. -4. **`FileReadOutcome.limit` + `.version`** (`packages/fs/tool-fs/src/read-render.ts`): populated by the read tool, but `formatReadOutput` renders `offset`/`lines`/`totalLines`/`truncatedByBytes` only, and the `fs/observed` emit uses `info.version` directly rather than the outcome copy. - -## Proposal - -Delete the fs-local constant, its re-export, and the `streamMinSize` knob (the remaining `FsIoInternals` knobs are genuinely used by the atomic-write tests); drop `inputPath` from `FsTarget`; shrink `FsEditOutcome` to `{ version, before, after }` and pass `replaceAll` to `formatEditOutput` from the parsed args; drop `limit`/`version` from `FileReadOutcome`. Update the [filesystem.md](../../../core-data-structures/filesystem.md) pastes, the type-equiv manifest, `packages/fs/fs/README.md`, and the test fakes that currently must fabricate the removed fields. - -## Why not keep them? - -A future permission/containment layer might want the pre-resolution path for error text — but it would want the *request*, which every call site still holds. "N occurrences replaced" might become model-facing text — a behavior change to design when wanted, and the backend-internal count survives for its error message. A read footer might display `limit` — everything the footer shows already derives from `lines`/`totalLines`. Meanwhile every current and future backend (remote, native) must fabricate wire fields nobody consumes, and every test fake must satisfy them. - -## Acceptance criteria - -- The removed surfaces are gone — `STREAM_MIN_SIZE`/`streamMinSize` in `dsh-fs-local`, `FsTarget.inputPath`, `FsEditOutcome.replacements`/`.replaceAll`, and `FileReadOutcome.limit`/`.version` — while the request-side `replaceAll` (`FsEditSpec`) and the version fields on the other outcome types are untouched; doc pastes and the manifest in sync; the suite is green with the shrunk fakes. -- `formatEditOutput`'s emitted text is unchanged for both `replace_all` branches, so no snapshot golden churns. - -## Risks - -The in-flight fs discovery work (glob/grep tools) touches the same `dsh-fs` type files — a textual, not design, conflict; land in either order and reconcile mechanically. Backends gain no new obligations; they shed four. diff --git a/docs/rfc/proposed/simplification/2026-07-04-remove-agent-steering-mirror.md b/docs/rfc/proposed/simplification/2026-07-04-remove-agent-steering-mirror.md deleted file mode 100644 index 579401cb75..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-remove-agent-steering-mirror.md +++ /dev/null @@ -1,28 +0,0 @@ -# RFC: Remove the `agent/steering` mirror emit - -Status: proposed - -## Problem - -`agent/steering` is the last remaining transient mirror of a durable session event. The loop's steering drain appends the durable `steering/message { turn, content, source }` and, on the very next line, emits `agent/steering(agent, turn, content, source)` — the identical fact as a fire-and-forget event (`packages/core/agent-loop/src/loop.ts`, `drainSteering`). It has zero production listeners: the only subscriber anywhere is a loop regression test asserting the emit carries `source` — the same fact the durable event already records one line above. - -Both mirror-removal RFCs retained it while explicitly deferring the decision this RFC now makes. The [boundary-mirror removal](../../implemented/simplification/2026-06-20-remove-agent-boundary-mirror-events.md) kept it as "a live control signal, not a boundary"; the [stream-chunk removal](../../implemented/simplification/2026-07-02-remove-stream-chunk-mirror.md) kept it as "a live control signal with no durable twin, retained (its fate is a separate future decision)". The second rationale does not survive the code: the durable twin is `steering/message`, appended immediately before the emit with the same payload. The mirrored-vs-live-only line the taxonomy actually draws puts it on the mirror side: `agent/queued` is genuinely live-only (it fires at enqueue time, before any durable event exists, and already carries a `steering: boolean` flag — cancelled queued work never enters the log), while `agent/steering` fires at the exact moment its durable twin lands, carrying nothing the log does not. - -Steering carries real production traffic — the hook bridges' turn-continuation decisions inject their reasons through `inbox.steer()`, landing as durable `steering/message` events that the hook-matrix goldens pin — and every one of those consumers observes the durable event. Nothing observes the mirror. - -## Proposal - -Remove the `agent/steering` declaration from `packages/core/agent/src/types.ts` (and its mention in the live-events JSDoc list there), the emit in `drainSteering` (whose `ctx` parameter becomes unused and goes too), the row in `packages/core/agent/README.md`, and the emit line in the loop-pseudocode blocks (`packages/core/agent-loop/src/loop.ts` module doc and [architecture.md](../../../architecture.md)); run `pnpm run gen-cordis-catalog`. Retarget the one regression test at the durable `steering/message` event — the source-preservation fact it pins lives on the log. The implementing PR amends the two retaining RFCs' scope lines per [implemented/AGENTS.md](../../implemented/AGENTS.md): the boundary RFC's retained-list entry and the stream-chunk RFC's "no durable twin" clause. - -## Why not keep it? - -"It is a control signal, not a boundary" — but the taxonomy's operative distinction is mirrored-vs-live-only, not control-vs-boundary, and this event mirrors. A consumer that wants enqueue-time notification has `agent/queued` (with its steering flag); a consumer that wants drain-time notification is by definition asking for the moment `steering/message` is appended, which `session/event` delivers with the same payload plus durability. The rejected [retire-mid-turn-steering RFC](../../rejected/simplification/2026-06-20-retire-mid-turn-steering.md) defended the steering *capability* — `steer()`, the durable event, continuation forcing — all of which this removal keeps untouched. - -## Acceptance criteria - -- No `agent/steering` spelling outside this RFC and the two amended RFCs; the catalog is regenerated and fresh. -- The retargeted test pins source preservation on `steering/message`; the suite is green. - -## Risks - -None known: zero production listeners exist to migrate, and both live-notification needs (enqueue, drain) have surviving homes (`agent/queued`, `session/event`). diff --git a/docs/rfc/proposed/simplification/2026-07-04-share-app-bin-boot-glue.md b/docs/rfc/proposed/simplification/2026-07-04-share-app-bin-boot-glue.md deleted file mode 100644 index adc3ec6da3..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-share-app-bin-boot-glue.md +++ /dev/null @@ -1,27 +0,0 @@ -# RFC: Share the app bins' boot glue instead of maintaining twin copies - -Status: proposed - -## Problem - -`packages/ui/stdio-agent/src/bin.ts` and `packages/ui/acp-agent/src/bin.ts` carry four near-twin helpers — `loadEnv`, `installFailLoud`, `assertEntriesLoaded`, `boot` — whose bodies differ essentially in the diagnostic prefix, plus two copies of the hardest-won boot lore in the repo: the `Promise.allSettled` swallow inside `loader.await()`, the silent-exit-0 import-failure guard, and the `--expose-internals` resolution note (the failure classes behind AGENTS.md's "real entry path means the published artifact" pattern). Drift has already begun: `boot(configPath)` resolves the path internally in one bin but requires a pre-resolved absolute path in the other, and the twin JSDoc prose has forked. - -The duplication is aggravated by a coverage hole: all of this logic sits OUTSIDE the per-file 100% gate — `vitest.config.ts` excludes `packages/*/*/src/bin.ts` because importing a self-executing bin (top-level `await main()`) runs it — which also makes the `export` keywords on these helpers decorative: no spec can import them, so the only exercisers are the subprocess smokes, and the two `built-bin.e2e.ts` suites duplicate their temp-node_modules scaffolding as well. The genuinely per-app pieces are small and real: the ACP bin owns snapshot-mode config selection (`resolveConfigPath`), replay-mode env skipping, the stdin-EOF dispose lifecycle, and stdout purity; the stdio bin owns nothing extra. - -## Proposal - -Extract the four helpers, parameterized by the bin's diagnostic prefix, into an importable non-bin module shared by both apps — a small published package in the `ui` group (the bins are published artifacts, so their runtime dependency must be published too, not `support/`). Each `bin.ts` becomes a thin self-executing `main()` plus its app-specific glue. The shared module gains unit tests and falls under the coverage gate; the loader-failure lore gets one home; the subprocess smokes remain the artifact-level guard — the published-bin smoke is NOT replaced by unit tests, per the "real entry path" defensive pattern. The implementing PR amends the [extract example app packages RFC](../../implemented/architecture/2026-06-20-extract-example-app-packages.md)'s facts ("boot glue moved into that bin, owned by the app" is the sentence that changes). - -## Why not keep the duplication? - -The bins were framed as independently-owned published artifacts, and a new package carries fixed overhead (manifest, README, tsconfig reference, publint surface) that rivals the deduplicated line count. But app-vs-app sharing was never weighed by that RFC — it consolidated three example `start.ts` copies INTO the bins and stopped there; the drift is now observed fact rather than speculation; and the coverage-gap argument is independent of the dedup argument: this is the only nontrivial runtime logic in the repo exempt from the per-file 100% gate. The alternative of a copy-by-convention shared source file is the current state with extra steps. - -## Acceptance criteria - -- The four helpers exist once, unit-tested, under the coverage gate; both bins are thin mains plus app-specific glue. -- Both built-bin smokes still pass under plain node in the node_modules-shaped temp dir, including the missing-config non-zero exit. -- The app-packages RFC's facts are amended in the same change. - -## Risks - -Churn in two published bins and one new package boundary; the shared module must stay dependency-light (cordis plus the loader). If the implementing PR finds the package overhead genuinely exceeds the dedup — the honest failure mode of this proposal — the fallback that still pays is extracting only the coverage-exempt pure logic (`assertEntriesLoaded`, `resolveConfigPath`) into an importable module within each app package, ending the coverage exemption without a new package. diff --git a/docs/rfc/proposed/simplification/2026-07-04-tighten-hook-protocol-contract.md b/docs/rfc/proposed/simplification/2026-07-04-tighten-hook-protocol-contract.md deleted file mode 100644 index 42d222c6be..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-tighten-hook-protocol-contract.md +++ /dev/null @@ -1,32 +0,0 @@ -# RFC: Tighten the hook-protocol contract — dialect, discarded fields, double defaults, and lib-owned `hook/result` semantics - -Status: proposed - -## Problem - -Five pieces of the `dsh-hook-protocol`/bridge contract miss the discipline the [subagent-observe-enrich RFC](../../implemented/feature/2026-06-30-subagent-observe-enrich.md) records — it dropped an `agentType` lifecycle field for lacking a consumer, and these fail the same test: - -1. **`HookDialect`'s `'native'` variant** (`packages/hooks/hook-protocol/src/types.ts`) has zero producers — the bridges stamp `'claude'` and `'codex'`; the only `'native'` constructor anywhere is the lib's own unit test. The field's own JSDoc defines `dialect` as "the bridge that ran it", and native is not a bridge: the [interception-seams RFC](../../implemented/feature/2026-06-30-interception-seams.md) records that native hooks are not a package and that "a native plugin can already use the typed Decisions" without the durable hook log, and the flagship native-plugin worked example asserts exactly that (no `hook/*` events at all). -2. **`HookOutput.suppressOutput`** (same file) is parsed by the codec and discarded on every path: no bridge branch, no merge fold, no warn, no deferred-list row — uniquely among its parsed-but-unhonored siblings, each of which carries a stated deferral (`updatedInput` → a logged warn plus the [pre-tool-input-rewrite proposal](../feature/2026-06-30-pre-tool-input-rewrite.md); `systemMessage` → a logged warn plus a README deferred row; `continue`/`stopReason` → a `TODO(hook-continue-false)` anchor plus the `'stop'` decision record). Structurally there is nothing to suppress: hook stdout never enters any transcript (context flows only via `additionalContext`; the log records only `decision`/`stderrSummary`), so a hook author setting `suppressOutput: true` gets silent nothing with no warn. -3. **`hook/result.durationMs`** is durable timing telemetry with no reader. Both bridges write it, and the ACP snapshot normalizer scrubs it to `0` because wall-clock hook runtime is replay noise (`examples/acp-agent/tests/snapshot-normalize.ts`); the remaining consumers are tests and the goldens that exist because the field exists. Deterministic provenance fields (`point`, `matcher`, `turn`, `handlerId`) earn their durability as audit facts; a nondeterministic field that replay must erase and nothing reads earns neither its bytes nor its special-case scrub. -4. **`defaultTimeoutMs` is double-defaulted in both bridge configs** — a schema `.default(600_000)` AND a `?? 600_000` fallback (`packages/hooks/hooks-claude/src/index.ts`, `packages/hooks/hooks-codex/src/index.ts`) — the same two-homes-for-one-literal shape the ACP bridge's `TODO(double-default)` flags, for a knob no shipped config sets; the per-hook `timeoutSec` is the real timeout surface. -5. **The `hook/result` semantics live in the bridges, twice, not in the lib that owns the event.** `summarize()` — the 500-character stderr truncation rule — is byte-identical in `packages/hooks/hooks-claude/src/index.ts` and `packages/hooks/hooks-codex/src/index.ts`, and so is the decision-string rule `output.decision ?? (output.continue === false ? 'stop' : 'pass')`; yet `dsh-hook-protocol` declares `hook/result`, documents `stderrSummary` as "truncated" without owning the truncation, and documents the decision values without owning the mapping. If one bridge drifts (a different cap, a different fallback), the shared durable event's semantics fork silently. - -## Proposal - -Narrow `HookDialect` to `'claude' | 'codex'` and fix its JSDoc; retarget the lib's one `'native'` test. Drop `suppressOutput` from `HookOutput`, the codec's parse lines, its codec-test assertions, and the parsed-superset lists in the lib README and [hook-protocol-lib RFC](../../implemented/feature/2026-06-30-hook-protocol-lib.md) (amended per [implemented/AGENTS.md](../../implemented/AGENTS.md)). Drop `durationMs` from `HookResultRecord`, `RunHookResult`, the `hook/result` event, the bridge appends, the docs/catalog, and the snapshot normalizer's special-case scrub (retiring `runHook`'s injected clock if nothing else needs it); the hook goldens refresh mechanically as the scrubbed field disappears. Replace the bridges' `defaultTimeoutMs` config knob with one shared reference-default constant in `dsh-hook-protocol` (per-hook `timeoutSec` stays the override surface). Move the `hook/result` semantics into the lib: `appendHookResult` (or a helper it exposes) derives `stderrSummary` and the decision string from the `HookOutput` + exit outcome, and both bridges delete their private copies. Rider: un-export `BLOCKING_EXIT_CODE` (zero importers; even the codec tests spell the literal `2`). - -## Why not keep them? - -The [hook-protocol-lib RFC](../../implemented/feature/2026-06-30-hook-protocol-lib.md) deliberately records "parses the full CC superset" — the strongest counterargument is that this proposal re-litigates decisions that RFC records. But parsing a field whose value can never influence anything is not protocol faithfulness, it is a reader trap; a dialect variant that the design's own thesis says will never be stamped is vocabulary without an interpreter; and durable telemetry that replay must scrub is a cost with no buyer. Each returns trivially with its first real consumer (a transcript surface with hook stdout to suppress; a native-provenance feature that logs hook events; a trace viewer that reads timings — as live diagnostics or a deliberately durable telemetry event designed for it). On item 5, the lib RFC chose per-bridge explicitness over a parameterized engine — but that choice governed payload construction and Decision mapping; the semantics of the SHARED durable event are precisely the "primitives where duplication would actually be dangerous" that the same RFC assigns to the lib. - -## Acceptance criteria - -- `HookDialect` is two-valued; `rg "'native'"` in the hooks packages returns only this RFC's amended references. -- `suppressOutput` and `durationMs` appear nowhere in source, parsed-field doc lists, the catalog, or the normalizer; the hook goldens are re-recorded or refreshed without the field. -- Both bridge configs lose `defaultTimeoutMs`; the reference default lives once, in the lib; per-hook `timeoutSec` still overrides it. -- One definition each of the truncation rule and the decision-string rule, in `dsh-hook-protocol`, exercised by both bridges' suites. - -## Risks - -The `dialect`, `suppressOutput`, `defaultTimeoutMs`, and semantics changes are invisible on the wire and in the goldens; the `durationMs` removal churns the hook goldens once (a mechanical refresh — the field was already normalized to a constant). The cost is churn in `dsh-hook-protocol` and both bridges — cheap under the pre-release stance, and cheaper than letting two copies of a durable event's semantics age apart. diff --git a/docs/rfc/proposed/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md b/docs/rfc/proposed/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md deleted file mode 100644 index a4bfbbf896..0000000000 --- a/docs/rfc/proposed/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md +++ /dev/null @@ -1,27 +0,0 @@ -# RFC: Trim unreachable ACP bridge surface — the branding knobs and the kind-sniffing fallback - -Status: proposed - -## Problem - -Two pieces of `dsh-acp` surface are unreachable from any shipped configuration: - -1. **`AcpConfig.agentName` / `agentVersion`** (`packages/ui/acp/src/index.ts`). The shipped app package hands the bridge only `{ model, systemPrompt }` (`packages/ui/acp-agent/src/index.ts`), so no leaf `cordis.yml` — the only production config surface — can set the knobs at all; they are settable solely by direct-mounting the bridge, which only a unit test does. Every snapshot golden — the hook-matrix scenarios included — pins the schema defaults (`deepseek-harness-acp` / `0.0.1`). The pair also carries a live `TODO(double-default)`: the literals exist twice (schema `.default(...)` plus `??` fallbacks), with the TODO asking to pick one home. -2. **The `toolKindFor` name heuristic** (same file) special-cases `bash*`/`read*`/`write`/`edit*` tool names in the generic-fallback path. Since the [render-intent union](../../implemented/architecture/2026-07-02-tool-render-intent-union.md), every first-party tool those arms match ships its own `presentCall` carrying its kind, and the presenter-less production tools (`subagent`, `subagent_fork`) fall through to `other` anyway. The arms are production-reachable only when a tool's `presentCall` THROWS (the containment fallback) — and the bridge's own module doc states the design rule the heuristic violates: "the bridge never special-cases tool names". - -## Proposal - -Hardcode `agentInfo` at the `initialize` site (`{ name: 'deepseek-harness-acp', version: '0.0.1' }`), deleting the two config fields, their schema defaults, the `??` fallbacks, and the `TODO(double-default)` whose subject vanishes; drop the knob half of the direct-mount config test, the two rows in `packages/ui/acp/README.md`, and the `packages/ui/acp/acp-feature-support.md` cell that cites the knobs. Zero golden churn — the emitted wire value is unchanged. Replace `toolKindFor` with the constant `'other'` in both fallback sites (the presenter fallback and `nullToolPresenter`) and delete the heuristic with its test rows. - -## Why not keep them? - -`agentInfo` is client-visible branding a deployment will eventually want configurable — but a knob no shipped config can reach is not configurability, it is drift surface (the double-default TODO is its symptom), and the honest re-add must include the `dsh-acp-agent` plumb-through that does not exist today either; both arrive together with the deployment that needs them. For the heuristic: a hypothetical third-party presenter-less tool named `read_docs` would lose its inferred `read` icon — but inferring kinds from unknown plugins' names is exactly the special-casing the render-intent design rejected. The behavior delta on shipped paths is confined to the presenter-throw fallback, where rendering kind `other` makes the client show the raw input instead of a masquerading first-party card — strictly better diagnostics for a broken presenter. - -## Acceptance criteria - -- `agentName`/`agentVersion` and `toolKindFor` appear only in this RFC; snapshot goldens are byte-identical; bridge tests are green with the constant fallback. -- The `initialize` handshake continues to report `deepseek-harness-acp`/`0.0.1` (pinned by the handshake snapshot). - -## Risks - -None beyond the presenter-throw rendering delta described above — an error path whose new behavior is more diagnosable than the old. diff --git a/docs/rfc/proposed/testing/2026-07-04-single-source-acp-replay-config.md b/docs/rfc/proposed/testing/2026-07-04-single-source-acp-replay-config.md deleted file mode 100644 index 3cb986a3e7..0000000000 --- a/docs/rfc/proposed/testing/2026-07-04-single-source-acp-replay-config.md +++ /dev/null @@ -1,26 +0,0 @@ -# RFC: Single-source the acp-agent replay config - -Status: proposed - -## Problem - -`examples/acp-agent` ships two hand-maintained configs: `cordis.yml` (the live tree) and `cordis.snapshot.yml` (the keyless replay tree). Stripped of comments and blanks, their entire difference is ONE plugin entry — the eight-line `llm-deepseek` stanza (with its `!!js` env keys and model list) versus the two-line `llm-replay` stanza. Every other entry is byte-identical, including the multi-line system prompt and both hook-bridge stanzas. Every app-shape change must therefore be made twice, and the [hook-snapshot-matrix RFC](../../implemented/testing/2026-07-04-hook-snapshot-matrix.md) records paying exactly that tax: "hence the symmetric edit to both configs". - -Nothing gates the symmetry. If the copies drift, the snapshot tier silently exercises a different app than the one that ships — the ["green units, broken product" class of gap](../../../postmortem/0001-acp-default-export-drops-inject.md) the snapshot tier exists to close, reintroduced one level up, with reviewer vigilance as the only defense. - -## Proposal - -Make the replay tree derive from the live tree instead of mirroring it. Preferred endpoint: a single source — either `cordis.snapshot.yml` becomes a thin overlay that includes `cordis.yml` and swaps only the llm entry (if the vendored loader/include config supports entry-level override), or the acp-agent bin's existing `DSH_SNAPSHOT=replay` branch performs the one-entry swap on the parsed config and `cordis.snapshot.yml` is deleted. Fallback endpoint, if single-sourcing is judged too magical for a teaching example: keep both files and add a boring verify gate (in the `doc-sync`/`hygiene` family) asserting the two configs' entry sets are equal modulo the llm entry. The implementing PR picks after checking the loader's include/override capability, updates the recording docs, and amends the snapshot RFCs' facts per [implemented/AGENTS.md](../../implemented/AGENTS.md). - -## Why not keep the twin? - -An explicit replay file is transparently readable and teaches replay semantics — the strongest counterargument, and the reason the fallback keeps the file and adds only the gate. YAML surgery inside the published bin is real complexity in a shipping artifact, and an include-overlay depends on loader capability that may not exist. But the status quo — a 125-line hand-maintained near-copy of a 141-line file whose one meaningful difference is two lines, defended by nothing — is the one option with a silent failure mode, and it grows with every plugin the app gains (the hook-bridge stanzas are twins in both files). - -## Acceptance criteria - -- Either one config file plus a mechanical llm-entry swap exercised by the snapshot suite itself, or two files plus a symmetry gate that fails CI on any non-llm divergence. -- All snapshot scenarios (hook matrix included) pass unchanged; `pnpm run test:snapshot:record` still boots the live tree. - -## Risks - -The include-overlay shape may be unsupported by the vendored loader — then the bin-side swap or the gate. `echo-agent`/`coding-agent` are unaffected (no snapshot twin). If the gate route is chosen, it is one more bespoke verify script — the cost the repo's gate-friendly policy explicitly accepts for encoding an invariant no human reliably remembers. diff --git a/examples/acp-agent/cordis.snapshot.yml b/examples/acp-agent/cordis.snapshot.yml index 3bef79d83a..fd8a9dcad2 100644 --- a/examples/acp-agent/cordis.snapshot.yml +++ b/examples/acp-agent/cordis.snapshot.yml @@ -1,125 +1,33 @@ -# Snapshot-test REPLAY config: the acp-agent plugin tree with the model backend -# swapped to llm-replay (serves a recorded session JSONL — no API key, no -# network). The dsh-acp-agent bin selects this file for DSH_SNAPSHOT=replay. +# Snapshot-test REPLAY overlay: the SAME app tree as cordis.yml, derived from +# it by an include — the one difference is the model backend. A keyless replay +# run cannot boot the real adapter (llm-deepseek's apply() throws without +# DEEPSEEK_API_KEY), so the include patches the live tree at load time: the +# llm-deepseek entry is disabled by id, and the llm-replay entry (which serves +# a recorded session JSONL — no API key, no network) is inserted. Every other +# entry — the app, the bash executor, the fs/subagent/todo tools, both hook +# bridges, the system prompt — IS the live tree, so replay exercises exactly +# what ships and an app-shape change lands once, in cordis.yml. # -# Same app as cordis.yml (@deepseek-ai/dsh-acp-agent: the agent-core spine + -# JSONL persistence + the ACP bridge) — only the LLM backend differs: llm-replay -# here, llm-deepseek there. It can't reuse the real adapter because llm-deepseek's -# apply() throws without DEEPSEEK_API_KEY, killing a keyless replay run at boot. -# -# stdout is reserved for the ACP JSON-RPC protocol — no stdout logger (the app -# package omits it). The replay fixture path comes from $DSH_SNAPSHOT_FILE (and -# an optional $DSH_SNAPSHOT_OVERRIDE sidecar), set by the snapshot harness. - -# The replay adapter: short-circuits llm/stream with the recorded log's chunks, -# in place of llm-deepseek. -- id: llm-replay - name: '@deepseek-ai/dsh-llm-replay' - -# Local bash executor for agent-core's tool-bash schema. -# FIXME(config-comments): keep this executor note from implying bash is the -# whole tool set; filesystem, subagent, and todo_write are loaded below. -- id: bash - name: '@deepseek-ai/dsh-bash-local' +# The dsh-acp-agent bin selects this file for DSH_SNAPSHOT=replay. The replay +# fixture path comes from $DSH_SNAPSHOT_FILE (and an optional +# $DSH_SNAPSHOT_OVERRIDE sidecar), set by the snapshot harness. stdout stays +# reserved for the ACP JSON-RPC protocol (the app package loads no stdout +# logger). Patches apply when the include loads the file — a one-shot replay +# boot, so the load-time-only patch semantics are exactly enough. +- id: base + name: '@cordisjs/plugin-include' config: - timeoutMs: 60000 - -# The ACP server app — identical to cordis.yml's entry. -- id: acp-agent - name: '@deepseek-ai/dsh-acp-agent' - config: - model: deepseek-v4-flash - persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' - systemPrompt: | - You are a coding assistant driven over the Agent Client Protocol. - - Your tools are read/write/edit for file operations, bash (plus - bash_output/bash_kill for background tasks), and subagent. Use read to - inspect UTF-8 text files, write to create or replace files, and edit for - targeted literal replacements. Use bash for shell commands, tests, - searches, and operations that are not ordinary file reads or edits. Each - bash call runs in a fresh shell — pass workdir instead of cd. Check the - [exit code: N] marker; verify your work. Keep answers brief and factual. - - Use the subagent tool to delegate a focused, self-contained subtask to - a fresh child agent (it works in its own context and returns only its - final result) — give it a complete, standalone instruction. Use - subagent_fork instead when the subtask needs THIS conversation's - context: the child inherits the log so far. - - For multi-step work, use the todo_write tool to track a task list: - send the WHOLE list each call (it replaces the previous one), keep at - most one task in_progress (exactly one while work remains), and mark a - task completed as soon as it is done. Skip it for trivial single-step - tasks. - -# The subagent seam + both in-process backends + two model-facing tools — -# identical to cordis.yml's wiring (only the LLM backend differs above): spawn -# and fork are each reachable via a dsh-tool-subagent bound to it with a distinct -# toolName (subagent → spawn, subagent_fork → fork). -- id: subagent - name: '@deepseek-ai/dsh-subagent' - -- id: subagent-spawn - name: '@deepseek-ai/dsh-subagent-spawn' - config: - providerName: spawn - -- id: subagent-fork - name: '@deepseek-ai/dsh-subagent-fork' - config: - providerName: fork - -- id: tool-subagent - name: '@deepseek-ai/dsh-tool-subagent' - config: - provider: spawn - toolName: subagent - -- id: tool-subagent-fork - name: '@deepseek-ai/dsh-tool-subagent' - config: - provider: fork - toolName: subagent_fork - -# The model-facing todo_write tool — identical to cordis.yml's wiring, so a -# replayed todo_write tool call resolves to a real tool during snapshot replay. -- id: tool-todo - name: '@deepseek-ai/dsh-tool-todo' - -# Filesystem capability stack — identical to cordis.yml's wiring, so replayed -# read/write/edit tool calls resolve to the real tools during snapshot replay. -- id: fs-local - name: '@deepseek-ai/dsh-fs-local' - config: - cwd: !!js process.cwd() - -- id: fs-policy - name: '@deepseek-ai/dsh-fs-policy' - -- id: tool-fs - name: '@deepseek-ai/dsh-tool-fs' - -# The Claude Code hook bridge. `configPath` is read ONCE at load and resolves -# `./hooks.json` against the PROCESS cwd (not per-session) — in these snapshot -# runs the harness launches the subprocess with process cwd = the scenario's temp -# workspace, so a scenario that ships `workspace/hooks.json` (copied into that cwd -# before the run) exercises the hooks path end-to-end; every other scenario has no -# such file, so the parse fails-soft and the bridge registers nothing (a silent -# no-op — the ACP app loads no logger exporter, so the warning never reaches -# stdout). Hooks themselves run in the session cwd (the bridge passes it as workdir). -- id: hooks-claude - name: '@deepseek-ai/dsh-hooks-claude' - config: - configPath: ./hooks.json - -# The Codex hook bridge, loaded alongside the Claude one (symmetric with -# cordis.yml so a recorded Codex scenario fires the hook during recording too). It -# reads its OWN file `./codex-hooks.json` (Codex's dialect) — the two bridges -# cannot share one config. Same fails-soft-when-absent contract: a scenario that -# ships `workspace/codex-hooks.json` exercises the Codex path end-to-end; a -# scenario without one registers nothing (a silent no-op, never reaching stdout). -- id: hooks-codex - name: '@deepseek-ai/dsh-hooks-codex' - config: - configPath: ./codex-hooks.json + path: ./cordis.yml + patches: + # The name is an assertion, not an override: the include skips the patch + # (warning if a logger exists) when the id points at a different plugin, + # so this can never disable the wrong entry. If cordis.yml ever RENAMES + # the id, the patch degrades to a skip — replay output stays correct + # (llm-replay still short-circuits the stream) but the stale patch and a + # futile keyless adapter entry linger until review catches them. + - id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' + disabled: true + - insert: + - id: llm-replay + name: '@deepseek-ai/dsh-llm-replay' diff --git a/examples/acp-agent/tests/snapshots/hook-codex-posttool-block/stdout.golden.jsonl b/examples/acp-agent/tests/snapshots/hook-codex-posttool-block/stdout.golden.jsonl index a2979ec54f..bc189b2cc2 100644 --- a/examples/acp-agent/tests/snapshots/hook-codex-posttool-block/stdout.golden.jsonl +++ b/examples/acp-agent/tests/snapshots/hook-codex-posttool-block/stdout.golden.jsonl @@ -79,7 +79,7 @@ {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" or"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" command"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} -{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_XLxXjqVljRIC2KwFZCKX3369","title":"bash","kind":"execute","status":"in_progress","rawInput":{"command":"echo HELLO"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_XLxXjqVljRIC2KwFZCKX3369","title":"bash","kind":"other","status":"in_progress","rawInput":{"command":"echo HELLO"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_XLxXjqVljRIC2KwFZCKX3369","status":"failed","content":[{"type":"content","content":{"type":"text","text":"tool output rejected by codex policy: summarize instead"}}]}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"The"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" tool"}}}} @@ -99,7 +99,7 @@ {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" invoke"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" it"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"."}}}} -{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_twNfzWSJ9WJn8YeoGVIi1932","title":"bash","kind":"execute","status":"in_progress","rawInput":{"command":"printf 'HELLO\\n'"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_twNfzWSJ9WJn8YeoGVIi1932","title":"bash","kind":"other","status":"in_progress","rawInput":{"command":"printf 'HELLO\\n'"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_twNfzWSJ9WJn8YeoGVIi1932","status":"failed","content":[{"type":"content","content":{"type":"text","text":"tool output rejected by codex policy: summarize instead"}}]}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"Every"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" attempt"}}}} @@ -139,7 +139,7 @@ {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":" is"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":" working"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":":"}}}} -{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_xOnTckie072jHuPhX0Mz5115","title":"bash","kind":"execute","status":"in_progress","rawInput":{"command":"pwd"}}}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call","toolCallId":"call_00_xOnTckie072jHuPhX0Mz5115","title":"bash","kind":"other","status":"in_progress","rawInput":{"command":"pwd"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"tool_call_update","toolCallId":"call_00_xOnTckie072jHuPhX0Mz5115","status":"failed","content":[{"type":"content","content":{"type":"text","text":"tool output rejected by codex policy: summarize instead"}}]}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":"It"}}}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_thought_chunk","content":{"type":"text","text":" seems"}}}} diff --git a/package.json b/package.json index 6ecbb3f9f0..8a0218f1e7 100644 --- a/package.json +++ b/package.json @@ -42,10 +42,12 @@ "verify-tool-catalog": "tsx scripts/gen-tool-catalog.ts --check", "gen-doc-graphs": "tsx scripts/gen-doc-graphs.ts", "verify-doc-graphs": "tsx scripts/gen-doc-graphs.ts --check", + "gen-persistence-catalog": "tsx scripts/gen-persistence-catalog.ts", + "verify-persistence-catalog": "tsx scripts/gen-persistence-catalog.ts --check", "gen-module-graph": "tsx scripts/gen-module-graph.ts", "verify-module-graph": "tsx scripts/gen-module-graph.ts --check", "constraints": "tsx scripts/check-workspace-constraints.ts", - "doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-tool-catalog && pnpm run verify-doc-graphs && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-mermaid && pnpm run verify-rfc-classification && pnpm run verify-type-equiv && pnpm run verify-translation-pairing && pnpm run verify-doc-budgets", + "doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-tool-catalog && pnpm run verify-persistence-catalog && pnpm run verify-doc-graphs && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-mermaid && pnpm run verify-rfc-classification && pnpm run verify-type-equiv && pnpm run verify-translation-pairing && pnpm run verify-doc-budgets", "hygiene": "pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types", "demo:echo": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/echo-agent/cordis.yml", "demo:repl": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/coding-agent/cordis.yml", diff --git a/packages/bash/tool-bash/tests/tools.spec.ts b/packages/bash/tool-bash/tests/tools.spec.ts index a01bdc0c86..0187302ef1 100644 --- a/packages/bash/tool-bash/tests/tools.spec.ts +++ b/packages/bash/tool-bash/tests/tools.spec.ts @@ -824,7 +824,7 @@ describe('tool-owned UI presentation (presentCall / presentResult)', () => { const ctx = await setup() const present = ctx.tools.get('bash')!.presentResult!( { command: 'x', description: 'x' }, - { content: [{ type: 'image', url: 'https://x/y.png' }], isError: false }, + { content: [{ type: 'reasoning', text: 'unexpected' }], isError: false }, ) expect(present).toBeUndefined() }) diff --git a/packages/compact/compact-basic/README.md b/packages/compact/compact-basic/README.md index ba976d2833..81163973fc 100644 --- a/packages/compact/compact-basic/README.md +++ b/packages/compact/compact-basic/README.md @@ -8,10 +8,10 @@ This is the implementation tier of the compaction capability — see the [interf The abstract contract states only WHAT compaction does; this backend owns every HOW decision: -- **Token estimation** — `estimateContentTokens()`: chars divided by the `charsPerToken` config (default 4) with per-block structural overhead (`text`/`reasoning` = `ceil(len/charsPerToken) + 4`, `tool-call` from name + arguments, `tool-result` recursive, `image` = 85, unknown blocks via JSON length). +- **Token estimation** — `estimateContentTokens()`: chars divided by the `charsPerToken` config (default 4) with per-block structural overhead (`text`/`reasoning` = `ceil(len/charsPerToken) + 4`, `tool-call` from name + arguments, `tool-result` recursive, unknown blocks via JSON length). - **Retention policy** — `compactIfNeeded()` walks the surface nodes tail→head summing per-node token estimates, and retains the smallest tail-run of WHOLE units (a closed step, or a single no-step node such as a pre-step `user/message` or inter-step `steering/message`) whose total reaches `retainTokens`; everything older is compacted. Retention is **turn-agnostic** — turn boundaries play no role, so a single runaway turn that alone exceeds the window compacts its OWN early closed steps rather than being retained verbatim (the failure mode that motivated dropping turn-protection: a tool-heavy turn must stay compactable or the harness dies exactly when compaction is needed). The only structural guard is **tool-pairing balance**: the compacted region's edges are balanced cuts on the surface (no unanswered tool-call crosses either edge), so it never splits a step's `assistant/message` tool-calls from their `tool/result`s. When the only compactable content left is an un-splittable open tail step, it declines (returns `null`) and retries once an older step closes. **Single-unit overflow is out of scope, by design**: if one retained unit (a single closed step, or a large pasted `user/message`) ALONE exceeds the budget, compaction cannot help and the call may go out over-budget — bounding an individual unit's size is a separate concern. `compactRegion()` enforces tool-pairing balance strictly, throwing on a boundary that would split a step. `dsh-session` exports `isToolPairingBalanced` for the check. - **Dynamic convergence** — no static summary-length config pretends to bound what the model will write. If framing/estimator/system overhead leaves the compacted surface above threshold, `compactIfNeeded()` re-compacts the head checkpoint up to `compactionRetries` extra times; if it still cannot get below threshold, it throws. A summary whose estimated stored size is not smaller than the shadowed content fails closed before it mutates the surface. -- **Summarization** — `summarize()`: a `GenerateOptions` request assembled via `BlockAssembler` with a fixed system prompt that asks for a structured checkpoint (Primary Request and Intent · Key Technical Concepts · Files and Code · Errors and Fixes · Pending Tasks · Current Work · Next Step · Critical Context), every section mandatory, exact paths/commands/identifiers preserved. The request runs through the `agent/request` waterfall before `ctx.llm.stream()`, so router agents that choose the concrete model there also route compaction summaries. `maxTokens` is the provider-side generation cap; only text blocks from the model's reply are kept before the checkpoint is stored (reasoning is dropped so private chain-of-thought never leaks into the durable summary, and a stray `tool-call` is dropped so the synthesized `user/message` summary cannot land an orphaned call with no matching `tool-result`). The compacted region is flattened to a plain-text transcript first: text and reasoning contribute their text, and every non-text block (image, tool-call, tool-result, plugin-added types) contributes a type-tagged placeholder (`[image]`, `[tool-call: name(args)]`, …) so the summarizer is told what existed rather than silently dropping it. +- **Summarization** — `summarize()`: a `GenerateOptions` request assembled via `BlockAssembler` with a fixed system prompt that asks for a structured checkpoint (Primary Request and Intent · Key Technical Concepts · Files and Code · Errors and Fixes · Pending Tasks · Current Work · Next Step · Critical Context), every section mandatory, exact paths/commands/identifiers preserved. The request runs through the `agent/request` waterfall before `ctx.llm.stream()`, so router agents that choose the concrete model there also route compaction summaries. `maxTokens` is the provider-side generation cap; only text blocks from the model's reply are kept before the checkpoint is stored (reasoning is dropped so private chain-of-thought never leaks into the durable summary, and a stray `tool-call` is dropped so the synthesized `user/message` summary cannot land an orphaned call with no matching `tool-result`). The compacted region is flattened to a plain-text transcript first: text and reasoning contribute their text, and every non-text block (tool-call, tool-result, plugin-added types) contributes a type-tagged placeholder (`[tool-call: name(args)]`, `[tool-result: …]`, …) so the summarizer is told what existed rather than silently dropping it. - **Checkpoint framing** — the raw summary is not landed directly. `compactRegion()` wraps it in a checkpoint preamble (so a resuming model reads it as a checkpoint, not a fresh user request, and builds on the captured context rather than restating it) plus `…` tags. Because region compaction can be invoked manually, a surface may hold several checkpoints, so the framing does not claim everything after it is recent or verbatim. The tags make a prior checkpoint detectable in the transcript on the next compaction cycle: the summarization prompt then instructs the model to merge it in place (preserve still-true facts, drop stale ones) rather than re-summarize it verbatim — a cheap incremental merge that needs no extra log/event machinery. The unframed summary stays on the `compact/summary` provenance event. - **Surface mutation** — `compactRegion()` appends the `compact/start` → `compact/summary` → `compact/end` log records and the single `user/message` replace node carrying the framed summary (see the interface README). - **Auto-compaction** — an `agent/pre-step` listener delegates to `compactIfNeeded()` before every step (not just a turn's first — a tool-heavy turn grows the surface mid-turn, so a runaway turn still compacts, and per-step firing is the only moment to rescue it before overflow). `agent/pre-step` is a serial (awaited, in-order) surface-mutation checkpoint that fires after `turn/start` and BEFORE the step opens (`step/start`) and its request history is derived, so compaction mutates the surface — with its log-only `compact/*` records landing cleanly outside any step — and the loop derives once from the result: no double-derive, and the listener cannot see (or need to rewrite) an already-assembled `messages` array. The listener owns no threshold logic of its own (the single token-pressure check lives in `compactIfNeeded()`); because Cordis `serial` bails early on non-void return values, the listener returns `void` and does not use the dispatcher's bail channel as a veto surface. diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index a1a83cd407..2bc7b65a80 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -45,9 +45,6 @@ export { resolveConfig } from './types.ts' /** Per-block structural overhead for JSON framing / type tag. */ const BLOCK_OVERHEAD = 4 -/** Heuristic token count for an image block (~85 tokens for low-res URL). */ -const IMAGE_TOKEN_COST = 85 - /** Role-field framing overhead added per message in {@link BasicCompactService.estimateTokens}. */ const ROLE_OVERHEAD = 4 @@ -236,9 +233,6 @@ export class BasicCompactService extends CompactService { case 'tool-result': tokens += this.estimateContentTokens(block.content) + BLOCK_OVERHEAD break - case 'image': - tokens += IMAGE_TOKEN_COST - break default: // Unknown block types (merge-extensible ContentBlockMap): // estimate conservatively via JSON stringify. @@ -712,10 +706,10 @@ export class BasicCompactService extends CompactService { /** * Render content blocks to a single plain-text string for the summarization * prompt. Text and reasoning contribute their text; every other block type - * contributes a type-tagged placeholder (`[image]`, `[tool-call: name(args)]`, - * …) so the summarizer is told what non-text content existed in the region - * rather than silently losing it. Blocks join with newlines; empty-text - * blocks contribute nothing. + * contributes a type-tagged placeholder (`[tool-call: name(args)]`, + * `[tool-result: …]`, …) so the summarizer is told what non-text content + * existed in the region rather than silently losing it. Blocks join with + * newlines; empty-text blocks contribute nothing. */ private _blocksToText(blocks: readonly ContentBlock[]): string { const parts: string[] = [] @@ -735,9 +729,6 @@ export class BasicCompactService extends CompactService { parts.push(inner ? `[tool-result: ${inner}]` : '[tool-result]') break } - case 'image': - parts.push('[image]') - break // ContentBlockMap is merge-extensible — render an unknown block as a // bare type-tagged placeholder so a plugin-added block type is still // signalled to the summarizer rather than dropped. diff --git a/packages/compact/compact-basic/tests/compact-basic.spec.ts b/packages/compact/compact-basic/tests/compact-basic.spec.ts index 7d5d67b6db..80c35d4b06 100644 --- a/packages/compact/compact-basic/tests/compact-basic.spec.ts +++ b/packages/compact/compact-basic/tests/compact-basic.spec.ts @@ -811,11 +811,6 @@ describe('BasicCompactService token estimation (char/4 heuristic)', () => { ])).toBe(10) }) - it('estimates image blocks at fixed 85 tokens', () => { - const svc = new BasicCompactService(new Context(), cfg({ auto: false })) - expect(svc.estimateContentTokens([{ type: 'image', url: 'https://example.com/img.png' }])).toBe(85) - }) - it('returns 0 for empty content blocks', () => { const svc = new BasicCompactService(new Context(), cfg({ auto: false })) expect(svc.estimateContentTokens([])).toBe(0) @@ -1341,7 +1336,7 @@ describe('BasicCompactService edge cases', () => { s.append('assistant/message', { turn: 1, step: 1, content: [ - { type: 'tool-result', toolCallId: CallId('n1'), content: [{ type: 'image', url: 'https://x/n.png' }] }, + { type: 'tool-result', toolCallId: CallId('n1'), content: [{ type: 'chart', data: 'x' } as unknown as ContentBlock] }, { type: 'custom-widget', payload: 'x' } as unknown as ContentBlock, { type: 'tool-call', id: CallId('b1'), name: 'bash', arguments: '{}' }, ], @@ -1360,7 +1355,7 @@ describe('BasicCompactService edge cases', () => { const nodes = s.surface.nodes await compactRegion(svc, s, nodes[0]!.seq, nodes[nodes.length - 1]!.seq, 'm') const { text } = svc.summarizeCalls[0]! - expect(text).toContain('[tool-result: [image]]') // nested tool-result with content + expect(text).toContain('[tool-result: [chart]]') // nested tool-result with content expect(text).toContain('[custom-widget]') // unknown block placeholder expect(text).toContain('Tool result (call b1): [tool-result]') // empty nested → bare placeholder }) @@ -1527,25 +1522,28 @@ describe('BasicCompactService edge cases', () => { it('renders non-text blocks as type-tagged placeholders across all message kinds', async () => { const svc = createTestService() const s = new Session(SessionId('placeholders')) + // A plugin-added block type (merge-extensible ContentBlockMap) — the + // placeholder path must cover every message kind, not just assistant. + const chart = (id: string): ContentBlock => ({ type: 'chart', data: id } as unknown as ContentBlock) s.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) s.append('step/start', { turn: 1, step: 1 }) - // user/message with only an image block → '[image]' placeholder. - s.append('user/message', { content: [{ type: 'image', url: 'https://x/y.png' }], source: { kind: 'user' } }, { surfaceOp: 'append' }) - // assistant/message with an image block AND the tool-call its tool/result - // answers (so the surface is tool-pairing balanced) → '[image]' placeholder. + // user/message with only a plugin-added block → '[chart]' placeholder. + s.append('user/message', { content: [chart('y')], source: { kind: 'user' } }, { surfaceOp: 'append' }) + // assistant/message with a plugin-added block AND the tool-call its + // tool/result answers (so the surface is tool-pairing balanced). s.append('assistant/message', { turn: 1, step: 1, content: [ - { type: 'image', url: 'https://x/z.png' }, + chart('z'), { type: 'tool-call', id: CallId('e1'), name: 'bash', arguments: '{}' }, ], }, { surfaceOp: 'append' }) - // tool/result with an image block → '[image]' placeholder. + // tool/result with a plugin-added block → '[chart]' placeholder. s.append('tool/call', { turn: 1, step: 1, callId: CallId('e1'), name: 'bash', arguments: '{}' }) - s.append('tool/result', { turn: 1, step: 1, callId: CallId('e1'), content: [{ type: 'image', url: 'https://x/r.png' }], isError: false }, { surfaceOp: 'append' }) - // context/message and steering/message with image content. - s.append('context/message', { content: [{ type: 'image', url: 'https://x/c.png' }], source: { kind: 'user' } }, { surfaceOp: 'append' }) - s.append('steering/message', { turn: 1, content: [{ type: 'image', url: 'https://x/s.png' }], source: { kind: 'user' } }, { surfaceOp: 'append' }) + s.append('tool/result', { turn: 1, step: 1, callId: CallId('e1'), content: [chart('r')], isError: false }, { surfaceOp: 'append' }) + // context/message and steering/message with plugin-added content. + s.append('context/message', { content: [chart('c')], source: { kind: 'user' } }, { surfaceOp: 'append' }) + s.append('steering/message', { turn: 1, content: [chart('s')], source: { kind: 'user' } }, { surfaceOp: 'append' }) s.append('step/end', { turn: 1, step: 1 }) s.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) s.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } }) @@ -1554,11 +1552,11 @@ describe('BasicCompactService edge cases', () => { await compactRegion(svc, s, nodes[0]!.seq, nodes[nodes.length - 1]!.seq, 'm') const { text } = svc.summarizeCalls[0]! // Every non-text block surfaces as a placeholder rather than being dropped. - expect(text).toContain('User: [image]') - expect(text).toContain('Assistant: [image]') - expect(text).toContain('Tool result (call e1): [image]') - expect(text).toContain('[Context: [image]]') - expect(text).toContain('[Steering: [image]]') + expect(text).toContain('User: [chart]') + expect(text).toContain('Assistant: [chart]') + expect(text).toContain('Tool result (call e1): [chart]') + expect(text).toContain('[Context: [chart]]') + expect(text).toContain('[Steering: [chart]]') }) }) diff --git a/packages/compact/compact/README.md b/packages/compact/compact/README.md index 424ee12bcf..b49070c3b5 100644 --- a/packages/compact/compact/README.md +++ b/packages/compact/compact/README.md @@ -43,13 +43,7 @@ Compaction is serialized via a log-recorded lock: `compactRegion` refuses to sta ## Events -The `compact/*` events extend `SessionEventMap` (merge-extensible) via declaration merging — they are session events, not cordis `Events`: - -| Event | Payload | On surface? | -|---|---|---| -| `compact/start` | `{ turn }` | no (log-only) | -| `compact/summary` | `{ summary, shadowedRange, shadowedSeqs, shadowedTokenCount }` | no (log-only) | -| `compact/end` | `{ turn, error? }` | no (log-only) | +The `compact/*` events extend `SessionEventMap` (merge-extensible) via declaration merging — they are session events, not cordis `Events`, and all three are log-only (no `surfaceOp`). Per-event payloads and semantics are in the generated [persistence log event catalog](../../../docs/persistence-catalog/log-events.md). ## Implementing a backend diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 56c6cc1863..ef5c50b7ce 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -166,7 +166,7 @@ export interface LoopHandle { * → dispatch → tools/post-execute * session('tool/result') * append buffered post-execute additionalContext → session('context/message')(s) - * drain steering → session('steering/message'); emit agent/steering + * drain steering → session('steering/message') * session('step/end') ⟵ durable step boundary (no agent/* mirror) * cont = waterfall agent/turn-continuation ⟵ ContinuationDecision; default * {action: hadToolCalls||steered ? 'continue':'stop'}; a continue.reason is @@ -420,7 +420,7 @@ async function runTurn(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle, // Steering from the previous round's continuation listeners joins before // the request. - drainSteering(ctx, agent, turn) + drainSteering(agent, turn) // The step's AbortController exists BEFORE any async pre-step work so a // dispose() or cancel() — in a synchronous turn-start listener or an @@ -529,7 +529,7 @@ async function runTurn(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle, if (stepReason) reason = stepReason // Steering that arrived during streaming/tool execution. - const steered = drainSteering(ctx, agent, turn) + const steered = drainSteering(agent, turn) if (closeStep()) break @@ -635,11 +635,10 @@ async function runTurn(ctx: Context, agent: ReactLoopAgent, handle: LoopHandle, } /** Drain the steering queue into the session. Returns whether any arrived. */ -function drainSteering(ctx: Context, agent: ReactLoopAgent, turn: number): boolean { +function drainSteering(agent: ReactLoopAgent, turn: number): boolean { const messages = agent.inbox.drainSteering() for (const message of messages) { agent.session.append('steering/message', { turn, content: message.content, source: message.source }, { surfaceOp: 'append' }) - ctx.emit('agent/steering', agent, turn, message.content, message.source) } return messages.length > 0 } diff --git a/packages/core/agent-loop/tests/review-fixes.spec.ts b/packages/core/agent-loop/tests/review-fixes.spec.ts index eddc69a2e6..76c377fd61 100644 --- a/packages/core/agent-loop/tests/review-fixes.spec.ts +++ b/packages/core/agent-loop/tests/review-fixes.spec.ts @@ -410,7 +410,7 @@ describe('MEDIUM: misc registry and config fixes', () => { expect(agent.session.deriveMessages().at(-1)?.content).toEqual([{ type: 'text', text: 'routed' }]) }) - it('agent/queued carries the resolved source; agent/steering carries its source', async () => { + it('agent/queued carries the resolved source; steering/message records its source', async () => { const adapter = new MockAdapter([toolCallResponse('c1', 'noop', {}), textResponse('done')]) const ctx = await harness(adapter) const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) @@ -425,15 +425,16 @@ describe('MEDIUM: misc registry and config fixes', () => { })) const queuedSources: { source: MessageSource; steering: boolean }[] = [] - const steeringSources: MessageSource[] = [] ctx.on('agent/queued', (_agent, _content, info) => void queuedSources.push(info)) - ctx.on('agent/steering', (_agent, _turn, _content, source) => void steeringSources.push(source)) send(agent, 'go') // no explicit source → default {kind:'user'} must be visible await waitForIdle(ctx, agent) expect(queuedSources[0]).toEqual({ source: { kind: 'user' }, steering: false }) expect(queuedSources[1]).toEqual({ source: { kind: 'plugin', plugin: 'goal' }, steering: true }) + // The drain appends the durable steering/message with the caller's source + // intact — the log, not a transient emit, is where consumers read it. + const steeringSources = agent.session.events.flatMap(e => e.type === 'steering/message' ? [e.data.source] : []) expect(steeringSources).toEqual([{ kind: 'plugin', plugin: 'goal' }]) }) }) diff --git a/packages/core/agent/README.md b/packages/core/agent/README.md index f4eb61c2bd..a6446bf7d1 100644 --- a/packages/core/agent/README.md +++ b/packages/core/agent/README.md @@ -50,9 +50,8 @@ Turn and step boundaries are NOT mirrored as `agent/*` emits: a consumer that ne Tool interception is the `tools/pre-execute` / `tools/post-execute` pair in [`dsh-tools`](../tools/README.md) (`PreToolDecision` allow/deny/ask, `PostToolDecision` accept/block) — same typed-Decision idiom, owned there because it is the tool registry's seam. -#### Live control notifications (emit) +#### Error notifications (emit) -- `agent/steering` — steering content injected mid-turn - `agent/error` — step/turn error The model's token stream is NOT an `agent/*` event: read it off the durable `session/event` feed as `assistant/chunk` (the same feed persistence and the ACP bridge use). diff --git a/packages/core/agent/src/types.ts b/packages/core/agent/src/types.ts index cbc6743870..2a5d713b85 100644 --- a/packages/core/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -20,7 +20,7 @@ * `agent/request`/`agent/step-result`/`agent/turn-continuation` waterfalls and * the serial `agent/pre-step`) that mutate/veto, and TRANSIENT emits * (`agent/status`, `agent/error`, `agent/created`/ - * `agent/disposed`, `agent/queued`, `agent/steering`, `agent/session-start`) + * `agent/disposed`, `agent/queued`, `agent/session-start`) * that notify with the `Agent` in hand. Turn/step boundaries are NOT here — * they are durable `session/event` records. Answers "right now, with the agent * object — intercept or observe." @@ -367,16 +367,7 @@ declare module 'cordis' { */ 'agent/turn-continuation'(agent: Agent, turn: number, defaultDecision: ContinuationDecision, next: () => Promise): Promise - // ---- streaming + tool notifications (emit) ---- - /** - * Steering content was injected into a running turn. - * @param agent - the agent that absorbed the steering. - * @param turn - the running turn that received it. - * @param content - the injected blocks. - * @param source - the steering message's resolved source. - * @mode emit - */ - 'agent/steering'(agent: Agent, turn: number, content: ContentBlock[], source: MessageSource): void + // ---- error notifications (emit) ---- /** * A step or turn errored. The loop reports a failure here (plus the logger) * even when the error has no in-turn position for a session `error` event. diff --git a/packages/core/session/README.md b/packages/core/session/README.md index 96d267c79b..72ca52bfef 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -49,9 +49,9 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. ### Session event vocabulary (`types.ts`) -The append-only log: `turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `prompt/blocked`, `assistant/message`, `assistant/chunk`, `tool/call`, `tool/result`, `steering/message`, `context/message`, `todo/write`. Token usage rides on `assistant/message.usage`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. +The append-only log's event types, enumerated member by member — payloads, surface badges, provenance — in the generated [persistence log event catalog](../../../docs/persistence-catalog/log-events.md). Token usage rides on `assistant/message.usage`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. -Merge-extensible via `SessionEventMap` — the compaction seam adds `compact/start`, `compact/summary`, and `compact/end`. +Merge-extensible via `SessionEventMap` — a plugin declaration-merges its own types (the compaction seam's `compact/*`, the hook bridges' `hook/*`); merged members appear in the same catalog. Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types for typed turn boundaries — `kind`-tagged instead of strings). diff --git a/packages/core/session/src/types.ts b/packages/core/session/src/types.ts index b9a4c3b89b..82e080aa1a 100644 --- a/packages/core/session/src/types.ts +++ b/packages/core/session/src/types.ts @@ -91,7 +91,6 @@ export interface CreateSessionOptions { */ export interface TurnTriggerMap { message: { kind: 'message'; source: MessageSource } - continuation: { kind: 'continuation' } /** * An out-of-band context injection (`agent.inject()`) made while the agent * was idle. The loop wraps the injected `context/message` in a one-shot turn @@ -198,9 +197,22 @@ export interface TodoItem { * the invariants plugin checks, is a breaking change to the on-disk format. */ export interface SessionEventMap { + /** + * Opens turn `turn`. `trigger` records what started it — a drained message + * batch or an idle-time injection. The turn is the durability/replay + * boundary: every event sits between a `turn/start` and its matching + * `turn/end` (the turn-enclosure invariant). + */ 'turn/start': { turn: number; trigger: TurnTrigger } + /** + * Closes turn `turn` with the {@link TurnEndReason} that ended it. The loop + * fires the awaited `session/flush` checkpoint at every turn end, so the turn + * boundary is also the durable-commit boundary. + */ 'turn/end': { turn: number; reason: TurnEndReason } + /** Opens step `step` of turn `turn` — one model call plus the tool executions it requested. */ 'step/start': { turn: number; step: number } + /** Closes step `step` of turn `turn`. */ 'step/end': { turn: number; step: number } /** A user-visible prompt (queued message drained at turn start). */ 'user/message': { content: ContentBlock[]; source: MessageSource } @@ -230,6 +242,11 @@ export interface SessionEventMap { * usage record). `usage` is absent when the adapter reported none. */ 'assistant/message': { turn: number; step: number; content: ContentBlock[]; usage?: TokenUsage } + /** + * The model requested one tool invocation: `name` with the raw `arguments` + * JSON string exactly as the model produced it (unparsed). `callId` pairs the + * call with its `tool/result`. + */ 'tool/call': { turn: number; step: number; callId: CallId; name: string; arguments: string } /** * A completed tool call's model-facing result, plus an optional tool-private diff --git a/packages/core/session/tests/gen-persistence-catalog.spec.ts b/packages/core/session/tests/gen-persistence-catalog.spec.ts new file mode 100644 index 0000000000..bc4ab4fbb1 --- /dev/null +++ b/packages/core/session/tests/gen-persistence-catalog.spec.ts @@ -0,0 +1,225 @@ +/** + * Negative-path tests for the persistence log catalog generator + * (`scripts/gen-persistence-catalog.ts`). + * + * The generated catalog is frozen by a regenerate-and-diff freshness gate, so + * the freshness half is exercised by `pnpm run verify-persistence-catalog` in + * CI. What a freshness diff CANNOT prove is that the generator REJECTS + * malformed source the way it promises to — a member without description + * prose, a forbidden `@mode` tag, a non-literal member name, a duplicate event + * declaration, a missing or ambiguous `SurfaceEventType` union, a stale union + * member. These tests drive the exported collectors against synthetic fixture + * packages to prove each guard fires (and that well-formed declarations pass), + * mirroring the gen-cordis-catalog negative tests. + */ + +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { + annotateSurface, + collectLogEvents, + collectSurfaceEventTypes, + render, +} from '../../../../scripts/gen-persistence-catalog.ts' + +/** Create a fixture scan root; `files` maps `packages/…`-relative paths to source. */ +function fixtureRoot(files: Record): string { + const root = mkdtempSync(join(tmpdir(), 'persistence-catalog-')) + for (const [rel, source] of Object.entries(files)) { + const abs = join(root, rel) + mkdirSync(join(abs, '..'), { recursive: true }) + writeFileSync(abs, source) + } + return root +} + +const roots: string[] = [] +const make = (files: Record): string => { + const r = fixtureRoot(files) + roots.push(r) + return r +} + +/** A merge-form declaration file wrapping `members` in the session module. */ +const merge = (members: string): string => + `declare module '@deepseek-ai/dsh-session' {\n interface SessionEventMap {\n${members}\n }\n}\n` + +afterEach(() => { + while (roots.length) rmSync(roots.pop()!, { recursive: true, force: true }) +}) + +/** The manifest that marks a fixture package as the owning session package. */ +const OWNER_MANIFEST = '{ "name": "@deepseek-ai/dsh-session" }\n' + +describe('gen-persistence-catalog collectLogEvents', () => { + it('extracts a documented member of the owning top-level interface', () => { + const events = collectLogEvents(make({ + 'packages/core/fix/package.json': OWNER_MANIFEST, + 'packages/core/fix/src/types.ts': + 'export interface SessionEventMap {\n /** A thing was recorded. */\n \'fix/happened\': { turn: number }\n}\n', + })) + expect(events).toHaveLength(1) + expect(events[0]).toMatchObject({ + name: 'fix/happened', + scope: 'fix', + doc: 'A thing was recorded.', + payload: '{ turn: number }', + source: 'packages/core/fix/src/types.ts:3', + }) + }) + + it('hard-errors on a top-level interface outside the owning package', () => { + expect(() => collectLogEvents(make({ + 'packages/group/alien/package.json': '{ "name": "@deepseek-ai/dsh-alien" }\n', + 'packages/group/alien/src/types.ts': + 'export interface SessionEventMap {\n /** Not the real vocabulary. */\n \'alien/event\': { turn: number }\n}\n', + }))).toThrow(/top-level interface SessionEventMap .* is outside @deepseek-ai\/dsh-session \(package @deepseek-ai\/dsh-alien\)/) + }) + + it('hard-errors on a non-exported top-level interface even in the owning package', () => { + expect(() => collectLogEvents(make({ + 'packages/core/fix/package.json': OWNER_MANIFEST, + 'packages/core/fix/src/helper.ts': + 'interface SessionEventMap {\n /** A local helper, not the vocabulary. */\n \'fix/local\': { turn: number }\n}\nexport const use: SessionEventMap | null = null\n', + }))).toThrow(/is not exported; the owning vocabulary is the single exported declaration/) + }) + + it('hard-errors when the owning interface is exported from two files', () => { + expect(() => collectLogEvents(make({ + 'packages/core/fix/package.json': OWNER_MANIFEST, + 'packages/core/fix/src/a.ts': 'export interface SessionEventMap {\n /** First home. */\n \'fix/a\': { turn: number }\n}\n', + 'packages/core/fix/src/b.ts': 'export interface SessionEventMap {\n /** Second home. */\n \'fix/b\': { turn: number }\n}\n', + }))).toThrow(/is already declared at packages\/core\/fix\/src\/a\.ts:1; the owning vocabulary has exactly one home/) + }) + + it('hard-errors on an extends clause (inherited keys would escape the catalog)', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': + 'interface Extra { \'fix/hidden\': { turn: number } }\ndeclare module \'@deepseek-ai/dsh-session\' {\n interface SessionEventMap extends Extra {\n /** Declared directly. */\n \'fix/direct\': { turn: number }\n }\n}\n', + }))).toThrow(/uses extends; inherited keys would join keyof SessionEventMap without a catalog row/) + }) + + it('extracts a member declaration-merged via the session module', () => { + const events = collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' /** Merged provenance. */\n \'fix/merged\': { id: string }'), + })) + expect(events).toHaveLength(1) + expect(events[0]).toMatchObject({ name: 'fix/merged', doc: 'Merged provenance.' }) + }) + + it('collapses a newline-separated multi-line payload to a valid one-line fragment', () => { + const events = collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge( + ' /** Wide payload. */\n \'fix/wide\': {\n alpha: string[]\n range: { start: number; end: number }\n count: number\n }', + ), + })) + expect(events[0]?.payload).toBe('{ alpha: string[]; range: { start: number; end: number }; count: number }') + }) + + it('hard-errors on a member with no description prose', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' \'fix/undocumented\': { turn: number }'), + }))).toThrow(/no description prose/) + }) + + it('hard-errors on an @mode tag (a log event has no dispatch mode)', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' /**\n * Documented, but mistagged.\n * @mode emit\n */\n \'fix/tagged\': { turn: number }'), + }))).toThrow(/carries an @mode tag/) + }) + + it('hard-errors on an extra-indented @mode tag (does not leak into prose)', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' /**\n * Documented, but mistagged.\n * @mode emit\n */\n \'fix/indented\': { turn: number }'), + }))).toThrow(/carries an @mode tag/) + }) + + it('hard-errors on a method-form member (it still joins keyof SessionEventMap)', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' /** Documented, wrong shape. */\n \'fix/method\'(turn: number): void'), + }))).toThrow(/not a property signature with an explicit payload type/) + }) + + it('hard-errors on a property member with no payload type annotation', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' /** Documented, no payload. */\n \'fix/bare\''), + }))).toThrow(/not a property signature with an explicit payload type/) + }) + + it('hard-errors on a non-literal member name', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' /** Not a literal. */\n unquoted: { turn: number }'), + }))).toThrow(/non-literal name/) + }) + + it('hard-errors when the same event is declared twice', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/a.ts': merge(' /** First. */\n \'fix/dup\': { turn: number }'), + 'packages/group/fix/src/b.ts': merge(' /** Second. */\n \'fix/dup\': { turn: number }'), + }))).toThrow(/already declared at packages\/group\/fix\/src\/a\.ts/) + }) + + it('aggregates every violation into one error instead of failing fast', () => { + expect(() => collectLogEvents(make({ + 'packages/group/fix/src/types.ts': merge(' \'fix/one\': { turn: number }\n \'fix/two\': { turn: number }'), + }))).toThrow(/2 JSDoc completeness violation\(s\)[\s\S]*fix\/one[\s\S]*fix\/two/) + }) +}) + +describe('gen-persistence-catalog collectSurfaceEventTypes', () => { + it('parses the literal union', () => { + const types = collectSurfaceEventTypes(make({ + 'packages/core/fix/src/types.ts': 'export type SurfaceEventType = \'fix/a\' | \'fix/b\'\n', + })) + expect(types).toEqual(['fix/a', 'fix/b']) + }) + + it('hard-errors when no union is declared', () => { + expect(() => collectSurfaceEventTypes(make({ + 'packages/core/fix/src/types.ts': 'export const unrelated = 1\n', + }))).toThrow(/no SurfaceEventType union found/) + }) + + it('hard-errors when the union is declared more than once', () => { + expect(() => collectSurfaceEventTypes(make({ + 'packages/core/fix/src/a.ts': 'export type SurfaceEventType = \'fix/a\'\n', + 'packages/core/fix/src/b.ts': 'export type SurfaceEventType = \'fix/b\'\n', + }))).toThrow(/declared more than once/) + }) + + it('hard-errors on a non-string-literal union member', () => { + expect(() => collectSurfaceEventTypes(make({ + 'packages/core/fix/src/types.ts': 'export type SurfaceEventType = \'fix/a\' | number\n', + }))).toThrow(/non-string-literal member/) + }) +}) + +describe('gen-persistence-catalog annotateSurface + render', () => { + const entry = (name: string) => ({ + name, + scope: name.split('/')[0] ?? name, + payload: '{ turn: number }', + doc: `Records ${name}.`, + source: 'packages/core/fix/src/types.ts:3', + }) + + it('badges union members surface and everything else log-only', () => { + const annotated = annotateSurface([entry('fix/message'), entry('fix/marker')], ['fix/message']) + expect(annotated.map(e => [e.name, e.surface])).toEqual([['fix/message', true], ['fix/marker', false]]) + }) + + it('hard-errors on a union member naming no declared event', () => { + expect(() => annotateSurface([entry('fix/marker')], ['fix/ghost'])) + .toThrow(/'fix\/ghost' name no declared log event/) + }) + + it('renders badges, payload fences, and the generated-file header', () => { + const out = render(annotateSurface([entry('fix/message'), entry('fix/marker')], ['fix/message'])) + expect(out).toContain('Generated by scripts/gen-persistence-catalog.ts') + expect(out).toContain('#### `fix/message` — surface') + expect(out).toContain('#### `fix/marker` — log-only') + expect(out).toContain('```ts persistence-catalog\n\'fix/marker\': { turn: number }\n```') + }) +}) diff --git a/packages/core/tools/src/index.ts b/packages/core/tools/src/index.ts index 063e14708e..dd0ed918db 100644 --- a/packages/core/tools/src/index.ts +++ b/packages/core/tools/src/index.ts @@ -317,21 +317,20 @@ export class ToolRegistry extends Service { /** * Return all registered tool schemas — exactly the model-facing fields - * (`name`, `description`, `parameters`, and `strict` when set), as sent to the - * model via the system-prompt assembly. Constructed EXPLICITLY rather than by - * stripping known non-schema members: a `ToolDefinition` also carries - * `execute` and the optional `presentCall`/`presentResult` UI callbacks, and - * those (especially the functions) must never leak into a model request. An - * allowlist can't drift when a new non-schema member is added to the - * definition; a denylist (rest-destructure) would silently leak it. + * (`name`, `description`, `parameters`), as sent to the model via the + * system-prompt assembly. Constructed EXPLICITLY rather than by stripping + * known non-schema members: a `ToolDefinition` also carries `execute` and the + * optional `presentCall`/`presentResult` UI callbacks, and those (especially + * the functions) must never leak into a model request. An allowlist can't + * drift when a new non-schema member is added to the definition; a denylist + * (rest-destructure) would silently leak it. * @returns one deep-cloned schema per registered tool, in registration order. */ schemas(): ToolSchema[] { - return [...this.store.values()].map(({ name, description, parameters, strict }): ToolSchema => ({ + return [...this.store.values()].map(({ name, description, parameters }): ToolSchema => ({ name, description, parameters: structuredClone(parameters), - ...strict !== undefined ? { strict } : {}, })) } diff --git a/packages/core/tools/src/schema.ts b/packages/core/tools/src/schema.ts index 0539c1f07b..9149241225 100644 --- a/packages/core/tools/src/schema.ts +++ b/packages/core/tools/src/schema.ts @@ -312,8 +312,6 @@ export interface DefineToolOptions { * free for the same replay reason. See {@link ToolResultView}. */ presentResult?(args: InferArgs, result: ToolResult): ToolResultView | undefined - /** Whether the tool requires structured output (default false). */ - strict?: boolean } /** @@ -355,7 +353,6 @@ export function defineTool(options: DefineToolOptions): name: options.name, description: options.description, parameters: schemaSpecToJsonSchema(options.parameters) as unknown as Record, - ...options.strict !== undefined ? { strict: options.strict } : {}, async execute(args: unknown, exec: ToolExecution): Promise { // Validate the model-generated args before the typed body runs. On // mismatch we throw ToolArgsError; the registry turns it into an diff --git a/packages/core/tools/tests/gen-tool-catalog.spec.ts b/packages/core/tools/tests/gen-tool-catalog.spec.ts index efc9ae44e5..cacc2eef66 100644 --- a/packages/core/tools/tests/gen-tool-catalog.spec.ts +++ b/packages/core/tools/tests/gen-tool-catalog.spec.ts @@ -106,17 +106,4 @@ describe('gen-tool-catalog render', () => { expect(md).toContain('```json') expect(md).toContain('Source: [`packages/demo/tool-demo/src/index.ts`]') }) - - it('renders the strict flag when a schema sets it', () => { - const catalog: ToolCatalog = [ - { - pkg: '@deepseek-ai/dsh-tool-demo', - source: 'packages/demo/tool-demo/src/index.ts', - requires: ['ctx.tools'], - writes: ['tool/result'], - schemas: [{ name: 'demo', description: '', parameters: { type: 'object', properties: {} }, strict: true }], - }, - ] - expect(render(catalog)).toContain('Strict: `true`') - }) }) diff --git a/packages/core/tools/tests/tools.spec.ts b/packages/core/tools/tests/tools.spec.ts index ca63338258..09158b8398 100644 --- a/packages/core/tools/tests/tools.spec.ts +++ b/packages/core/tools/tests/tools.spec.ts @@ -62,18 +62,6 @@ describe('ToolRegistry', () => { expect(schema.execute).toBeUndefined() }) - it('schemas() preserves `strict` when set (allowlist keeps the model-facing fields)', async () => { - const ctx = await setup() - ctx.tools.register(defineTool({ - name: 'strict-tool', - description: 'd', - parameters: { x: { type: 'string', required: true } }, - strict: true, - async execute() { return [] }, - })) - expect(ctx.tools.schemas()[0]).toMatchObject({ name: 'strict-tool', strict: true }) - }) - it('executes a tool and returns its content', async () => { const ctx = await setup() ctx.tools.register(echoTool) @@ -611,44 +599,6 @@ describe('schema DSL edge cases', () => { }) }) - it('defineTool passes through strict flag when set to true', () => { - const tool = defineTool({ - name: 'strict-tool', - description: 'A strict tool', - parameters: { input: { type: 'string' } }, - strict: true, - async execute(args) { - return [{ type: 'text' as const, text: args.input ?? '' }] - }, - }) - expect(tool.strict).toBe(true) - }) - - it('defineTool omits strict when not provided', () => { - const tool = defineTool({ - name: 'non-strict-tool', - description: 'A non-strict tool', - parameters: { input: { type: 'string' } }, - async execute(args) { - return [{ type: 'text' as const, text: args.input ?? '' }] - }, - }) - expect('strict' in tool).toBe(false) - }) - - it('defineTool strict=false is included', () => { - const tool = defineTool({ - name: 'explicitly-non-strict', - description: 'Explicitly non-strict', - parameters: { input: { type: 'string' } }, - strict: false, - async execute(args) { - return [{ type: 'text' as const, text: args.input ?? '' }] - }, - }) - expect(tool.strict).toBe(false) - }) - it('handles enum and default together in one property', () => { const spec = { level: { type: 'string', enum: ['low', 'high'], default: 'low' }, diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index 64754b2dd4..b1b9c25397 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -82,8 +82,9 @@ function versionOf(info: Stats): FsVersion { } /** - * Test seam: lets specs pin the temp-file name (to prove exclusive-open - * behavior) without a name race. + * Test seam: lets specs pin the atomic-write temp names (to prove + * exclusive-open behavior without a name race) and observe the staged temp + * file before it is renamed over the target. */ export interface FsIoInternals { /** Override the generated private staging-dir name (relative to the target dir). */ diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index abd8d047e6..0ad8365d4a 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -104,7 +104,7 @@ export class LocalFileSystem extends FileSystem { override async resolve(path: string, opts?: { cwd?: string }): Promise { const local = await resolveLocalTarget(opts?.cwd ?? this.config.cwd, path) - return { inputPath: path, targetKey: local.targetKey, displayPath: local.displayPath } + return { targetKey: local.targetKey, displayPath: local.displayPath } } override async stat(target: FsTarget, signal?: AbortSignal): Promise { @@ -127,7 +127,7 @@ export class LocalFileSystem extends FileSystem { return entries.map(entry => ({ name: entry.name, type: entry.type, - target: { inputPath: entry.name, targetKey: entry.target.targetKey, displayPath: entry.target.displayPath }, + target: { targetKey: entry.target.targetKey, displayPath: entry.target.displayPath }, ...(entry.version !== undefined ? { version: entry.version } : {}), ...(entry.size !== undefined ? { size: entry.size } : {}), })) @@ -210,8 +210,6 @@ export class LocalFileSystem extends FileSystem { const after = await probe(target.targetKey) return { - replacements: edited.replacements, - replaceAll: edit.replaceAll, version: this.versionAfterWrite(after, target), // The LF-normalized before/after text (the applied-hunk diff basis); // line-ending restoration is a storage detail the diff ignores. diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index 212074ddae..997021d346 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -138,12 +138,6 @@ describe('listDir', () => { join(dir, 'skills', 'dir-skill'), join(dir, 'skills', 'zeta.md'), ]) - expect(entries.map(entry => entry.target.inputPath)).toEqual([ - 'alpha.md', - 'broken-link', - 'dir-skill', - 'zeta.md', - ]) const materializedEntries = entries.filter(entry => entry.version !== undefined) expect(materializedEntries.map(entry => entry.target.targetKey)) .toEqual(await Promise.all(materializedEntries.map(entry => realpath(entry.target.displayPath)))) @@ -335,7 +329,7 @@ describe('editText', () => { await writeFile(join(dir, 'a.txt'), 'hello world') const target = await fs.resolve('a.txt') const outcome = await fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }, { version: await versionOf(target) }) - expect(outcome.replacements).toBe(1) + expect(outcome.after).toBe('hello there') expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') }) @@ -365,7 +359,7 @@ describe('editText', () => { const target = await fs.resolve('a.txt') // No version guard: any current content is edited, regardless of version. const outcome = await fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }) - expect(outcome.replacements).toBe(1) + expect(outcome.after).toBe('hello there') expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there') }) @@ -411,7 +405,7 @@ describe('editText', () => { await writeFile(join(dir, 'a.txt'), 'a a a') const target = await fs.resolve('a.txt') const outcome = await fs.editText(target, { oldString: 'a', newString: 'b', replaceAll: true }, { version: await versionOf(target) }) - expect(outcome.replacements).toBe(3) + expect(outcome.after).toBe('b b b') expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('b b b') }) @@ -457,7 +451,7 @@ describe('editText', () => { // The version the first edit returned is a valid guard for a second edit — // no intervening re-stat needed. const second = await fs.editText(target, { oldString: 'two', newString: 'TWO', replaceAll: false }, { version: first.version }) - expect(second.replacements).toBe(1) + expect(second.after).toBe('ONE TWO') expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('ONE TWO') }) diff --git a/packages/fs/fs-policy/tests/policy.spec.ts b/packages/fs/fs-policy/tests/policy.spec.ts index 2ea61ffcd1..c41cf45701 100644 --- a/packages/fs/fs-policy/tests/policy.spec.ts +++ b/packages/fs/fs-policy/tests/policy.spec.ts @@ -18,7 +18,7 @@ import * as FsPolicy from '@deepseek-ai/dsh-fs-policy' import type { FsPolicyExec } from '@deepseek-ai/dsh-fs-policy' function target(path: string): FsTarget { - return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } + return { targetKey: FsTargetKey(path), displayPath: path } } const ownerExec = (session: object): FsPolicyExec => ({ agent: { session } }) diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 4bd152cab9..7e7c7250de 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -19,7 +19,7 @@ A backend subclasses `FileSystem` and implements seven primitives. | Member | Semantics | |---|---| -| `resolve(path, opts?)` | Resolve a path into a stable `FsTarget` (`inputPath`, opaque `targetKey`, `displayPath`). `opts.cwd` is the base a relative `path` resolves against (a caller supplies its session workspace; absolute paths ignore it; omitted ⇒ the backend default). Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | +| `resolve(path, opts?)` | Resolve a path into a stable `FsTarget` (opaque `targetKey`, `displayPath`). `opts.cwd` is the base a relative `path` resolves against (a caller supplies its session workspace; absolute paths ignore it; omitted ⇒ the backend default). Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | | `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | | `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | | `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | diff --git a/packages/fs/fs/src/types.ts b/packages/fs/fs/src/types.ts index ed946389d2..9351a373db 100644 --- a/packages/fs/fs/src/types.ts +++ b/packages/fs/fs/src/types.ts @@ -52,8 +52,6 @@ export function FsVersion(v: string): FsVersion { * this; every other operation takes it. */ export interface FsTarget { - /** The original model/plugin-supplied path, for diagnostics only. */ - inputPath: string /** Opaque key for stale guards and target lookup. */ targetKey: FsTargetKey /** @@ -142,10 +140,6 @@ export interface FsEditRequest { /** Outcome of a literal edit. */ export interface FsEditOutcome { - /** Number of literal replacements applied. */ - replacements: number - /** Whether every match was replaced. */ - replaceAll: boolean /** Opaque version of the file after the edit. */ version: FsVersion /** diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index d4edd9260f..86ba782c96 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -23,7 +23,7 @@ class FakeFileSystem extends FileSystem { files = new Map() override async resolve(path: string): Promise { - return { inputPath: path, targetKey: FsTargetKey(path), displayPath: path } + return { targetKey: FsTargetKey(path), displayPath: path } } override async stat(target: FsTarget): Promise { const content = this.files.get(target.targetKey) @@ -45,7 +45,7 @@ class FakeFileSystem extends FileSystem { { name: 'alpha.md', type: 'file', - target: { inputPath: 'skills/alpha.md', targetKey: FsTargetKey('skills/alpha.md'), displayPath: 'skills/alpha.md' }, + target: { targetKey: FsTargetKey('skills/alpha.md'), displayPath: 'skills/alpha.md' }, size: 2, version: FsVersion('v1'), }, @@ -60,7 +60,7 @@ class FakeFileSystem extends FileSystem { const content = this.files.get(target.targetKey) ?? '' const after = content.split(edit.oldString).join(edit.newString) this.files.set(target.targetKey, after) - return { replacements: 1, replaceAll: edit.replaceAll, version: FsVersion('v3'), before: content, after } + return { version: FsVersion('v3'), before: content, after } } } @@ -108,7 +108,7 @@ describe('FileSystem provider seam', () => { expect(entries).toEqual([{ name: 'alpha.md', type: 'file', - target: { inputPath: 'skills/alpha.md', targetKey: 'skills/alpha.md', displayPath: 'skills/alpha.md' }, + target: { targetKey: 'skills/alpha.md', displayPath: 'skills/alpha.md' }, size: 2, version: 'v1', }]) diff --git a/packages/fs/tool-fs/src/edit.ts b/packages/fs/tool-fs/src/edit.ts index 1a39cb63dd..5ede545b03 100644 --- a/packages/fs/tool-fs/src/edit.ts +++ b/packages/fs/tool-fs/src/edit.ts @@ -16,7 +16,6 @@ import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' import type { DiffCallView, DiffResultView, ToolResult } from '@deepseek-ai/dsh-tools' import type { ContentBlock } from '@deepseek-ai/dsh-llm' -import type { FsEditOutcome } from '@deepseek-ai/dsh-fs' import type {} from '@deepseek-ai/dsh-fs' import type {} from '@deepseek-ai/dsh-system-prompt' import { computeHunkDiffs, diffsFromMeta, type FsDiffMeta } from './diff.ts' @@ -43,9 +42,9 @@ export function parseEditArgs(args: { file_path: string; old_string: string; new } } -/** Format an edit outcome as a Claude-style model-facing success message. */ -export function formatEditOutput(displayPath: string, outcome: FsEditOutcome): string { - return outcome.replaceAll +/** Format an edit success (single-match or replace-all) as a Claude-style model-facing message. */ +export function formatEditOutput(displayPath: string, replaceAll: boolean): string { + return replaceAll ? `The file ${displayPath} has been updated. All occurrences were successfully replaced.` : `The file ${displayPath} has been updated successfully.` } @@ -91,7 +90,7 @@ export function applyEditTool(ctx: Context): void { // relativizes it). const diffs = computeHunkDiffs(input.filePath, outcome.before, outcome.after) return { - content: [{ type: 'text', text: formatEditOutput(target.displayPath, outcome) }], + content: [{ type: 'text', text: formatEditOutput(target.displayPath, input.replaceAll) }], meta: { diffs }, } }, diff --git a/packages/fs/tool-fs/src/read-render.ts b/packages/fs/tool-fs/src/read-render.ts index ba0f01f214..c54279baa9 100644 --- a/packages/fs/tool-fs/src/read-render.ts +++ b/packages/fs/tool-fs/src/read-render.ts @@ -17,7 +17,6 @@ */ import { FsError } from '@deepseek-ai/dsh-fs' -import type { FsVersion } from '@deepseek-ai/dsh-fs' /** Default maximum characters returned for a single line (the `readMaxLineLength` config). */ export const READ_MAX_LINE_LENGTH = 2000 @@ -59,16 +58,12 @@ export interface WindowResult { export interface FileReadOutcome { /** 1-based first line requested. */ offset: number - /** Maximum number of lines requested. */ - limit: number /** Returned lines, already numbered. */ lines: FileTextLine[] /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */ totalLines: number /** Whether selected output hit the byte cap before EOF or the requested limit. */ truncatedByBytes?: true - /** Opaque version of the file at read time. */ - version: FsVersion } interface WindowAccumulator { diff --git a/packages/fs/tool-fs/src/read.ts b/packages/fs/tool-fs/src/read.ts index 68f21231b2..b211d6d80e 100644 --- a/packages/fs/tool-fs/src/read.ts +++ b/packages/fs/tool-fs/src/read.ts @@ -109,10 +109,8 @@ export function applyReadTool(ctx: Context, caps: ReadToolCaps): void { const outcome: FileReadOutcome = { offset: input.offset, - limit: input.limit, lines: window.lines, totalLines: window.totalLines, - version: info.version, ...window.truncatedByBytes ? { truncatedByBytes: true } : {}, } // Record the observed version (a no-op when no policy plugin listens). The diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index efad0a86f7..ac99d176d4 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -40,7 +40,7 @@ class FakeFs extends FileSystem { } override async resolve(path: string): Promise { - return { inputPath: path, targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` } + return { targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` } } override async stat(target: FsTarget): Promise { this.throwIfArmed() @@ -71,7 +71,7 @@ class FakeFs extends FileSystem { const content = this.files.get(target.targetKey) ?? '' const after = content.split(edit.oldString).join(edit.newString) this.files.set(target.targetKey, after) - return { replacements: 1, replaceAll: edit.replaceAll, version: FsVersion('v3'), before: content, after } + return { version: FsVersion('v3'), before: content, after } } } @@ -252,7 +252,7 @@ describe('read tool', () => { }) describe('formatReadOutput footer variants', () => { - const base: FileReadOutcome = { offset: 1, limit: 2000, lines: [{ number: 1, text: 'x' }], totalLines: 1, version: FsVersion('v') } + const base: FileReadOutcome = { offset: 1, lines: [{ number: 1, text: 'x' }], totalLines: 1 } it('reports a byte-capped read', () => { const out = formatReadOutput('/f', { ...base, totalLines: 99, truncatedByBytes: true }) diff --git a/packages/hooks/hook-protocol/README.md b/packages/hooks/hook-protocol/README.md index 8478f8aa74..64ed64b000 100644 --- a/packages/hooks/hook-protocol/README.md +++ b/packages/hooks/hook-protocol/README.md @@ -12,21 +12,18 @@ Why a shared lib at all: Codex deliberately reimplements a *subset* of the Claud | Run a hook | `runHook(bash, hook, opts, now)` — stdin payload + env via `ctx.bash`, decode | builds the per-event stdin **payload** + the dialect's **env** | | Decode output | `parseHookOutput(exit, stdout, stderr)` → neutral `HookOutput` | maps the neutral `HookOutput` onto a seam-specific typed Decision | | Merge N hooks | `mergeHookOutputs(outputs)` → most-restrictive `MergedHookOutcome` | — | -| Durable record | `appendHookInvoked` / `appendHookResult` (`hook/*` session events) | calls them around each invocation | +| Durable record | `appendHookInvoked` / `appendHookResult` (`hook/*` session events; the result's `decision`/`stderrSummary` derive from the `HookOutput` here) | calls them around each invocation | ## Primitives - **`matchesMatcher(matcher, query, mode)`** — match-all on absent/`''`/`'*'`; `claude` mode treats a pure `[A-Za-z0-9_|]+` pattern as a literal (pipe = exact-match alternation) and anything else as a regex; `codex` mode is always an unanchored regex. An invalid regex matches nothing (never throws). -- **`runHook(bash, hook, options, now)`** — serialize `options.payload` to the hook's stdin (with a trailing newline iff `options.trailingNewline`), merge `options.env` after the executor's credential scrub (the `dsh-bash` trusted-plugin surface), honor the hook's `timeoutSec` (else `defaultTimeoutMs`), and decode the result (threading `options.expectedEventName` to the codec). Never throws: an executor rejection (infra fault) becomes a `HookOutput` with `exitCode: undefined` (a non-blocking error). `now` is injected for testable durations. -- **`parseHookOutput(exitCode, stdout, stderr, expectedEventName?)`** — the exit-code + structured-stdout codec. Exit `0` → parse JSON stdout (lenient: non-JSON is left for the bridge); exit `2` → blocking error, `stderr` is the block reason (surfaced as `decision: 'block'`); other → non-blocking error. `hookSpecificOutput.permissionDecision` (allow/deny/ask) overrides a legacy top-level `decision`; `additionalContext`/`updatedInput`/`systemMessage`/`continue`/`stopReason`/`suppressOutput` are parsed too. The schemas key the `hookSpecificOutput` block by `hookEventName`, so passing `expectedEventName` (the firing event) DISCARDS a block whose `hookEventName` names a different event — or omits it entirely — its event-scoped fields don't take effect (a `PreToolUse` block on a `Stop` hook is malformed, and so is a discriminator-less block that would otherwise apply to any event), while the event-agnostic top-level fields still apply. Pure and total. +- **`runHook(bash, hook, options, now)`** — serialize `options.payload` to the hook's stdin (with a trailing newline iff `options.trailingNewline`), merge `options.env` after the executor's credential scrub (the `dsh-bash` trusted-plugin surface), honor the hook's `timeoutSec` (else `options.defaultTimeoutMs` — the bridge owns the default, its config defaulting to the lib's `DEFAULT_HOOK_TIMEOUT_MS` 10-minute reference), and decode the result (threading `options.expectedEventName` to the codec). Never throws: an executor rejection (infra fault) becomes a `HookOutput` with `exitCode: undefined` (a non-blocking error). `now` is injected for testable durations. +- **`parseHookOutput(exitCode, stdout, stderr, expectedEventName?)`** — the exit-code + structured-stdout codec. Exit `0` → parse JSON stdout (lenient: non-JSON is left for the bridge); exit `2` → blocking error, `stderr` is the block reason (surfaced as `decision: 'block'`); other → non-blocking error. `hookSpecificOutput.permissionDecision` (allow/deny/ask) overrides a legacy top-level `decision`; `additionalContext`/`updatedInput`/`systemMessage`/`continue`/`stopReason` are parsed too. The schemas key the `hookSpecificOutput` block by `hookEventName`, so passing `expectedEventName` (the firing event) DISCARDS a block whose `hookEventName` names a different event — or omits it entirely — its event-scoped fields don't take effect (a `PreToolUse` block on a `Stop` hook is malformed, and so is a discriminator-less block that would otherwise apply to any event), while the event-agnostic top-level fields still apply. Pure and total. - **`mergeHookOutputs(outputs)`** — fold the results of every hook that matched one point: permission precedence **deny > ask > allow**, halt sticky on the first `continue:false`, block reasons joined with `\n\n`, `additionalContext`/`systemMessages` accumulated in order. ## `hook/*` session events -Declaration-merged into `SessionEventMap` (log-only, like `compact/*` — NOT a `SurfaceEventType`, no `surfaceOp`): - -- `hook/invoked` — `{ turn, point, dialect, matcher?, handlerId }`: a hook command ran. -- `hook/result` — `{ turn, point, handlerId, decision, exitCode?, stderrSummary?, durationMs }`: its outcome, paired by `handlerId`. +Declaration-merged into `SessionEventMap` (log-only, like `compact/*` — NOT a `SurfaceEventType`, no `surfaceOp`): `hook/invoked` (a hook command ran) and `hook/result` (its outcome, paired by `handlerId`, with `appendHookResult` owning the decision rule). Payloads and per-event JSDoc are in the generated [persistence log event catalog](../../../docs/persistence-catalog/log-events.md); `stderrSummary` is truncated to the record's `stderrSummaryMaxChars` (the bridge's config, reference default `DEFAULT_STDERR_SUMMARY_MAX_CHARS` = 500; omitted when empty). Like every event they must sit inside an open turn. The mid-turn points (`PreToolUse`/`PostToolUse`/`UserPromptSubmit`/`Stop`) fire inside the loop's open turn by construction; `SessionStart` gets no `hook/*` record (its injected `context/message` is the durable evidence) — see the hooks RFC. diff --git a/packages/hooks/hook-protocol/src/codec.ts b/packages/hooks/hook-protocol/src/codec.ts index b5170028c2..7cf97feca9 100644 --- a/packages/hooks/hook-protocol/src/codec.ts +++ b/packages/hooks/hook-protocol/src/codec.ts @@ -21,7 +21,7 @@ import type { HookOutput } from './types.ts' /** The exit code a hook uses to signal a blocking error (stderr → model). */ -export const BLOCKING_EXIT_CODE = 2 +const BLOCKING_EXIT_CODE = 2 /** Read a string field from a parsed object, or `undefined` if absent/wrong type. */ function str(obj: Record, key: string): string | undefined { @@ -72,7 +72,7 @@ function permissionDecisionOf(value: string | undefined): HookOutput['decision'] * `additionalContext`/`updatedInput`) are DISCARDED — a `PreToolUse` block on a * `Stop` hook must not deny the `Stop`. The block's `hookEventName` is still * surfaced (for the log/diagnostics), and the event-agnostic top-level fields - * (`decision`/`reason`/`continue`/`stopReason`/`suppressOutput`/`systemMessage`) + * (`decision`/`reason`/`continue`/`stopReason`/`systemMessage`) * are unaffected. Omit `expectedEventName` (or pass a matching one) to apply the * block as-is — a caller that doesn't key by event opts out of the check. */ @@ -125,8 +125,6 @@ function applyStructured(output: HookOutput, parsed: Record, ex if (cont !== undefined) output.continue = cont const stopReason = str(parsed, 'stopReason') if (stopReason !== undefined) output.stopReason = stopReason - const suppress = bool(parsed, 'suppressOutput') - if (suppress !== undefined) output.suppressOutput = suppress const sysMsg = str(parsed, 'systemMessage') if (sysMsg !== undefined) output.systemMessage = sysMsg diff --git a/packages/hooks/hook-protocol/src/events.ts b/packages/hooks/hook-protocol/src/events.ts index d7529b0f45..5e14f964fb 100644 --- a/packages/hooks/hook-protocol/src/events.ts +++ b/packages/hooks/hook-protocol/src/events.ts @@ -16,7 +16,7 @@ */ import type { Session } from '@deepseek-ai/dsh-session' -import type { HookDialect } from './types.ts' +import type { HookDialect, HookOutput } from './types.ts' /** What identifies a hook invocation across its invoked/result pair. */ export interface HookInvocation { @@ -37,16 +37,42 @@ export interface HookResultRecord { turn: number point: string handlerId: string - /** The dialect-neutral decision the bridge resolved (`deny`/`allow`/`block`/…). */ - decision: string - /** The process exit code (absent when the hook could not run). */ - exitCode?: number - /** A truncated stderr summary (the block-reason source on exit 2). */ - stderrSummary?: string - /** Wall-clock duration of the run. */ + /** + * The decoded outcome the run produced. {@link appendHookResult} derives the + * durable `decision`/`exitCode`/`stderrSummary` fields from it, so the shared + * event's semantics live here, in the lib that declares it, not per-bridge. + */ + output: HookOutput + /** + * Character cap for the derived `stderrSummary`. The bound is the bridge's + * to own (its `stderrSummaryMaxChars` config) and is passed in explicitly — + * {@link DEFAULT_STDERR_SUMMARY_MAX_CHARS} is the reference default. + */ + stderrSummaryMaxChars: number + /** Wall-clock duration of the run (from `runHook`) — durable audit timing. */ durationMs: number } +/** + * The reference default for {@link HookResultRecord.stderrSummaryMaxChars} + * (both bridges' config default). It lives here, once, next to the truncation + * rule it bounds, so the bridges cannot drift apart on the shared event's + * default cap. + */ +export const DEFAULT_STDERR_SUMMARY_MAX_CHARS = 500 + +/** + * Truncate a hook's stderr for {@link HookResultRecord.stderrSummary}: trimmed, + * `undefined` when empty, cut at `maxChars` with an ellipsis when over. The + * bound is a parameter — like `runHook`'s `defaultTimeoutMs`, each bridge owns + * the config default and passes it in. + */ +export function summarizeStderr(stderr: string, maxChars: number): string | undefined { + const t = stderr.trim() + if (t.length === 0) return undefined + return t.length > maxChars ? t.slice(0, maxChars) + '…' : t +} + /** Append a `hook/invoked` provenance event to `session`. */ export function appendHookInvoked(session: Session, invocation: HookInvocation): void { session.append('hook/invoked', { @@ -59,26 +85,23 @@ export function appendHookInvoked(session: Session, invocation: HookInvocation): } /** - * Truncate a hook's stderr for {@link HookResultRecord.stderrSummary}: trimmed, - * `undefined` when empty, cut at `maxChars` with an ellipsis when over. The - * bound is a parameter — like `runHook`'s `defaultTimeoutMs`, each bridge owns - * the config default and passes it in. + * Append a `hook/result` outcome event to `session` (pairs with a prior + * `hook/invoked`). Owns the durable event's semantics: `decision` is the hook's + * parsed decision, else `'stop'` when it asked to halt (`continue: false`), + * else `'pass'`; `stderrSummary` is the trimmed stderr truncated to + * `record.stderrSummaryMaxChars` characters (omitted when empty); `exitCode` + * is omitted when the hook never ran. */ -export function summarizeStderr(stderr: string, maxChars: number): string | undefined { - const t = stderr.trim() - if (t.length === 0) return undefined - return t.length > maxChars ? t.slice(0, maxChars) + '…' : t -} - -/** Append a `hook/result` outcome event to `session` (pairs with a prior `hook/invoked`). */ export function appendHookResult(session: Session, record: HookResultRecord): void { + const { output } = record + const stderrSummary = summarizeStderr(output.stderr, record.stderrSummaryMaxChars) session.append('hook/result', { turn: record.turn, point: record.point, handlerId: record.handlerId, - decision: record.decision, - ...record.exitCode !== undefined ? { exitCode: record.exitCode } : {}, - ...record.stderrSummary !== undefined ? { stderrSummary: record.stderrSummary } : {}, + decision: output.decision ?? (output.continue === false ? 'stop' : 'pass'), + ...output.exitCode !== undefined ? { exitCode: output.exitCode } : {}, + ...stderrSummary !== undefined ? { stderrSummary } : {}, durationMs: record.durationMs, }) } diff --git a/packages/hooks/hook-protocol/src/index.ts b/packages/hooks/hook-protocol/src/index.ts index a99fd11b6f..fc9f658e6f 100644 --- a/packages/hooks/hook-protocol/src/index.ts +++ b/packages/hooks/hook-protocol/src/index.ts @@ -12,7 +12,9 @@ * - {@link mergeHookOutputs} — fold multiple matched hooks into one * most-restrictive {@link MergedHookOutcome} (deny > ask > allow). * - {@link appendHookInvoked} / {@link appendHookResult} — the log-only `hook/*` - * session-event helpers (declaration-merged into `SessionEventMap`). + * session-event helpers (declaration-merged into `SessionEventMap`); + * `appendHookResult` derives the durable `decision`/`stderrSummary` from the + * {@link HookOutput} so the shared event's semantics live in one place. * * Each bridge owns what genuinely DIFFERS: building the per-event stdin payload * (CC vs Codex field sets), the dialect's env/substitution, and mapping the @@ -29,10 +31,10 @@ export type { MatcherMode, } from './types.ts' export { matchesMatcher } from './matcher.ts' -export { BLOCKING_EXIT_CODE, parseHookOutput } from './codec.ts' -export { runHook } from './runner.ts' +export { parseHookOutput } from './codec.ts' +export { DEFAULT_HOOK_TIMEOUT_MS, runHook } from './runner.ts' export type { RunHookOptions, RunHookResult } from './runner.ts' export { mergeHookOutputs } from './merge.ts' export type { MergedDecision, MergedHookOutcome } from './merge.ts' -export { appendHookInvoked, appendHookResult, summarizeStderr } from './events.ts' +export { appendHookInvoked, appendHookResult, DEFAULT_STDERR_SUMMARY_MAX_CHARS, summarizeStderr } from './events.ts' export type { HookInvocation, HookResultRecord } from './events.ts' diff --git a/packages/hooks/hook-protocol/src/runner.ts b/packages/hooks/hook-protocol/src/runner.ts index cea09c1fe7..f5a892c468 100644 --- a/packages/hooks/hook-protocol/src/runner.ts +++ b/packages/hooks/hook-protocol/src/runner.ts @@ -17,6 +17,15 @@ import type { BashExecutor } from '@deepseek-ai/dsh-bash' import { parseHookOutput } from './codec.ts' import type { CommandHook, HookOutput } from './types.ts' +/** + * The reference default per-hook timeout, in ms (10 minutes) — the value both + * Claude Code and Codex apply to a hook whose config sets no `timeout`. It + * lives here, once, as the protocol's default; the bridges' `defaultTimeoutMs` + * config defaults to it, and a per-hook {@link CommandHook.timeoutSec} is the + * override surface. + */ +export const DEFAULT_HOOK_TIMEOUT_MS = 600_000 + /** Everything a single hook invocation needs beyond its command line. */ export interface RunHookOptions { /** The JSON payload object written to the hook's stdin (the bridge builds it). */ @@ -27,10 +36,14 @@ export interface RunHookOptions { cwd?: string /** Abort signal — cancels the hook run when fired (the parent step aborts). */ signal?: AbortSignal - /** Default timeout (ms) when the hook config sets none. */ - defaultTimeoutMs: number /** Whether to append a trailing newline to the stdin payload (CC yes, Codex no). */ trailingNewline: boolean + /** + * Timeout applied when the hook's config sets no `timeout` of its own. The + * bridge owns the default (its `defaultTimeoutMs` config, reference default + * {@link DEFAULT_HOOK_TIMEOUT_MS}) and passes it in explicitly. + */ + defaultTimeoutMs: number /** * The event this hook is firing for (e.g. `'PreToolUse'`). When set, a * structured `hookSpecificOutput` block whose `hookEventName` names a DIFFERENT @@ -43,18 +56,20 @@ export interface RunHookOptions { /** The {@link HookOutput} plus the wall-clock duration of the run (for `hook/result`). */ export interface RunHookResult { output: HookOutput + /** Wall-clock duration of the run, from `now` — durable on the `hook/result` event. */ durationMs: number } /** * Run `hook` via `bash` with `options.payload` serialized to its stdin, then - * decode the result. `now` is injected (a monotonic-ms source) so the duration - * is testable without a real clock. The hook's configured `timeoutSec` (wire - * unit: seconds) overrides `defaultTimeoutMs`. The command runs with the - * dialect's `env` merged after the executor's credential scrub (the trusted- - * plugin path). NEVER throws: an infrastructure failure (the executor rejecting) - * is surfaced as a {@link HookOutput} with `exitCode: undefined`, so the caller's - * merge logic treats it as a non-blocking error rather than crashing the turn. + * decode the result into a {@link HookOutput}. The hook's configured + * `timeoutSec` (wire unit: seconds) overrides `options.defaultTimeoutMs`. + * The command runs with the dialect's `env` merged after the executor's + * credential scrub (the trusted-plugin path). NEVER throws: an infrastructure + * failure (the executor rejecting) is surfaced as a {@link HookOutput} with + * `exitCode: undefined`, so the caller's merge logic treats it as a + * non-blocking error rather than crashing the turn. `now` is injected for + * testable durations. */ export async function runHook( bash: BashExecutor, diff --git a/packages/hooks/hook-protocol/src/types.ts b/packages/hooks/hook-protocol/src/types.ts index c3b75e7c08..ef9fc7f9b9 100644 --- a/packages/hooks/hook-protocol/src/types.ts +++ b/packages/hooks/hook-protocol/src/types.ts @@ -18,12 +18,11 @@ declare module '@deepseek-ai/dsh-session' { /** * A hook command was invoked at a hook point — log-only provenance (like * `compact/*`; NOT a {@link SurfaceEventType}, carries no `surfaceOp`). - * `dialect` is the bridge that ran it (`claude`/`codex`/`native`), `point` + * `dialect` is the bridge that ran it (`claude`/`codex`), `point` * the hook point (`PreToolUse`, `Stop`, …), `matcher` the matcher-group * pattern that selected it (absent for match-all), `handlerId` a stable id * for the command (so an invoked/result pair correlates). `turn` is the open * turn the invocation lives inside. - * @mode emit */ 'hook/invoked': { turn: number @@ -34,12 +33,14 @@ declare module '@deepseek-ai/dsh-session' { } /** * A hook command's outcome — log-only, paired with a prior `hook/invoked` - * (same `handlerId`). `decision` is the resolved dialect-neutral outcome the - * bridge mapped it to (`allow`/`deny`/`ask`/`block`/`continue`/`stop`/`pass`), - * `exitCode` the process exit (absent if it never ran), `stderrSummary` a - * truncated stderr (the block reason source on exit 2), `durationMs` the wall - * time. `turn` matches the `hook/invoked`. - * @mode emit + * (same `handlerId`). `decision` is the dialect-neutral outcome derived by + * `appendHookResult` (which owns the rule): the hook's parsed decision + * (`approve`/`allow`/`block`/`deny`/`ask`), else `'stop'` when it asked to + * halt via `continue:false`, else `'pass'`. `exitCode` is the process exit + * (absent if it never ran), `stderrSummary` the trimmed stderr truncated to + * the bridge's configured cap (the block reason source on exit 2), + * `durationMs` the wall-clock runtime (audit timing; snapshot replay + * normalizes it). `turn` matches the `hook/invoked`. */ 'hook/result': { turn: number @@ -53,8 +54,12 @@ declare module '@deepseek-ai/dsh-session' { } } -/** Which protocol dialect a hook config / invocation belongs to. */ -export type HookDialect = 'claude' | 'codex' | 'native' +/** + * The bridge that ran a hook — the CC bridge stamps `'claude'`, the Codex + * bridge `'codex'`. A native plugin on the interception seams is not a bridge + * and writes no `hook/*` provenance (see the interception-seams RFC). + */ +export type HookDialect = 'claude' | 'codex' /** * One configured command hook (the `{ type: 'command', command, timeout? }` @@ -115,8 +120,6 @@ export interface HookOutput { continue?: boolean /** Human-readable reason shown when {@link continue} is `false`. */ stopReason?: string - /** Hide the hook's stdout from the transcript (CC `suppressOutput`). */ - suppressOutput?: boolean /** * The neutral blocking decision a hook expressed, folded from the two channels * the reference protocols keep DISTINCT: the legacy top-level `decision` diff --git a/packages/hooks/hook-protocol/tests/codec.spec.ts b/packages/hooks/hook-protocol/tests/codec.spec.ts index 5f72753c57..252caa4cff 100644 --- a/packages/hooks/hook-protocol/tests/codec.spec.ts +++ b/packages/hooks/hook-protocol/tests/codec.spec.ts @@ -38,13 +38,12 @@ describe('parseHookOutput — exit code semantics', () => { }) describe('parseHookOutput — structured stdout (exit 0 only)', () => { - it('parses top-level continue/stopReason/suppressOutput/systemMessage', () => { + it('parses top-level continue/stopReason/systemMessage', () => { const out = parseHookOutput(0, JSON.stringify({ - continue: false, stopReason: 'budget exceeded', suppressOutput: true, systemMessage: 'heads up', + continue: false, stopReason: 'budget exceeded', systemMessage: 'heads up', }), '') expect(out.continue).toBe(false) expect(out.stopReason).toBe('budget exceeded') - expect(out.suppressOutput).toBe(true) expect(out.systemMessage).toBe('heads up') }) diff --git a/packages/hooks/hook-protocol/tests/events.spec.ts b/packages/hooks/hook-protocol/tests/events.spec.ts index 9f705916b9..f978da2645 100644 --- a/packages/hooks/hook-protocol/tests/events.spec.ts +++ b/packages/hooks/hook-protocol/tests/events.spec.ts @@ -1,6 +1,11 @@ import { describe, expect, it } from 'vitest' import { Session, SessionId } from '@deepseek-ai/dsh-session' -import { appendHookInvoked, appendHookResult, summarizeStderr } from '@deepseek-ai/dsh-hook-protocol' +import { appendHookInvoked, appendHookResult, summarizeStderr, type HookOutput } from '@deepseek-ai/dsh-hook-protocol' + +/** A {@link HookOutput} with the required stream fields defaulted. */ +function output(over: Partial = {}): HookOutput { + return { exitCode: 0, stderr: '', stdout: '', ...over } +} describe('hook/* session events', () => { it('appendHookInvoked records a log-only hook/invoked (with matcher when present)', () => { @@ -18,7 +23,7 @@ describe('hook/* session events', () => { it('omits matcher when absent (match-all hook)', () => { const session = new Session(SessionId('s')) - appendHookInvoked(session, { turn: 2, point: 'Stop', dialect: 'native', handlerId: 'h2' }) + appendHookInvoked(session, { turn: 2, point: 'Stop', dialect: 'codex', handlerId: 'h2' }) const ev = [...session.events].find(e => e.type === 'hook/invoked') if (ev?.type === 'hook/invoked') { @@ -26,32 +31,72 @@ describe('hook/* session events', () => { } }) - it('appendHookResult records the decided outcome, omitting absent optionals', () => { + it('appendHookResult derives decision/exitCode/stderrSummary from the output', () => { const session = new Session(SessionId('s')) appendHookResult(session, { - turn: 1, point: 'PreToolUse', handlerId: 'h1', decision: 'deny', - exitCode: 2, stderrSummary: 'blocked', durationMs: 12, + turn: 1, point: 'PreToolUse', handlerId: 'h1', + stderrSummaryMaxChars: 500, durationMs: 5, output: output({ exitCode: 2, stderr: 'blocked', decision: 'deny' }), }) const full = [...session.events].find(e => e.type === 'hook/result') if (full?.type === 'hook/result') { - expect(full.data).toMatchObject({ turn: 1, point: 'PreToolUse', handlerId: 'h1', decision: 'deny', exitCode: 2, stderrSummary: 'blocked', durationMs: 12 }) + expect(full.data).toEqual({ turn: 1, point: 'PreToolUse', handlerId: 'h1', decision: 'deny', exitCode: 2, stderrSummary: 'blocked', durationMs: 5 }) } // A result with no exit code / no stderr (e.g. a hook that could not run) omits both keys. const session2 = new Session(SessionId('s2')) - appendHookResult(session2, { turn: 1, point: 'Stop', handlerId: 'h3', decision: 'allow', durationMs: 3 }) + appendHookResult(session2, { + turn: 1, point: 'Stop', handlerId: 'h3', + stderrSummaryMaxChars: 500, durationMs: 5, output: output({ exitCode: undefined, decision: 'allow' }), + }) const sparse = [...session2.events].find(e => e.type === 'hook/result') if (sparse?.type === 'hook/result') { expect('exitCode' in sparse.data).toBe(false) expect('stderrSummary' in sparse.data).toBe(false) - expect(sparse.data.durationMs).toBe(3) + expect(sparse.data.decision).toBe('allow') + } + }) + + it('the decision falls back to stop on continue:false, else pass', () => { + const session = new Session(SessionId('s')) + appendHookResult(session, { turn: 1, point: 'Stop', handlerId: 'halt', stderrSummaryMaxChars: 500, durationMs: 5, output: output({ continue: false }) }) + appendHookResult(session, { turn: 1, point: 'Stop', handlerId: 'noop', stderrSummaryMaxChars: 500, durationMs: 5, output: output() }) + // An explicit decision wins over the continue:false fallback. + appendHookResult(session, { turn: 1, point: 'Stop', handlerId: 'both', stderrSummaryMaxChars: 500, durationMs: 5, output: output({ continue: false, decision: 'block' }) }) + + const decisions = [...session.events] + .filter(e => e.type === 'hook/result') + .map(e => e.type === 'hook/result' ? [e.data.handlerId, e.data.decision] : []) + expect(decisions).toEqual([['halt', 'stop'], ['noop', 'pass'], ['both', 'block']]) + }) + + it('stderrSummary is trimmed and truncated to 500 characters with an ellipsis', () => { + const session = new Session(SessionId('s')) + appendHookResult(session, { + turn: 1, point: 'PreToolUse', handlerId: 'long', + stderrSummaryMaxChars: 500, durationMs: 5, output: output({ exitCode: 2, stderr: ` ${'x'.repeat(600)} ` }), + }) + const ev = [...session.events].find(e => e.type === 'hook/result') + if (ev?.type === 'hook/result') { + expect(ev.data.stderrSummary).toBe('x'.repeat(500) + '…') + } + }) + + it('a 500-character stderr is kept verbatim (the cap is exclusive)', () => { + const session = new Session(SessionId('s')) + appendHookResult(session, { + turn: 1, point: 'PreToolUse', handlerId: 'edge', + stderrSummaryMaxChars: 500, durationMs: 5, output: output({ exitCode: 2, stderr: 'y'.repeat(500) }), + }) + const ev = [...session.events].find(e => e.type === 'hook/result') + if (ev?.type === 'hook/result') { + expect(ev.data.stderrSummary).toBe('y'.repeat(500)) } }) it('an invoked/result pair correlates by handlerId', () => { const session = new Session(SessionId('s')) appendHookInvoked(session, { turn: 1, point: 'PreToolUse', dialect: 'claude', handlerId: 'pair-1' }) - appendHookResult(session, { turn: 1, point: 'PreToolUse', handlerId: 'pair-1', decision: 'allow', exitCode: 0, durationMs: 7 }) + appendHookResult(session, { turn: 1, point: 'PreToolUse', handlerId: 'pair-1', stderrSummaryMaxChars: 500, durationMs: 5, output: output({ decision: 'allow' }) }) const invoked = [...session.events].find(e => e.type === 'hook/invoked') const result = [...session.events].find(e => e.type === 'hook/result') diff --git a/packages/hooks/hook-protocol/tests/runner.spec.ts b/packages/hooks/hook-protocol/tests/runner.spec.ts index 1cbe1b46de..1972a39c99 100644 --- a/packages/hooks/hook-protocol/tests/runner.spec.ts +++ b/packages/hooks/hook-protocol/tests/runner.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' import type { BashExecRequest, BashExecSpec, BashExecutor, BashRunResult } from '@deepseek-ai/dsh-bash' -import { runHook } from '@deepseek-ai/dsh-hook-protocol' +import { DEFAULT_HOOK_TIMEOUT_MS, runHook } from '@deepseek-ai/dsh-hook-protocol' /** * A minimal stand-in for the bits of {@link BashExecutor} that {@link runHook} @@ -89,6 +89,7 @@ describe('runHook — payload + env + stdin plumbing', () => { const { bash, specs } = recordingBash(async () => result()) await runHook(bash, { command: 'h' }, { payload: {}, defaultTimeoutMs: 60000, trailingNewline: true }, clock()) expect(specs[0]!.timeoutMs).toBe(60000) + expect(DEFAULT_HOOK_TIMEOUT_MS).toBe(600_000) // the CC/Codex reference default (10 minutes) }) it('passes the abort signal through', async () => { diff --git a/packages/hooks/hooks-claude/README.md b/packages/hooks/hooks-claude/README.md index 984a86ba18..554fb4b849 100644 --- a/packages/hooks/hooks-claude/README.md +++ b/packages/hooks/hooks-claude/README.md @@ -26,7 +26,7 @@ In a `cordis.yml`: projectDir: . ``` -The config is parsed **once** at load. `configPath` is **process-level**: a relative path resolves against the process's launch cwd at load time, so a single config applies to the whole process — there is no per-session (`session/new.cwd`) config discovery yet (`TODO(per-session-hook-config)`). A read/parse failure is contained — the bridge logs a warning and registers nothing rather than crashing boot (a typo'd path must not take the agent down). Only `type: 'command'` hooks run; a `prompt`/`agent`/HTTP hook is parsed-and-skipped with a warning. +The config is parsed **once** at load. `configPath` is **process-level**: a relative path resolves against the process's launch cwd at load time, so a single config applies to the whole process — there is no per-session (`session/new.cwd`) config discovery yet (`TODO(per-session-hook-config)`). A read/parse failure is contained — the bridge logs a warning and registers nothing rather than crashing boot (a typo'd path must not take the agent down). Only `type: 'command'` hooks run; a `prompt`/`agent`/HTTP hook is parsed-and-skipped with a warning. A hook with no per-hook `timeout` runs under the protocol's reference default (`DEFAULT_HOOK_TIMEOUT_MS` from `dsh-hook-protocol`, 10 minutes — the CC default). The hooks **themselves** run in the agent's session workspace: for the agent-scoped points the bridge passes the session's `cwd` (the `session/new.cwd`) as the hook process's working directory, so a hook's `pwd`/relative-path/marker operates in the user's project tree, not the server launch dir. diff --git a/packages/hooks/hooks-claude/src/index.ts b/packages/hooks/hooks-claude/src/index.ts index 06a405bf09..4ca9ee001e 100644 --- a/packages/hooks/hooks-claude/src/index.ts +++ b/packages/hooks/hooks-claude/src/index.ts @@ -31,10 +31,11 @@ import type { PostToolDecision, PreToolDecision, ToolExecution, ToolExecutionRes import { appendHookInvoked, appendHookResult, + DEFAULT_HOOK_TIMEOUT_MS, + DEFAULT_STDERR_SUMMARY_MAX_CHARS, matchesMatcher, mergeHookOutputs, runHook, - summarizeStderr, type HookOutput, type MatcherGroup, type MergedHookOutcome, @@ -82,8 +83,8 @@ export const Config: z = z.object({ configPath: z.string().required(), pluginRoot: z.string(), projectDir: z.string(), - defaultTimeoutMs: z.number().default(600_000), - stderrSummaryMaxChars: z.number().default(500), + defaultTimeoutMs: z.number().default(DEFAULT_HOOK_TIMEOUT_MS), + stderrSummaryMaxChars: z.number().default(DEFAULT_STDERR_SUMMARY_MAX_CHARS), }) /** A stable per-handler id so an invoked/result pair correlates in the log. */ @@ -105,8 +106,9 @@ function assertPositiveInteger(name: string, value: number): void { export function apply(ctx: Context, config: Config): void { // Validate the cap BEFORE the config-file parse: a bad value must fail the // load loudly, not be skipped by the parse-failure early return. - const stderrSummaryMaxChars = config.stderrSummaryMaxChars ?? 500 + const stderrSummaryMaxChars = config.stderrSummaryMaxChars ?? DEFAULT_STDERR_SUMMARY_MAX_CHARS assertPositiveInteger('stderrSummaryMaxChars', stderrSummaryMaxChars) + const defaultTimeoutMs = config.defaultTimeoutMs ?? DEFAULT_HOOK_TIMEOUT_MS // --- Parse the config ONCE at load. A read/parse failure is contained: the // bridge logs and registers nothing rather than crashing boot (a typo'd path // must not take the agent down). --- @@ -126,8 +128,6 @@ export function apply(ctx: Context, config: Config): void { return } - const defaultTimeoutMs = config.defaultTimeoutMs ?? 600_000 - /** * Run every command hook configured for `point` whose matcher selects * `matchQuery`, with the per-event `payload` on stdin, and fold the results. @@ -173,10 +173,10 @@ export function apply(ctx: Context, config: Config): void { } const { output, durationMs } = await runHook(ctx.bash, hook, { payload, + defaultTimeoutMs, ...hookEnv ? { env: hookEnv } : {}, ...workdir !== undefined ? { cwd: workdir } : {}, ...opts.signal ? { signal: opts.signal } : {}, - defaultTimeoutMs, trailingNewline: true, // Discard a `hookSpecificOutput` block whose `hookEventName` names a // different event than the one firing (the schemas key it by event). @@ -190,14 +190,7 @@ export function apply(ctx: Context, config: Config): void { ctx.logger.warn(`hooks-claude: ${point} hook emitted a systemMessage, which is not yet surfaced (ignored)`) } if (session && opts.turn !== undefined) { - const stderrSummary = summarizeStderr(output.stderr, stderrSummaryMaxChars) - appendHookResult(session, { - turn: opts.turn, point, handlerId, - decision: output.decision ?? (output.continue === false ? 'stop' : 'pass'), - ...output.exitCode !== undefined ? { exitCode: output.exitCode } : {}, - ...stderrSummary !== undefined ? { stderrSummary } : {}, - durationMs, - }) + appendHookResult(session, { turn: opts.turn, point, handlerId, output, stderrSummaryMaxChars, durationMs }) } } } diff --git a/packages/hooks/hooks-claude/tests/coverage.spec.ts b/packages/hooks/hooks-claude/tests/coverage.spec.ts index 45ca8f113b..f06cdcf6b4 100644 --- a/packages/hooks/hooks-claude/tests/coverage.spec.ts +++ b/packages/hooks/hooks-claude/tests/coverage.spec.ts @@ -322,8 +322,8 @@ describe('hooks-claude coverage — more default/sparse arms', () => { }) }) -describe('hooks-claude coverage — schema-bypass default + unspawnable hook', () => { - it('a direct apply() (schema bypass) defaults the timeout and runs', async () => { +describe('hooks-claude coverage — schema-bypass apply + unspawnable hook', () => { + it('a direct apply() (schema bypass) with only configPath runs', async () => { const d = dir() const marker = join(d, 'ran') const s = sh(d, 'h.sh', `#!/usr/bin/env bash\ntouch "${marker}"\n`) @@ -337,8 +337,9 @@ describe('hooks-claude coverage — schema-bypass default + unspawnable hook', ( await ctx.plugin(AgentRegistry) await ctx.plugin(AgentLoop, { agents: [] }) await ctx.plugin(LocalBashExecutor, { timeoutMs: 10_000 }) - // Direct apply with only configPath — bypasses schemastery's defaults, so the - // runtime `defaultTimeoutMs ?? 600_000` fallback is exercised. + // Direct apply with only configPath — bypasses schemastery's defaults, so + // the bridge must run on the raw minimal config (the per-hook timeout is + // the protocol lib's reference default, not a config knob). HooksClaude.apply(ctx, { configPath: join(d, 'hooks.json') }) ctx.llm.registerAdapter(['mock'], adapter) const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) diff --git a/packages/hooks/hooks-codex/README.md b/packages/hooks/hooks-codex/README.md index 68de9a49ae..fa060b46af 100644 --- a/packages/hooks/hooks-codex/README.md +++ b/packages/hooks/hooks-codex/README.md @@ -32,7 +32,7 @@ In a `cordis.yml`: model: deepseek-v4 ``` -The config is parsed **once** at load. `configPath` is **process-level** — a relative path resolves against the process launch cwd at load time, not per-session (`TODO(per-session-hook-config)`). A read/parse failure is contained (logs + registers nothing). Only sync `type: 'command'` hooks run — a non-command or `async: true` hook is parsed-and-skipped with a warning. A hook accepts `timeout` or the `timeoutSec` alias. Events outside the five Codex points are dropped at parse. +The config is parsed **once** at load. `configPath` is **process-level** — a relative path resolves against the process launch cwd at load time, not per-session (`TODO(per-session-hook-config)`). A read/parse failure is contained (logs + registers nothing). Only sync `type: 'command'` hooks run — a non-command or `async: true` hook is parsed-and-skipped with a warning. A hook accepts `timeout` or the `timeoutSec` alias; one that sets neither runs under the protocol's reference default (`DEFAULT_HOOK_TIMEOUT_MS` from `dsh-hook-protocol`, 10 minutes). Events outside the five Codex points are dropped at parse. The hooks themselves run in the agent's session workspace: for the agent-scoped points the bridge passes the session's `cwd` as the hook process's working directory, so a hook operates in the user's project tree, not the server launch dir. diff --git a/packages/hooks/hooks-codex/src/index.ts b/packages/hooks/hooks-codex/src/index.ts index accc36714a..8c15ba5a87 100644 --- a/packages/hooks/hooks-codex/src/index.ts +++ b/packages/hooks/hooks-codex/src/index.ts @@ -24,10 +24,11 @@ import type { PostToolDecision, PreToolDecision, ToolExecution, ToolExecutionRes import { appendHookInvoked, appendHookResult, + DEFAULT_HOOK_TIMEOUT_MS, + DEFAULT_STDERR_SUMMARY_MAX_CHARS, matchesMatcher, mergeHookOutputs, runHook, - summarizeStderr, type HookOutput, type MatcherGroup, type MergedHookOutcome, @@ -57,8 +58,8 @@ export interface Config { export const Config: z = z.object({ configPath: z.string().required(), model: z.string().default(''), - defaultTimeoutMs: z.number().default(600_000), - stderrSummaryMaxChars: z.number().default(500), + defaultTimeoutMs: z.number().default(DEFAULT_HOOK_TIMEOUT_MS), + stderrSummaryMaxChars: z.number().default(DEFAULT_STDERR_SUMMARY_MAX_CHARS), }) let handlerCounter = 0 @@ -78,8 +79,9 @@ function assertPositiveInteger(name: string, value: number): void { export function apply(ctx: Context, config: Config): void { // Validate the cap BEFORE the config-file parse: a bad value must fail the // load loudly, not be skipped by the parse-failure early return. - const stderrSummaryMaxChars = config.stderrSummaryMaxChars ?? 500 + const stderrSummaryMaxChars = config.stderrSummaryMaxChars ?? DEFAULT_STDERR_SUMMARY_MAX_CHARS assertPositiveInteger('stderrSummaryMaxChars', stderrSummaryMaxChars) + const defaultTimeoutMs = config.defaultTimeoutMs ?? DEFAULT_HOOK_TIMEOUT_MS let parsed: CodexHookConfig = {} try { const raw: unknown = JSON.parse(readFileSync(config.configPath, 'utf8')) @@ -93,7 +95,6 @@ export function apply(ctx: Context, config: Config): void { return } - const defaultTimeoutMs = config.defaultTimeoutMs ?? 600_000 const model = config.model ?? '' async function runPoint( @@ -122,9 +123,9 @@ export function apply(ctx: Context, config: Config): void { } const { output, durationMs } = await runHook(ctx.bash, hook, { payload, + defaultTimeoutMs, ...workdir !== undefined ? { cwd: workdir } : {}, ...opts.signal ? { signal: opts.signal } : {}, - defaultTimeoutMs, trailingNewline: false, // Codex writes stdin WITHOUT a trailing newline. // Discard a `hookSpecificOutput` block naming a different event. expectedEventName: point, @@ -149,14 +150,7 @@ export function apply(ctx: Context, config: Config): void { ctx.logger.warn(`hooks-codex: ${point} hook emitted a systemMessage, which is not yet surfaced (ignored)`) } if (session && opts.turn !== undefined) { - const stderrSummary = summarizeStderr(output.stderr, stderrSummaryMaxChars) - appendHookResult(session, { - turn: opts.turn, point, handlerId, - decision: output.decision ?? (output.continue === false ? 'stop' : 'pass'), - ...output.exitCode !== undefined ? { exitCode: output.exitCode } : {}, - ...stderrSummary !== undefined ? { stderrSummary } : {}, - durationMs, - }) + appendHookResult(session, { turn: opts.turn, point, handlerId, output, stderrSummaryMaxChars, durationMs }) } } } diff --git a/packages/hooks/hooks-codex/tests/coverage.spec.ts b/packages/hooks/hooks-codex/tests/coverage.spec.ts index 040425cbbe..c83137df53 100644 --- a/packages/hooks/hooks-codex/tests/coverage.spec.ts +++ b/packages/hooks/hooks-codex/tests/coverage.spec.ts @@ -229,7 +229,7 @@ describe('hooks-codex coverage — decision mapping paths', () => { expect(res?.type === 'hook/result' && res.data.stderrSummary).toBe('x'.repeat(40) + '…') }) - it('warns on a skipped async hook and a direct apply() defaults the timeout', async () => { + it('warns on a skipped async hook and a direct apply() (schema bypass) runs', async () => { const d = dir() const marker = join(d, 'ran') hooks(d, { UserPromptSubmit: [{ hooks: [ @@ -243,7 +243,7 @@ describe('hooks-codex coverage — decision mapping paths', () => { await ctx.plugin(ToolRegistry); await ctx.plugin(AgentRegistry); await ctx.plugin(AgentLoop, { agents: [] }) await ctx.plugin(LocalBashExecutor, { timeoutMs: 10_000 }) ctx.logger.warn = warn as never - // Direct apply (schema bypass) → defaultTimeoutMs ?? 600_000 + model ?? '' fallbacks. + // Direct apply (schema bypass) → the `model ?? ''` fallback is exercised. HooksCodex.apply(ctx, { configPath: join(d, 'hooks.json') }) ctx.llm.registerAdapter(['mock'], adapter) const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' }) diff --git a/packages/llm/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md index 76af182580..15fad9f881 100644 --- a/packages/llm/llm-deepseek/README.md +++ b/packages/llm/llm-deepseek/README.md @@ -23,18 +23,19 @@ A second, independent implementation of the same seam exists in `@deepseek-ai/ds `thinking`/`reasoningEffort` are adapter-level request defaults serialized as the official top-level `thinking: {type}` / `reasoning_effort` wire fields. They live in adapter config (not `GenerateOptions`) to keep the core vocabulary provider-neutral. +## App attribution + +Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. + ## Wire-format notes (verified live + against the official docs) - Streaming only (`stream_options.include_usage` always on). `usage` may arrive attached to the finish chunk or as a trailing usage-only chunk — the translator defers both to `[DONE]`, so `usage` always precedes `finish` and nothing follows `finish`. - The first thinking-mode chunk carries `reasoning_content: ""` — handled (no spurious reasoning block). - **Reasoning passback rule**: on assistant turns that carried tool calls, `reasoning_content` is serialized back in history (required by the API in thinking mode); on tool-call-free turns it is dropped (ignored anyway — saves tokens). -- `strict` on tool schemas passes through (officially Beta; the public API wants the `/beta` base URL for it, the internal endpoint accepts it directly). - Cache accounting: `cacheReadTokens` ← `prompt_cache_hit_tokens` / `prompt_tokens_details.cached_tokens`; DeepSeek reports no cache-write metric. ## Limitations (MVP, documented deliberately) -- `prefill` throws `LlmError('UNSUPPORTED')` — DeepSeek's chat-prefix completion is a Beta feature on the `/beta` base URL; future work. -- `image` blocks are skipped (no vision support on these models). - `tool_choice` is not mapped (not part of the core vocabulary). ## Errors diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index fda527359a..9e5e5179b6 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -5,7 +5,7 @@ * @module dsh-llm-deepseek/adapter */ -import { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { serializeRequest } from './serialize.ts' import type { RequestDefaults } from './serialize.ts' @@ -21,13 +21,6 @@ export interface DeepSeekAdapterOptions { defaults?: RequestDefaults } -/** - * Attribution header sent on every request so the provider can identify the - * client. Bump in lockstep with this package's version (no build-time version - * injection is wired in this repo yet). - */ -const USER_AGENT = 'deepseek-harness/0.0.1' - /** Map an HTTP status to a stable LlmError code. */ export function httpErrorCode(status: number): string { if (status === 401 || status === 403) return 'AUTH' @@ -67,7 +60,7 @@ export class DeepSeekAdapter extends LlmAdapter { 'authorization': `Bearer ${this.options.apiKey}`, 'content-type': 'application/json', 'accept': 'text/event-stream', - 'user-agent': USER_AGENT, + ...attributionHeaders(), }, body: JSON.stringify(body), ...options.signal ? { signal: options.signal } : {}, diff --git a/packages/llm/llm-deepseek/src/serialize.ts b/packages/llm/llm-deepseek/src/serialize.ts index 4e967d6667..bbca37223f 100644 --- a/packages/llm/llm-deepseek/src/serialize.ts +++ b/packages/llm/llm-deepseek/src/serialize.ts @@ -11,12 +11,10 @@ * rule for thinking mode — required there, ignored elsewhere, so we save * the tokens elsewhere); `tool-call` → `tool_calls[]` * - `tool-result` → its own `{role: 'tool'}` message (text flattened) - * - `image` → skipped (MVP limitation, documented in the README) * * @module dsh-llm-deepseek/serialize */ -import { LlmError } from '@deepseek-ai/dsh-llm' import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm' import type { WireMessage, WireRequest, WireTool } from './types.ts' @@ -99,19 +97,8 @@ export function serializeMessages(messages: Message[]): WireMessage[] { return wire } -/** - * Build the full wire request. Throws `LlmError('UNSUPPORTED')` for - * `prefill` (DeepSeek's chat-prefix completion is a Beta feature on a - * different base URL — see README). - */ +/** Build the full wire request. */ export function serializeRequest(options: GenerateOptions, defaults: RequestDefaults = {}): WireRequest { - if (options.prefill !== undefined) { - throw new LlmError( - 'prefill is not supported by the DeepSeek adapter (Beta chat-prefix completion is future work)', - 'UNSUPPORTED', - ) - } - const messages: WireMessage[] = [] if (options.system !== undefined) { messages.push({ role: 'system', content: options.system }) @@ -124,8 +111,6 @@ export function serializeRequest(options: GenerateOptions, defaults: RequestDefa name: tool.name, description: tool.description, parameters: tool.parameters, - // strict is officially supported (Beta); pass the tool author's choice. - ...tool.strict !== undefined ? { strict: tool.strict } : {}, }, })) diff --git a/packages/llm/llm-deepseek/src/types.ts b/packages/llm/llm-deepseek/src/types.ts index 5c212897fd..072c43babb 100644 --- a/packages/llm/llm-deepseek/src/types.ts +++ b/packages/llm/llm-deepseek/src/types.ts @@ -78,8 +78,6 @@ export interface WireTool { name: string description: string parameters: Record - /** Beta: strict schema adherence (official: requires the /beta base URL). */ - strict?: boolean } } diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 1abbebc060..46f123a1c7 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -2,7 +2,7 @@ import { createServer } from 'node:http' import type { IncomingMessage, Server, ServerResponse } from 'node:http' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import LlmService, { LlmError } from '@deepseek-ai/dsh-llm' +import LlmService, { LlmError, userAgent } from '@deepseek-ai/dsh-llm' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { DeepSeekAdapter, httpErrorCode } from '@deepseek-ai/dsh-llm-deepseek' import { assemble } from './assemble.ts' @@ -109,8 +109,12 @@ describe('DeepSeekAdapter against a mock server', () => { stream: true, stream_options: { include_usage: true }, }) - // Attribution header identifies the harness to the provider. - expect(server.headers[0]?.['user-agent']).toMatch(/^deepseek-harness\//) + // Attribution reaches the wire: the exact shared User-Agent, and no + // provider-specific headers under the User-Agent-only contract. + expect(server.headers[0]?.['user-agent']).toBe(userAgent()) + expect(server.headers[0]).not.toHaveProperty('http-referer') + expect(server.headers[0]).not.toHaveProperty('x-openrouter-title') + expect(server.headers[0]).not.toHaveProperty('x-openrouter-categories') }) it('streams raw chunks through ctx.llm.stream', async () => { diff --git a/packages/llm/llm-deepseek/tests/serialize.spec.ts b/packages/llm/llm-deepseek/tests/serialize.spec.ts index 04387d7aa8..3e533f8e7c 100644 --- a/packages/llm/llm-deepseek/tests/serialize.spec.ts +++ b/packages/llm/llm-deepseek/tests/serialize.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' -import { CallId, LlmError } from '@deepseek-ai/dsh-llm' -import type { GenerateOptions, Message } from '@deepseek-ai/dsh-llm' +import { CallId } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm' import { serializeMessages, serializeRequest } from '@deepseek-ai/dsh-llm-deepseek' function request(overrides: Partial = {}): GenerateOptions { @@ -110,11 +110,17 @@ describe('serializeMessages', () => { ]) }) - it('skips image blocks (documented MVP limitation)', () => { + it('skips plugin-added block types (merge-extensible ContentBlockMap)', () => { const wire = serializeMessages([ - { role: 'user', content: [{ type: 'image', url: 'data:image/png;base64,x' }, { type: 'text', text: 'see image' }] }, + { + role: 'user', + content: [ + { type: 'chart', data: 'x' } as unknown as ContentBlock, + { type: 'text', text: 'see chart' }, + ], + }, ]) - expect(wire).toEqual([{ role: 'user', content: 'see image' }]) + expect(wire).toEqual([{ role: 'user', content: 'see chart' }]) }) it('emits an empty user message rather than dropping block-less messages', () => { @@ -149,17 +155,17 @@ describe('serializeRequest', () => { expect(wire.stop).toEqual(['END']) }) - it('maps tools with strict passthrough', () => { + it('maps tools to the wire function shape', () => { const wire = serializeRequest(request({ messages: history, tools: [ { name: 'a', description: 'A', parameters: { type: 'object', properties: {} } }, - { name: 'b', description: 'B', parameters: { type: 'object', properties: {} }, strict: true }, + { name: 'b', description: 'B', parameters: { type: 'object', properties: { x: { type: 'string' } } } }, ], })) expect(wire.tools).toEqual([ { type: 'function', function: { name: 'a', description: 'A', parameters: { type: 'object', properties: {} } } }, - { type: 'function', function: { name: 'b', description: 'B', parameters: { type: 'object', properties: {} }, strict: true } }, + { type: 'function', function: { name: 'b', description: 'B', parameters: { type: 'object', properties: { x: { type: 'string' } } } } }, ]) }) @@ -179,17 +185,6 @@ describe('serializeRequest', () => { expect(wire.thinking).toBeUndefined() expect(wire.reasoning_effort).toBeUndefined() }) - - it('rejects prefill with an UNSUPPORTED LlmError', () => { - expect(() => serializeRequest(request({ prefill: [{ type: 'text', text: 'Sure' }] }))) - .toThrow(LlmError) - try { - serializeRequest(request({ prefill: [] })) - expect.unreachable() - } catch (error) { - expect((error as LlmError).code).toBe('UNSUPPORTED') - } - }) }) describe('review fixes: assistant content shapes', () => { diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index 70f61fdb62..f74ccd2246 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -9,7 +9,7 @@ DeepSeek adapter for the harness LLM seam backed by [`@earendil-works/pi-ai`](ht - pi-ai hands tool-call `arguments` around as **parsed objects**; the harness keeps raw JSON strings. The adapter patches replay payloads back to the original raw strings before sending them, and re-stringifies parsed output tool calls at `block-end`. - pi-ai reports failures as **in-stream error events** (it never throws mid-stream); these map to `finish {kind:'error'|'aborted'}` chunks — the protocol's other sanctioned error path besides throwing (which llm-deepseek uses). - pi-ai folds reasoning tokens into `usage.output`; there is no separate reasoning count to map. -- pi-ai's options omit some DeepSeek/OpenAI-compatible details; the adapter uses its `onPayload` hook to preserve the harness contract (`stop`, per-tool `strict`, omitted reasoning effort, raw replayed tool arguments). +- pi-ai's options omit some DeepSeek/OpenAI-compatible details; the adapter uses its `onPayload` hook to preserve the harness contract (`stop`, scrubbing pi-ai's own per-tool `strict` default — the hand-rolled twin sends no such field — omitted reasoning effort, raw replayed tool arguments). ## Config @@ -25,13 +25,17 @@ Same shape as llm-deepseek (one-line swap in cordis.yml), with pi-ai's thinking- reasoning: high # off | high | xhigh (xhigh → wire 'max') ``` +## App attribution + +Every request carries the shared attribution header from dsh-llm's `attributionHeaders()`, passed through pi-ai's `headers` stream option (pi-ai merges caller headers last, so it always reaches the wire - the unit suite asserts arrival on the mock server, same as llm-deepseek). OpenRouter-specific app attribution headers are intentionally not sent by this adapter contract; they are deferred to a future explicit OpenRouter adapter or mode. See [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts). + ## Dependency weight pi-ai declares the openai/anthropic/google/mistral/AWS SDKs as install-time dependencies. They are lazy-loaded — only the openai SDK actually loads for this adapter — but they do land in `node_modules`. Accepted for a package whose purpose is design verification. ## Limitations -Same MVP contract as llm-deepseek: `prefill` throws `UNSUPPORTED`, images are not representable, `tool_choice` is not mapped. +Same MVP contract as llm-deepseek: `tool_choice` is not mapped. ## Testing diff --git a/packages/llm/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts index e15cce8252..a24e335d8b 100644 --- a/packages/llm/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -13,9 +13,9 @@ import { stream as piStream } from '@earendil-works/pi-ai' import type { Model } from '@earendil-works/pi-ai' -import { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import { attributionHeaders, LlmAdapter } from '@deepseek-ai/dsh-llm' import { CallId } from '@deepseek-ai/dsh-llm' -import type { GenerateOptions, StreamChunk, ToolSchema } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { toPiContext, toStreamChunks } from './convert.ts' /** Reasoning levels surfaced by this adapter (DeepSeek wire: high|max). */ @@ -61,7 +61,7 @@ export function buildModel(modelId: string, options: PiAiAdapterOptions): Model< } type Payload = { - tools?: { function?: { name?: unknown; strict?: unknown } }[] + tools?: { function?: { strict?: unknown } }[] messages?: { role?: unknown tool_calls?: { id?: unknown; function?: { arguments?: unknown } }[] @@ -81,10 +81,6 @@ function rawToolArguments(options: GenerateOptions): Map { return raw } -function strictByToolName(tools: ToolSchema[] | undefined): Map { - return new Map((tools ?? []).map(tool => [tool.name, tool.strict])) -} - function patchPayload(payload: unknown, options: GenerateOptions, reasoning: PiAiReasoning | undefined): unknown { /* v8 ignore next -- pi-ai onPayload always receives an object; tolerate unusual future hooks defensively */ if (typeof payload !== 'object' || payload === null) return payload @@ -97,16 +93,13 @@ function patchPayload(payload: unknown, options: GenerateOptions, reasoning: PiA body.stop = options.stop } - const strictByName = strictByToolName(options.tools) + // pi-ai stamps its own `strict` default on every serialized tool; the + // harness tool contract has no strict field and the hand-rolled twin sends + // none, so scrub it for wire parity. for (const tool of body.tools ?? []) { /* v8 ignore next -- malformed pi-ai payload guard: real tool entries always carry function */ if (tool.function === undefined) continue - const name = tool.function.name - /* v8 ignore next -- malformed pi-ai payload guard: real function entries always carry a string name */ - if (typeof name !== 'string') continue - const strict = strictByName.get(name) - if (strict === undefined) delete tool.function.strict - else tool.function.strict = strict + delete tool.function.strict } const rawById = rawToolArguments(options) @@ -131,9 +124,9 @@ function patchPayload(payload: unknown, options: GenerateOptions, reasoning: PiA * * Implementation notes: * - `onPayload` patches provider payload details pi-ai cannot express directly: - * stop sequences, per-tool strict, omitted reasoning effort, and raw replayed - * tool-call arguments. - * - `prefill` throws UNSUPPORTED (same contract as dsh-llm-deepseek). + * stop sequences, scrubbing pi-ai's own per-tool `strict` default (the + * hand-rolled twin sends no such field), omitted reasoning effort, and raw + * replayed tool-call arguments. * - pi-ai reports request failures as in-stream error events; convert.ts * maps them to `finish {kind:'error'|'aborted'}` chunks rather than * throwing — both are sanctioned StreamChunk error paths. @@ -144,13 +137,6 @@ export class PiAiAdapter extends LlmAdapter { } async * stream(options: GenerateOptions): AsyncIterable { - if (options.prefill !== undefined) { - throw new LlmError( - 'prefill is not supported by the pi-ai adapter', - 'UNSUPPORTED', - ) - } - const model = buildModel(options.model, this.options) // Undefined config means "provider default" (DeepSeek: thinking ENABLED), // matching llm-deepseek's omission semantics. pi-ai derives the wire @@ -171,6 +157,9 @@ export class PiAiAdapter extends LlmAdapter { try { const events = piStream(model, toPiContext(options), { apiKey: this.options.apiKey, + // pi-ai merges caller headers last over its provider defaults, so the + // harness attribution always reaches the wire. + headers: attributionHeaders(), ...options.temperature !== undefined ? { temperature: options.temperature } : {}, ...options.maxTokens !== undefined ? { maxTokens: options.maxTokens } : {}, signal: controller.signal, diff --git a/packages/llm/llm-pi-ai/src/convert.ts b/packages/llm/llm-pi-ai/src/convert.ts index 0ddc41386a..6fa96a0597 100644 --- a/packages/llm/llm-pi-ai/src/convert.ts +++ b/packages/llm/llm-pi-ai/src/convert.ts @@ -93,7 +93,7 @@ export function toPiContext(options: GenerateOptions): PiContext { }) break default: - // image / plugin-added block types: not representable here. + // plugin-added block types: not representable here. break } } diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 63f9f90456..cefaa9f745 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -2,7 +2,7 @@ import { createServer } from 'node:http' import type { IncomingMessage, Server, ServerResponse } from 'node:http' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import LlmService, { CallId, LlmError } from '@deepseek-ai/dsh-llm' +import LlmService, { CallId, userAgent } from '@deepseek-ai/dsh-llm' import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' import { buildModel, PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai' import { assemble } from './assemble.ts' @@ -11,6 +11,8 @@ import { assemble } from './assemble.ts' interface MockServer { url: string requests: unknown[] + /** Header bags of received requests, in order (parallel to `requests`). */ + headers: IncomingMessage['headers'][] close(): Promise } @@ -22,11 +24,13 @@ afterEach(async () => { async function mockServer(script: { status?: number; events?: string[]; body?: string }[]): Promise { const requests: unknown[] = [] + const headers: IncomingMessage['headers'][] = [] const server = createServer((request: IncomingMessage, response: ServerResponse) => { let body = '' request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') }) request.on('end', () => { requests.push(JSON.parse(body)) + headers.push(request.headers) const behavior = script.shift() ?? { status: 500, body: 'script exhausted' } if (behavior.status !== undefined && behavior.status !== 200) { response.writeHead(behavior.status, { 'content-type': 'application/json' }) @@ -45,6 +49,7 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s return { url: `http://127.0.0.1:${address.port}`, requests, + headers, close: () => new Promise(resolve => server.close(() => { resolve() })), } } @@ -91,6 +96,14 @@ describe('PiAiAdapter against a mock server', () => { expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }]) expect(result.finish).toEqual({ kind: 'stop' }) expect(result.usage).toMatchObject({ inputTokens: 3, outputTokens: 1 }) + + // Attribution reaches the wire through pi-ai's headers hook: the exact + // shared User-Agent, and no provider-specific headers under the + // User-Agent-only contract. + expect(server.headers[0]?.['user-agent']).toBe(userAgent()) + expect(server.headers[0]).not.toHaveProperty('http-referer') + expect(server.headers[0]).not.toHaveProperty('x-openrouter-title') + expect(server.headers[0]).not.toHaveProperty('x-openrouter-categories') }) it('streams tool calls with re-stringified arguments', async () => { @@ -149,26 +162,26 @@ describe('PiAiAdapter against a mock server', () => { expect(server.requests[0]).toMatchObject({ stop: ['END'] }) }) - it('preserves per-tool strict exactly through onPayload', async () => { + it('scrubs pi-ai\'s own per-tool strict default through onPayload', async () => { const server = await mockServer([{ events: textEvents }]) const ctx = await harness(server.url) await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], tools: [ - { name: 'strict_true', description: 'true', parameters: {}, strict: true }, - { name: 'strict_false', description: 'false', parameters: {}, strict: false }, - { name: 'strict_omitted', description: 'omitted', parameters: {} }, + { name: 'alpha', description: 'a', parameters: {} }, + { name: 'beta', description: 'b', parameters: {} }, ], }) + // pi-ai stamps `strict` on every serialized tool function; the harness + // contract has none and the hand-rolled twin sends no such field, so the + // payload fixup must have deleted it from every tool. const request = server.requests[0] as { tools: { function: { name: string; strict?: boolean } }[] } - expect(request.tools.map(tool => [tool.function.name, tool.function.strict])).toEqual([ - ['strict_true', true], - ['strict_false', false], - ['strict_omitted', undefined], - ]) - expect('strict' in request.tools[2]!.function).toBe(false) + expect(request.tools.map(tool => tool.function.name)).toEqual(['alpha', 'beta']) + for (const tool of request.tools) { + expect('strict' in tool.function).toBe(false) + } }) it('preserves raw replayed tool-call arguments in the provider payload', async () => { @@ -209,15 +222,6 @@ describe('PiAiAdapter against a mock server', () => { expect(result.finish).toMatchObject({ kind: 'error', code }) }) - it('rejects prefill with UNSUPPORTED', async () => { - const ctx = await harness('http://127.0.0.1:1') - await expect(assemble(ctx,{ - model: 'deepseek-v4-flash', - messages: [], - prefill: [{ type: 'text', text: 'Sure' }], - })).rejects.toThrow(LlmError) - }) - it('registers/unregisters models on the llm service (HMR safety)', async () => { const ctx = new Context() await ctx.plugin(LlmService) diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index be42c9e9b4..078d2a4d3b 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' import { CallId } from '@deepseek-ai/dsh-llm' -import type { StreamChunk } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm' import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai' import { mapStopReason, mapUsage, toPiContext, toStreamChunks } from '@deepseek-ai/dsh-llm-pi-ai' @@ -171,13 +171,13 @@ describe('toPiContext', () => { expect(context.messages.map(message => message.role)).toEqual(['user', 'user', 'toolResult']) }) - it('skips image and unknown blocks in assistant content', () => { + it('skips plugin-added (unknown) blocks in assistant content', () => { const context = toPiContext({ model: 'm', messages: [{ role: 'assistant', content: [ - { type: 'image', url: 'data:,x' }, + { type: 'chart', data: 'x' } as unknown as ContentBlock, { type: 'text', text: 'visible' }, ], }], diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index 4227f5fdef..ab5fab49be 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -25,10 +25,14 @@ An adapter registry plus a single streaming call surface, interceptable via a wa ### Content-block vocabulary (`types.ts`) -Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`, `image`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. +Messages are arrays of typed content blocks: `text`, `reasoning`, `tool-call`, `tool-result`. The union is derived from the merge-extensible `ContentBlockMap`, so plugins can add block types via declaration merging. The core set is limited to blocks every shipping path honors — multimodal content (images, audio, …) has no core block type; a feature that needs one adds it via the map together with the adapter/UI/compaction support that honors it. Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. +### App attribution (`attribution.ts`) + +Every product adapter must identify the application on every provider HTTP request - attribution is part of the adapter contract, not an adapter-local nicety. `attributionHeaders(identity?)` builds the standard `User-Agent` header (`product/version (+url)`, from `userAgent()`) for every request. The default `APP_IDENTITY` carries only static public product facts (its version is read from this package's manifest); a white-label deployment passes its own `AppIdentity`, and omission falls back to the default - nothing can suppress attribution. OpenRouter-specific app attribution headers are intentionally not supported by this contract. An adapter proves compliance with a wire-level test: a mock server asserting the received header (or, for a library-backed adapter, that the library's header hook delivers the same value). Policy and rationale: [Mandatory `User-Agent` attribution](../../../docs/rfc/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md). + ### Classes - `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`. diff --git a/packages/llm/llm/src/attribution.ts b/packages/llm/llm/src/attribution.ts new file mode 100644 index 0000000000..61ee2f9ddd --- /dev/null +++ b/packages/llm/llm/src/attribution.ts @@ -0,0 +1,71 @@ +/** + * App-attribution vocabulary for provider requests. + * + * Every product LLM adapter must identify the application on every provider + * HTTP request (see the adapter contract on {@link ../index.ts LlmAdapter}): + * a static, non-secret product identity, sent as the standard `User-Agent`. + * Adapters obtain the headers from {@link attributionHeaders} instead of + * hand-copying constants, so the identity cannot drift between + * implementations. The policy and its rationale are pinned in + * docs/rfc/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md. + * + * @module @deepseek-ai/dsh-llm/attribution + */ + +import { createRequire } from 'node:module' + +// The package's own manifest is the single source of the version so the +// User-Agent cannot drift from what is published (`./package.json` is an +// export of this package; the relative path resolves from both `src/` and +// the bundled `lib/`). +const { version } = createRequire(import.meta.url)('../package.json') as { version: string } + +/** + * Static public application identity sent to LLM providers. + * + * Every field is a public product fact, safe on every request: no secrets, + * local paths, session ids, prompt text, or per-user identifiers belong here, + * and nothing per-request may influence the values. + */ +export interface AppIdentity { + /** `User-Agent` product token (lowercase, hyphenated). */ + product: string + /** Product version; sourced from package metadata, never hand-copied. */ + version: string + /** Public home URL of the app, used as the `User-Agent` comment. */ + url: string +} + +/** + * The harness's own identity: the default every adapter sends. Deployments + * that need a white-label identity pass their own {@link AppIdentity} to + * {@link attributionHeaders} — omission falls back to this default; nothing + * can suppress attribution entirely. + */ +export const APP_IDENTITY: AppIdentity = { + product: 'deepseek-harness', + version, + // FIXME: create the public deepseek-ai/deepseek-harness-sdk repository this + // URL promises before the first release ships attribution pointing at it. + url: 'https://github.com/deepseek-ai/deepseek-harness-sdk', +} + +/** + * The standard `User-Agent` value: `product/version (+url)`. The + * parenthesized `+url` comment is the conventional self-identification form + * (RFC 9110 §10.1.5 product + comment syntax). + */ +export function userAgent(identity: AppIdentity = APP_IDENTITY): string { + return `${identity.product}/${identity.version} (+${identity.url})` +} + +/** + * Build the attribution headers an adapter must send on every provider + * request. Header names are lowercase (HTTP field names are case-insensitive + * on the wire). + */ +export function attributionHeaders( + identity: AppIdentity = APP_IDENTITY, +): Record { + return { 'user-agent': userAgent(identity) } +} diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index 890a5a132e..220bb60062 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -10,6 +10,7 @@ import { Context, Service } from 'cordis' import type { GenerateOptions, StreamChunk } from './types.ts' import { HarnessError } from './error.ts' +export * from './attribution.ts' export * from './brand.ts' export * from './never.ts' export * from './error.ts' @@ -57,6 +58,13 @@ export class LlmError extends HarnessError { * fetch/SSE) and `@deepseek-ai/dsh-llm-pi-ai` (pi-ai-backed) — two * deliberately different internals over the same contract; see the * adapter contract documented on `StreamChunk` in `./types.ts`. + * + * App attribution is part of the adapter contract: every HTTP request to a + * provider carries the headers from `attributionHeaders()` (`./attribution.ts`) + * — the standard `User-Agent` baseline everywhere. An adapter proves it with + * a wire-level test (a mock server asserting the received header), or, for a + * library-backed adapter, by asserting the library's header hook delivers the + * same value to the wire. */ export abstract class LlmAdapter { /** Stream one model call as raw chunks. The only required method. */ diff --git a/packages/llm/llm/src/types.ts b/packages/llm/llm/src/types.ts index 7b1b9bdc47..7163ddc1d9 100644 --- a/packages/llm/llm/src/types.ts +++ b/packages/llm/llm/src/types.ts @@ -22,14 +22,10 @@ import type { Branded } from '@deepseek-ai/dsh-brand' import type { CallId } from './brand.ts' -/** Cache hint attached to a content block (provider-interpreted). */ -export type CacheHint = 'ephemeral' - /** Plain text visible to the end user. */ export interface TextBlock { type: 'text' text: string - cache?: CacheHint } /** Reasoning / thinking content, distinct from visible text. */ @@ -54,27 +50,24 @@ export interface ToolResultBlock { toolCallId: CallId content: ContentBlock[] isError?: boolean - cache?: CacheHint -} - -/** An image, by URL or data URL. */ -export interface ImageBlock { - type: 'image' - url: string - mimeType?: string - cache?: CacheHint } /** * All known content block shapes, keyed by their `type` tag. * Merge-extensible: plugins add new block types via declaration merging. + * + * The core set is deliberately limited to blocks every shipping path honors. + * Multimodal content (images, audio, …) has no core block type: a feature + * that needs one adds it via declaration merging in the same coordinated + * change that maps it in the adapters, surfaces it in the UI bridges, and + * prices it in compaction — a producer never lands without its consumers + * (see docs/rfc/implemented/simplification/2026-07-04-drop-image-content-block.md). */ export interface ContentBlockMap { 'text': TextBlock 'reasoning': ReasoningBlock 'tool-call': ToolCallBlock 'tool-result': ToolResultBlock - 'image': ImageBlock } export type ContentBlockType = keyof ContentBlockMap @@ -93,7 +86,6 @@ export interface Message { export interface MessageSourceMap { user: { kind: 'user' } plugin: { kind: 'plugin'; plugin: string } - agent: { kind: 'agent'; agentId: string } } export type MessageSource = MessageSourceMap[keyof MessageSourceMap] @@ -171,7 +163,6 @@ export interface ToolSchema { description: string /** JSON Schema object for the arguments. */ parameters: Record - strict?: boolean } /** A single model request, fully assembled. */ @@ -182,8 +173,6 @@ export interface GenerateOptions { system?: string /** Tool schemas (adapters map to the provider's `tools` field). */ tools?: ToolSchema[] - /** Assistant prefix continuation (prefill). */ - prefill?: ContentBlock[] temperature?: number maxTokens?: number /** diff --git a/packages/llm/llm/tests/assembler.spec.ts b/packages/llm/llm/tests/assembler.spec.ts index e8ad04e3b5..d9a4fe33f3 100644 --- a/packages/llm/llm/tests/assembler.spec.ts +++ b/packages/llm/llm/tests/assembler.spec.ts @@ -63,12 +63,12 @@ describe('BlockAssembler', () => { it('throws from assemble() when a partial has an unhandled blockType', () => { const assembler = new BlockAssembler() - // Directly push a block-end for an image block whose block-start never - // called ensure — but the image block-type flows through normally. - // What we really need is a partial whose blockType is not text/reasoning/tool-call. - // We can achieve this via a block-start for 'image' followed by blocks(). - assembler.push({ type: 'block-start', index: 0, blockType: 'image' } as unknown as StreamChunk) - expect(() => assembler.blocks()).toThrow('cannot assemble incomplete block of type "image"') + // A partial whose blockType is not text/reasoning/tool-call cannot be + // assembled without its block-end. A plugin-added block type (here + // 'video', via the merge-extensible ContentBlockMap) opened by a + // block-start with no closing block-end exercises that throw. + assembler.push({ type: 'block-start', index: 0, blockType: 'video' } as unknown as StreamChunk) + expect(() => assembler.blocks()).toThrow('cannot assemble incomplete block of type "video"') }) it('mustGet throws when an index is missing from the partials map (invariant violation)', () => { diff --git a/packages/llm/llm/tests/attribution.spec.ts b/packages/llm/llm/tests/attribution.spec.ts new file mode 100644 index 0000000000..e797af5b38 --- /dev/null +++ b/packages/llm/llm/tests/attribution.spec.ts @@ -0,0 +1,51 @@ +import { createRequire } from 'node:module' +import { describe, expect, it } from 'vitest' +import { APP_IDENTITY, attributionHeaders, userAgent } from '@deepseek-ai/dsh-llm' +import type { AppIdentity } from '@deepseek-ai/dsh-llm' + +const manifest = createRequire(import.meta.url)('../package.json') as { version: string } + +/** A white-label identity exercising every override seam. */ +const forkIdentity: AppIdentity = { + product: 'fork-agent', + version: '9.9.9', + url: 'https://example.com/fork-agent', +} + +describe('APP_IDENTITY', () => { + it('sources the version from the package manifest, never a hand-copied constant', () => { + expect(APP_IDENTITY.version).toBe(manifest.version) + }) + + it('carries only static public product facts', () => { + expect(APP_IDENTITY).toEqual({ + product: 'deepseek-harness', + version: manifest.version, + url: 'https://github.com/deepseek-ai/deepseek-harness-sdk', + }) + }) +}) + +describe('userAgent', () => { + it('renders product/version with the +url comment', () => { + expect(userAgent()).toBe( + `deepseek-harness/${manifest.version} (+https://github.com/deepseek-ai/deepseek-harness-sdk)`, + ) + }) + + it('renders a custom identity', () => { + expect(userAgent(forkIdentity)).toBe('fork-agent/9.9.9 (+https://example.com/fork-agent)') + }) +}) + +describe('attributionHeaders', () => { + it('defaults to the provider-neutral baseline: User-Agent and nothing else', () => { + expect(attributionHeaders()).toEqual({ 'user-agent': userAgent() }) + }) + + it('maps a custom identity onto the User-Agent header only', () => { + expect(attributionHeaders(forkIdentity)).toEqual({ + 'user-agent': 'fork-agent/9.9.9 (+https://example.com/fork-agent)', + }) + }) +}) diff --git a/packages/llm/llm/tests/properties.spec.ts b/packages/llm/llm/tests/properties.spec.ts index c63d56abbb..3bb2a76c38 100644 --- a/packages/llm/llm/tests/properties.spec.ts +++ b/packages/llm/llm/tests/properties.spec.ts @@ -81,7 +81,7 @@ describe('BlockAssembler properties', () => { fc.assert(fc.property(streamArb, (chunks) => { const blocks = feed(chunks).blocks() for (const block of blocks) { - expect(['text', 'reasoning', 'tool-call', 'tool-result', 'image']).toContain(block.type) + expect(['text', 'reasoning', 'tool-call', 'tool-result']).toContain(block.type) } })) }) diff --git a/packages/support/llm-replay/tests/llm-replay.spec.ts b/packages/support/llm-replay/tests/llm-replay.spec.ts index 2dd8357cc7..87cc62d165 100644 --- a/packages/support/llm-replay/tests/llm-replay.spec.ts +++ b/packages/support/llm-replay/tests/llm-replay.spec.ts @@ -118,7 +118,7 @@ describe('deriveReplayScript', () => { it('ignores non-assistant/chunk events', () => { let seq = 1 const events: SessionEvent[] = [ - { type: 'turn/start', seq: seq++, time: 0, data: { turn: 1, trigger: { kind: 'continuation' } } }, + { type: 'turn/start', seq: seq++, time: 0, data: { turn: 1, trigger: { kind: 'injection', source: { kind: 'user' } } } }, ...TEXT_CHUNKS.map(c => chunkEvent(seq++, 1, 1, c)), { type: 'turn/end', seq: seq++, time: 0, data: { turn: 1, reason: { kind: 'completed' } } }, ] diff --git a/packages/ui/README.md b/packages/ui/README.md index 1a3c45727d..8b0293cf9e 100644 --- a/packages/ui/README.md +++ b/packages/ui/README.md @@ -7,6 +7,7 @@ Integrations that expose the agent to an external editor or client. These are ** | `acp/` | Agent Client Protocol bridge: serves the agent to an ACP editor (Zed) over JSON-RPC stdio | (drives `ctx.agents`/`ctx.sessions`) | | `stdio-agent/` | Terminal stdio chat APP: the agent-core spine + console logger + readline UI + a pre-created `main` agent, with a `bin` | (composition + `bin`) | | `acp-agent/` | ACP server APP: the agent-core spine + JSONL persistence + the `acp` bridge (no stdout logger), with a `bin` | (composition + `bin`) | +| `app-boot/` | Shared boot glue for the two app bins: `.env` loading, fail-loud Loader guards, snapshot-aware config resolution, the settle-the-tree boot sequence | (library for the bins) | A UI integration is a client-driver plugin, not a loop change and not a capability seam: it consumes the existing `agent/*` event taxonomy and the `dsh-agent` factory. The readline UI is the unstructured analogue of the `acp` bridge and lives INSIDE the stdio app (the `stdio-chat` module of [`stdio-agent/`](stdio-agent/README.md)): it is scaffolding for that one front door, not an independently swappable integration, so it carries no package boundary of its own. diff --git a/packages/ui/acp-agent/package.json b/packages/ui/acp-agent/package.json index 72eb95b2f7..71f8fe0171 100644 --- a/packages/ui/acp-agent/package.json +++ b/packages/ui/acp-agent/package.json @@ -32,6 +32,7 @@ "peerDependencies": { "@cordisjs/plugin-include": "^1.0.4", "@cordisjs/plugin-loader": "^1.0.0-rc.4", + "@deepseek-ai/dsh-app-boot": "^0.0.1", "@deepseek-ai/dsh-acp": "^0.0.1", "@deepseek-ai/dsh-agent-core": "^0.0.1", "@deepseek-ai/dsh-session-persistence-jsonl": "^0.0.1", @@ -41,6 +42,7 @@ "devDependencies": { "@cordisjs/plugin-include": "workspace:^", "@cordisjs/plugin-loader": "workspace:^", + "@deepseek-ai/dsh-app-boot": "workspace:^", "@deepseek-ai/dsh-acp": "workspace:^", "@deepseek-ai/dsh-agent-core": "workspace:^", "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", diff --git a/packages/ui/acp-agent/src/bin.ts b/packages/ui/acp-agent/src/bin.ts index 24e31f3251..1cbdff64d1 100644 --- a/packages/ui/acp-agent/src/bin.ts +++ b/packages/ui/acp-agent/src/bin.ts @@ -2,167 +2,45 @@ /** * The `dsh-acp-agent` bin: boot the ACP server from a leaf `cordis.yml` that * loads the {@link @deepseek-ai/dsh-acp-agent} app plugin (plus an LLM adapter - * and a bash executor), speaking ACP JSON-RPC on stdio. + * and a bash executor), speaking ACP JSON-RPC on stdio. The shared boot glue — + * `.env` loading, the fail-loud Loader guards, snapshot-aware config + * resolution, the settle-the-tree boot sequence — lives in + * {@link @deepseek-ai/dsh-app-boot}; this bin owns only the ACP-specific + * lifecycle: * - * Owns the ACP-specific boot glue the example's `start.ts` once held: - * - `.env` loading (`DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL`) — SKIPPED in - * snapshot REPLAY so a stray key can never trigger a live model call. - * - snapshot-mode config selection: `DSH_SNAPSHOT=replay` swaps the given - * `cordis.yml` for its sibling `cordis.snapshot.yml` (the keyless replay - * tree: `llm-replay` in place of `llm-deepseek`). - * - the stdin-dispose lifecycle: in a snapshot run the harness closes stdin - * when done, so dispose the context (flushing persistence) and exit cleanly. + * - `.env` loading is SKIPPED in snapshot REPLAY so a stray key can never + * trigger a live model call. + * - `DSH_SNAPSHOT=replay` swaps the given `cordis.yml` for its sibling + * `cordis.snapshot.yml` (the keyless replay tree: `llm-replay` in place of + * `llm-deepseek`). + * - In a snapshot run the harness closes stdin when done, so dispose the + * context (flushing persistence) and exit cleanly. In a normal editor + * session stdin stays open for the connection's lifetime (the editor kills + * the process), so the EOF handler never fires. * * IMPORTANT: stdout is the ACP JSON-RPC channel. This bin writes diagnostics to - * STDERR only; the app plugin loads no stdout logger. A stray stdout write - * corrupts the protocol frames. + * STDERR only (the app plugin loads no stdout logger, and the shared guards + * write to stderr); a stray stdout write corrupts the protocol frames. * * Usage: `dsh-acp-agent [path-to-cordis.yml]` (default `./cordis.yml`). * * @module @deepseek-ai/dsh-acp-agent/bin */ -import { pathToFileURL } from 'node:url' -import { basename, dirname, resolve } from 'node:path' -import { Context } from 'cordis' -import Loader from '@cordisjs/plugin-loader' +import { boot, installFailLoud, loadEnv, resolveConfigPath } from '@deepseek-ai/dsh-app-boot' -/** - * Resolve the config to boot, honoring snapshot REPLAY. Given the requested - * path, replay mode swaps a `cordis.yml` basename for `cordis.snapshot.yml` in - * the SAME directory (the keyless replay tree). Other modes use the path as-is. - * Returns an absolute path resolved from the cwd. - */ -export function resolveConfigPath(configPath: string, snapshotMode: string | undefined): string { - const absolute = resolve(process.cwd(), configPath) - if (snapshotMode !== 'replay') return absolute - const dir = dirname(absolute) - const replayName = basename(absolute).replace(/cordis\.ya?ml$/, 'cordis.snapshot.yml') - return resolve(dir, replayName) -} +const NAME = 'dsh-acp-agent' -/** - * Load `DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL` from a gitignored `.env` in the - * cwd (Node native). Diagnostics go to STDERR (stdout is the protocol). In - * REPLAY mode the caller skips this entirely — replay must never reach the - * network, so a present `.env` must not enable a live call. - */ -function loadEnv(): void { - try { - process.loadEnvFile(resolve(process.cwd(), '.env')) - } catch (error) { - if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { - process.stderr.write(`dsh-acp-agent: failed to load .env: ${String(error)}\n`) - } - // ENOENT (no .env) is fine — rely on the ambient environment. - } -} - -/** - * Make a load failure fail loud with a clear message on stderr. Covers the - * failure path the entry-tree check below cannot: when the include's - * `[Service.init]` throws (e.g. a config FILE missing in a real directory), the - * cordis Loader surfaces it as an unhandled promise rejection AFTER `boot()` - * resolves — `loader.await()` does NOT rethrow it (`EntryTree.await()` uses - * `Promise.allSettled`, which swallows rejections). Node's default handler - * already exits non-zero on an unhandled rejection, so this does not change the - * exit code; it replaces the noisy stack dump with a single labelled line (on - * STDERR — stdout is the ACP JSON-RPC channel) and guarantees `process.exit(1)`. - * Install before `boot()`. - */ -export function installFailLoud(): void { - process.on('unhandledRejection', (err: unknown) => { - process.stderr.write(`dsh-acp-agent: fatal load failure: ${err instanceof Error ? err.stack ?? err.message : String(err)}\n`) - process.exit(1) +/* v8 ignore start -- thin self-executing composition over the unit-tested + dsh-app-boot helpers; exercised end-to-end by the snapshot suite and the + built-bin smoke */ +installFailLoud(NAME) +const snapshotMode = process.env['DSH_SNAPSHOT'] +if (snapshotMode !== 'replay') loadEnv(NAME) +const ctx = await boot(NAME, resolveConfigPath(process.argv[2] ?? './cordis.yml', snapshotMode)) +if (snapshotMode !== undefined) { + process.stdin.on('end', () => { + void ctx.fiber.dispose().then(() => { process.exit(0) }) }) } - -/** - * After the tree settles, assert every loader entry actually started. This is - * the load-bearing guard against the SILENT-exit-0 bug: a plugin module that - * fails to IMPORT (e.g. a config path in a non-existent directory) is caught and - * only LOGGED by the cordis Loader (`entry._init`), leaving the entry with no - * `fiber` and producing no rejection — so the process would otherwise exit 0. A - * started entry has a `fiber`; throw on any entry still missing one so `boot()` - * rejects. - * - * A `disabled` entry is the one legitimate fiber-less state: `Entry.refresh()` - * deliberately skips `init()` for it, so it settles without a fiber by design — - * a valid "plugin turned off" config, not a failed import. Exclude it. - */ -function assertEntriesLoaded(ctx: Context): void { - const failed = [...ctx.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled) - if (failed.length > 0) { - const names = failed.map(entry => entry.options.name).join(', ') - throw new Error(`dsh-acp-agent: plugin(s) failed to load: ${names} (see the error(s) logged above)`) - } -} - -/** - * Boot the Loader against `absoluteConfigPath`. The include is handed the - * config's ABSOLUTE `file://` URL as its `path`, so resolution never depends on - * `ctx.baseUrl` (an absolute URL ignores the base) and can never fall back to - * the cwd. `baseUrl` is still pinned to the config's directory so the config's - * OWN relative plugin/include paths resolve against it. Returns the root context - * once the whole tree has settled. - * - * The `await ctx.loader.await()` is load-bearing: `loader.create()` returns once - * the include ENTRY is registered, but the include then loads its child plugins - * asynchronously. Without awaiting the tree, `boot()` would resolve while the ACP - * bridge is still mounting — the process would have no stdin handle attached yet - * and could exit 0 silently. Awaiting keeps the process alive until the bridge - * is up. - * - * `loader.await()` does NOT rethrow load errors (`EntryTree.await()` uses - * `Promise.allSettled`), so failures are surfaced two ways: a plugin that fails - * to IMPORT leaves an entry with no fiber, caught here by - * {@link assertEntriesLoaded} (this `boot()` rejects); a plugin whose init THROWS - * surfaces as an unhandled rejection caught by {@link installFailLoud} (installed - * by `main()` before this runs). Together any load failure exits non-zero. - * - * Bare plugin specifiers in the config (`@deepseek-ai/dsh-*`, npm packages) are - * resolved by the cordis Loader's internal module loader, which is only active - * under `node --expose-internals`. The `demo:acp` script runs under tsx (whose - * tsconfig `paths` map resolves the workspace plugins instead), but a consumer - * running the built bin under plain node must pass `--expose-internals` so the - * Loader resolves the config's plugins from the config directory rather than - * relative to its own module. - */ -export async function boot(absoluteConfigPath: string): Promise { - const ctx = new Context() - ctx.baseUrl = pathToFileURL(dirname(absoluteConfigPath)).href + '/' - await ctx.plugin(Loader) - await ctx.loader.create({ - name: '@cordisjs/plugin-include', - config: { path: pathToFileURL(absoluteConfigPath).href }, - }) - await ctx.loader.await() - assertEntriesLoaded(ctx) - return ctx -} - -/** - * Entry point. Installs the fail-loud guard, selects the config (snapshot-aware), - * loads `.env` outside replay, boots, and — in a snapshot run — disposes the - * context on stdin EOF so the session log is fully flushed before exit and the - * harness's `waitForExit` resolves. In a normal editor session stdin stays open - * for the connection's lifetime (the editor kills the process), so the EOF - * handler never fires. - */ -export async function main(argv: string[] = process.argv.slice(2)): Promise { - installFailLoud() - const snapshotMode = process.env.DSH_SNAPSHOT - const configPath = resolveConfigPath(argv[0] ?? './cordis.yml', snapshotMode) - if (snapshotMode !== 'replay') loadEnv() - const ctx = await boot(configPath) - if (snapshotMode !== undefined) { - process.stdin.on('end', () => { - void ctx.fiber.dispose().then(() => { process.exit(0) }) - }) - } -} - -/* v8 ignore start -- top-level CLI invocation; the testable core is - resolveConfigPath()/boot()/main(), driven by the keyless snapshot + Loader-path tests */ -await main() /* v8 ignore stop */ diff --git a/packages/ui/acp-agent/tests/built-bin.e2e.ts b/packages/ui/acp-agent/tests/built-bin.e2e.ts index 51c5d53c0b..d900c14152 100644 --- a/packages/ui/acp-agent/tests/built-bin.e2e.ts +++ b/packages/ui/acp-agent/tests/built-bin.e2e.ts @@ -40,7 +40,7 @@ const acpBin = join(repoRoot, 'packages/ui/acp-agent/lib/bin.js') const dshPackages = [ 'core/agent-core', 'core/agent', 'core/session', 'core/system-prompt', 'core/tools', 'core/agent-loop', 'llm/llm', 'llm/llm-deepseek', 'bash/bash', - 'bash/bash-local', 'bash/tool-bash', 'support/invariants', + 'bash/bash-local', 'bash/tool-bash', 'support/invariants', 'ui/app-boot', 'session-persistence/session-persistence', 'session-persistence/session-persistence-jsonl', 'ui/acp', 'ui/acp-agent', ] diff --git a/packages/ui/acp-agent/tsconfig.json b/packages/ui/acp-agent/tsconfig.json index ffea8ec6f6..6cc211087c 100644 --- a/packages/ui/acp-agent/tsconfig.json +++ b/packages/ui/acp-agent/tsconfig.json @@ -17,6 +17,9 @@ { "path": "../../../vendor/loader" }, + { + "path": "../app-boot" + }, { "path": "../acp" }, diff --git a/packages/ui/acp/README.md b/packages/ui/acp/README.md index a311164c6b..38596b7fd0 100644 --- a/packages/ui/acp/README.md +++ b/packages/ui/acp/README.md @@ -16,8 +16,8 @@ It is a **client-driver / UI plugin**, the structured analogue of the readline ` |---|---|---| | `model` | — | Model name for created agents (must have a registered adapter). | | `systemPrompt` | — | Per-agent system prompt. | -| `agentName` | `deepseek-harness-acp` | Server name reported in `initialize`. | -| `agentVersion` | `0.0.1` | Server version reported in `initialize`. | + +The `initialize` handshake reports a fixed server identity (`agentInfo: { name: 'deepseek-harness-acp', version: '0.0.1' }`) — branding is a literal at the `initialize` site, not config. ## ACP method mapping @@ -48,7 +48,7 @@ How a tool call renders in the editor is owned by the TOOL, not the bridge — t - `{ card: 'terminal', title, description?, cwd? }` — a shell command → a terminal card (see Terminal card). - `{ card: 'diff', title, diffs, locations? }` — a file create/modify → an inline diff card; `diffs` is `FileDiff[]` (`{ path, oldText, newText }`, `oldText: null` ⇒ new file). The bridge emits each diff as an ACP `{ type: 'diff', path, oldText, newText }` `tool_call.content` block, which Zed renders as an inline diff / new-file preview. -`presentResult` returns a `ToolResultView`, one of three cards: `{ card: 'generic', title?, content? }` (an optional replacement `title` and reformatted `content`), `{ card: 'terminal', title?, output?, exitCode?, signal? }` (the captured run output + exit — see Terminal card), or `{ card: 'diff', title?, diffs }` (a completed file mutation → typically the applied hunks with context lines computed from the before/after content, or a whole-file diff for a create; a successful mutation ALWAYS returns this so the model-facing result text can't clobber the diff — an ACP `tool_call_update.content` replaces the call's content). The bridge looks the definition up by name in `ctx.tools` and `switch (view.card)`es to build the ACP `tool_call`/`tool_call_update` wire shape per card; a tool that declares neither gets a generic fallback (title = tool name, raw parsed args as `rawInput`, kind inferred from the name). For example `dsh-tool-bash` returns a `terminal` card for a foreground `bash` (title = the exact `command` "ls -la src", `description` = the model description); the `dsh-tool-fs` `write`/`edit` tools return a `diff` call card and a `diff` result card, and `read` returns a `generic` card (`kind: 'read'`, the read window in its title — `Read foo.txt (5 - 8)` — and a `locations` entry for the file). For a file card the bridge **relativizes the title** against the session cwd (mirroring `claude-agent-acp`'s `toDisplayPath` — `Read src/foo.ts`, not the absolute path) while keeping `locations[]`/`diffs[].path` **raw** so the editor opens the real path. +`presentResult` returns a `ToolResultView`, one of three cards: `{ card: 'generic', title?, content? }` (an optional replacement `title` and reformatted `content`), `{ card: 'terminal', title?, output?, exitCode?, signal? }` (the captured run output + exit — see Terminal card), or `{ card: 'diff', title?, diffs }` (a completed file mutation → typically the applied hunks with context lines computed from the before/after content, or a whole-file diff for a create; a successful mutation ALWAYS returns this so the model-facing result text can't clobber the diff — an ACP `tool_call_update.content` replaces the call's content). The bridge looks the definition up by name in `ctx.tools` and `switch (view.card)`es to build the ACP `tool_call`/`tool_call_update` wire shape per card; a tool that declares neither gets a generic fallback (title = tool name, raw parsed args as `rawInput`, kind `other` — the bridge never sniffs a kind from the tool name). For example `dsh-tool-bash` returns a `terminal` card for a foreground `bash` (title = the exact `command` "ls -la src", `description` = the model description); the `dsh-tool-fs` `write`/`edit` tools return a `diff` call card and a `diff` result card, and `read` returns a `generic` card (`kind: 'read'`, the read window in its title — `Read foo.txt (5 - 8)` — and a `locations` entry for the file). For a file card the bridge **relativizes the title** against the session cwd (mirroring `claude-agent-acp`'s `toDisplayPath` — `Read src/foo.ts`, not the absolute path) while keeping `locations[]`/`diffs[].path` **raw** so the editor opens the real path. The `tool/result` session event does not carry the tool name or args — so to call a tool's `presentResult` the bridge keeps a small per-session map from `callId` to the in-flight call's `(name, args)`, populated on `tool/call` and removed as each result is presented (it holds only currently-in-flight calls, never finished ones). This is bridge-local state — NOT a change to the event schema or a core service. The map lives on the `SessionRecord`, so two concurrent sessions never cross their in-flight tool state; a `session/load` replay uses a throwaway presenter that pairs each `tool/call` with its `tool/result` as the log replays in order, so replayed tool cards render identically to live ones. diff --git a/packages/ui/acp/acp-feature-support.md b/packages/ui/acp/acp-feature-support.md index 4a051bf5aa..d37c9b1eaf 100644 --- a/packages/ui/acp/acp-feature-support.md +++ b/packages/ui/acp/acp-feature-support.md @@ -63,7 +63,7 @@ These are capabilities the bridge would *drive* on the editor. The harness runs | `sessionCapabilities.*` | S | ❌ | ✅ | ✅ | None advertised (list/delete/resume/close/additionalDirectories/fork all off). | | `auth.logout` | S | ❌ | ✅ | ✅ | Not advertised. | | `authMethods[]` | S | ⚠️ | ✅ | ✅ | Advertised as empty (no auth required to reach the model). | -| `agentInfo` (name/version) | S | ✅ | ✅ | ✅ | From `agentName` / `agentVersion` config. | +| `agentInfo` (name/version) | S | ✅ | ✅ | ✅ | Fixed literals: `deepseek-harness-acp` / `0.0.1` (not config). | | `_meta` custom caps | S | ❌ | ✅ | — | E.g. Claude's `claudeCode.promptQueueing`. The bridge advertises no custom `_meta`. | ### 3b. `clientCapabilities` (consumed by the bridge) @@ -96,7 +96,7 @@ Tool-call presentation is **owned by each tool** (`presentCall` / `presentResult | Feature | Stable | Bridge | Claude | Codex | Notes | |---|---|---|---|---|---| -| `ToolCallKind` mapping | S | ✅ | ✅ | ✅ | `execute`/`read`/`edit`/`other` inferred from the tool; richer mapping possible. | +| `ToolCallKind` mapping | S | ✅ | ✅ | ✅ | `execute`/`read`/`edit` declared by each tool's `presentCall`; presenter-less tools render `other` (no name sniffing); richer mapping possible. | | `ToolCallStatus` | S | ✅ | ✅ | ✅ | `in_progress` → `completed`/`failed`. | | `content` blocks | S | ✅ | ✅ | ✅ | Text content; the description renders above the card. | | `diff` content | S | ✅ | ✅ | ✅ | The `write`/`edit` tools declare a `diff` render intent: `presentCall` → a call-time `{ card: 'diff' }` snippet, and `presentResult` → a result-time `{ card: 'diff' }`. For an edit or an overwrite it carries the applied hunk(s) with surrounding context (one per `replace_all` site), computed from the before/after text and persisted on the `tool/result` event as `meta`; for a create (no before-image) it is an args-derived whole-file diff. The bridge emits `{ type: 'diff', path, oldText, newText }` content blocks; a successful mutation ALWAYS returns the result diff (an ACP `tool_call_update.content` replaces the call's content, so the result diff — not the model-facing text — is what survives). | diff --git a/packages/ui/acp/src/codec.ts b/packages/ui/acp/src/codec.ts index 3b03a81c83..444e71545c 100644 --- a/packages/ui/acp/src/codec.ts +++ b/packages/ui/acp/src/codec.ts @@ -69,8 +69,8 @@ export function turnEndToStopReason(reason: TurnEndReason): StopReason { * client as message content. Today only `text` maps; `resource_link` is an * ACP prompt-only input rendered into text by {@link acpPromptToText}; * `reasoning` is surfaced via `agent_thought_chunk` - * streaming rather than as a message block, and `tool-call`/`tool-result`/ - * `image` are handled by the tool-call update path or not advertised. + * streaming rather than as a message block, and `tool-call`/`tool-result` + * are handled by the tool-call update path. */ export function harnessBlockToAcpContent(block: ContentBlock): AcpContentBlock | undefined { switch (block.type) { @@ -78,7 +78,7 @@ export function harnessBlockToAcpContent(block: ContentBlock): AcpContentBlock | return { type: 'text', text: block.text } // reasoning → streamed as agent_thought_chunk, not a message block // tool-call / tool-result → the tool_call / tool_call_update path - // image → not advertised + // plugin-added block types → not surfaced default: return undefined } diff --git a/packages/ui/acp/src/index.ts b/packages/ui/acp/src/index.ts index 70aa7a9493..717e57a1f5 100644 --- a/packages/ui/acp/src/index.ts +++ b/packages/ui/acp/src/index.ts @@ -67,7 +67,7 @@ import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent' import { AgentId } from '@deepseek-ai/dsh-agent' import { SessionId } from '@deepseek-ai/dsh-session' import type { SessionEvent, TodoItem, TurnEndReason } from '@deepseek-ai/dsh-session' -import type { ToolCallKind, ToolCallView, ToolRegistry, ToolResultView, TerminalResultView } from '@deepseek-ai/dsh-tools' +import type { ToolCallView, ToolRegistry, ToolResultView, TerminalResultView } from '@deepseek-ai/dsh-tools' // Side-effect type import: declaration-merges `ctx.sessionPersistence` onto // Context (the bridge injects it and reads `list()` for load cwd validation). import type {} from '@deepseek-ai/dsh-session-persistence' @@ -117,10 +117,6 @@ export interface AcpConfig { model?: string /** Per-agent system prompt. */ systemPrompt?: string - /** Agent/server name reported to the client in `initialize`. */ - agentName?: string - /** Agent/server version reported to the client in `initialize`. */ - agentVersion?: string /** * Transport stream override. Production omits this (the plugin wires * `process.stdin`/`process.stdout` via `ndJsonStream`). Tests inject an @@ -134,8 +130,6 @@ export interface AcpConfig { export const Config: Schema = Schema.object({ model: Schema.string(), systemPrompt: Schema.string(), - agentName: Schema.string().default('deepseek-harness-acp'), - agentVersion: Schema.string().default('0.0.1'), }) /** @@ -209,13 +203,6 @@ interface SessionRecord { * (settle-exactly-once). */ export function apply(ctx: Context, config: AcpConfig): void { - // TODO(double-default): these literals duplicate the Config schema defaults - // (`agentName`/`agentVersion` `.default(...)` above). The Loader applies the - // schema before apply() runs, so the `??` only fires for direct-apply unit - // tests. Pick one home for the default to avoid drift. - const agentName = config.agentName ?? 'deepseek-harness-acp' - const agentVersion = config.agentVersion ?? '0.0.1' - // Capture the injected services NOW, during apply(), while we are inside this // plugin's fiber (where `inject` grants access). The ACP method handlers run // LATER, from the AgentSideConnection's JSON-RPC read loop — a context that is @@ -430,7 +417,9 @@ export function apply(ctx: Context, config: AcpConfig): void { terminalOutputCap = params.clientCapabilities?._meta?.['terminal_output'] === true return Promise.resolve({ protocolVersion, - agentInfo: { name: agentName, version: agentVersion }, + // Fixed server identity: this bridge IS the harness ACP server, so the + // branding is a literal, not config (no shipped surface sets it). + agentInfo: { name: 'deepseek-harness-acp', version: '0.0.1' }, agentCapabilities: { loadSession: true, // Baseline prompt blocks only: text plus resource_link rendered as @@ -911,9 +900,11 @@ export class ToolPresenter { this.onError(`acp: tool "${name}" presentCall threw, using generic presentation: ${String(error)}`) present = undefined } - // No tool-owned presentation: fall back to the tool name as the title and the - // full parsed args as the raw input (the generic card). - const view: ToolCallView = present ?? { card: 'generic', title: name, kind: toolKindFor(name), rawInput: args } + // No tool-owned presentation: fall back to the tool name as the title, the + // full parsed args as the raw input, and kind `other` (the generic card). + // The kind is never sniffed from the name — the bridge does not special-case + // tool names; a tool that wants a richer kind declares `presentCall`. + const view: ToolCallView = present ?? { card: 'generic', title: name, kind: 'other', rawInput: args } this.pending.set(callId, { name, args, card: view.card }) return view } @@ -951,18 +942,10 @@ export class ToolPresenter { * results pass their raw content through unchanged. */ export const nullToolPresenter: Pick = { - call: (_callId, name, argsJson) => ({ card: 'generic', title: name, kind: toolKindFor(name), rawInput: parseToolArguments(argsJson) }), + call: (_callId, name, argsJson) => ({ card: 'generic', title: name, kind: 'other', rawInput: parseToolArguments(argsJson) }), result: (_callId, content) => ({ card: 'generic', content }), } -/** Map a harness tool name to an ACP ToolKind (best-effort; default `other`). */ -function toolKindFor(name: string): ToolCallKind { - if (name === 'bash' || name === 'bash_output' || name === 'bash_kill') return 'execute' - if (name === 'read' || name.startsWith('read')) return 'read' - if (name === 'write' || name === 'edit' || name.startsWith('edit')) return 'edit' - return 'other' -} - /** Parse a tool-call arguments JSON string for `rawInput`; raw string on failure. */ function parseToolArguments(args: string): unknown { try { diff --git a/packages/ui/acp/tests/bridge.spec.ts b/packages/ui/acp/tests/bridge.spec.ts index f94106f46c..6f8edd341f 100644 --- a/packages/ui/acp/tests/bridge.spec.ts +++ b/packages/ui/acp/tests/bridge.spec.ts @@ -33,7 +33,7 @@ describe('acp bridge', () => { expect(res.protocolVersion).toBe(PROTOCOL_VERSION) expect(res.agentCapabilities?.loadSession).toBe(true) expect(res.agentCapabilities?.promptCapabilities).toMatchObject({ image: false, audio: false }) - expect(res.agentInfo?.name).toBe('deepseek-harness-acp') + expect(res.agentInfo).toEqual({ name: 'deepseek-harness-acp', version: '0.0.1' }) }) it('session/new creates a session and a full prompt turn streams text then settles end_turn', async () => { @@ -148,14 +148,13 @@ describe('acp bridge', () => { await expect(harness.client.authenticate({ methodId: 'whatever' })).resolves.toBeDefined() }) - it('honors agentName/agentVersion/systemPrompt config', async () => { + it('honors systemPrompt config', async () => { harness = await makeBridgeHarness({ storageDir, script: [textResponse('ok')], - config: { agentName: 'custom-agent', agentVersion: '9.9.9', systemPrompt: 'be terse' }, + config: { systemPrompt: 'be terse' }, }) - const res = await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) - expect(res.agentInfo).toMatchObject({ name: 'custom-agent', version: '9.9.9' }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) // Create + prompt so the systemPrompt config flows through agentOptions and // reaches the model request. const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) diff --git a/packages/ui/acp/tests/codec.spec.ts b/packages/ui/acp/tests/codec.spec.ts index 859e8d40cd..b4f0c10792 100644 --- a/packages/ui/acp/tests/codec.spec.ts +++ b/packages/ui/acp/tests/codec.spec.ts @@ -1,4 +1,5 @@ import { describe, expect, it } from 'vitest' +import type { ContentBlock } from '@deepseek-ai/dsh-llm' import type { TurnEndReason } from '@deepseek-ai/dsh-session' import type { ContentBlock as AcpContentBlock } from '@agentclientprotocol/sdk' import { @@ -33,9 +34,9 @@ describe('harnessBlockToAcpContent', () => { expect(harnessBlockToAcpContent({ type: 'text', text: 'hi' })).toEqual({ type: 'text', text: 'hi' }) }) - it('returns undefined for non-text blocks (reasoning/tool/image)', () => { + it('returns undefined for non-text blocks (reasoning / plugin-added)', () => { expect(harnessBlockToAcpContent({ type: 'reasoning', text: 'think' })).toBeUndefined() - expect(harnessBlockToAcpContent({ type: 'image', url: 'https://x/y.png', mimeType: 'image/png' })).toBeUndefined() + expect(harnessBlockToAcpContent({ type: 'chart', data: 'x' } as unknown as ContentBlock)).toBeUndefined() }) }) diff --git a/packages/ui/acp/tests/stream-update.spec.ts b/packages/ui/acp/tests/stream-update.spec.ts index 2b4fcbd74f..311a853acd 100644 --- a/packages/ui/acp/tests/stream-update.spec.ts +++ b/packages/ui/acp/tests/stream-update.spec.ts @@ -50,32 +50,34 @@ describe('streamSessionEventUpdate', () => { .toEqual([]) }) - it('maps tool/call to an in_progress tool_call with inferred kind and parsed rawInput (generic fallback, no presenter)', () => { + it('maps tool/call to an in_progress tool_call with kind other and parsed rawInput (generic fallback, no presenter)', () => { const updates = updatesFor(evt('tool/call', { turn: 1, step: 1, callId: CallId('c1'), name: 'bash', arguments: '{"command":"ls"}' })) expect(updates).toEqual([{ sessionUpdate: 'tool_call', toolCallId: 'c1', title: 'bash', - kind: 'execute', + // The fallback never sniffs a kind from the tool name — even a name a + // first-party tool uses (`bash`) renders `other`; kinds are tool-owned + // via presentCall. + kind: 'other', status: 'in_progress', rawInput: { command: 'ls' }, }]) }) - it('infers tool kinds: read*/write*/edit*/other', () => { - const kind = (name: string): unknown => - updatesFor(evt('tool/call', { turn: 1, step: 1, callId: CallId('c'), name, arguments: '' }))[0] - expect((kind('read_file') as { kind: string }).kind).toBe('read') - expect((kind('write') as { kind: string }).kind).toBe('edit') - expect((kind('edit_file') as { kind: string }).kind).toBe('edit') - expect((kind('frobnicate') as { kind: string }).kind).toBe('other') - }) - it('falls back to the raw argument string when tool arguments are not JSON', () => { const update = updatesFor(evt('tool/call', { turn: 1, step: 1, callId: CallId('c1'), name: 'bash', arguments: 'not json' }))[0] expect((update as { rawInput: unknown }).rawInput).toBe('not json') }) + it('parses EMPTY tool arguments to an empty-object rawInput (a zero-arg call, not the raw-string fallback)', () => { + // `JSON.parse('')` throws, so without the empty-string guard a zero-arg + // call would render `rawInput: ''` via the non-JSON fallback; the guard + // normalizes it to `{}`. + const update = updatesFor(evt('tool/call', { turn: 1, step: 1, callId: CallId('c1'), name: 'noop', arguments: '' }))[0] + expect((update as { rawInput: unknown }).rawInput).toEqual({}) + }) + it('maps tool/result to completed/failed tool_call_update with text content', () => { const ok = updatesFor(evt('tool/result', { turn: 1, step: 1, callId: CallId('c1'), content: [{ type: 'text', text: 'out' }], isError: false })) expect(ok).toEqual([{ @@ -91,7 +93,7 @@ describe('streamSessionEventUpdate', () => { it('drops non-text tool-result content (text-only)', () => { const update = updatesFor(evt('tool/result', { turn: 1, step: 1, callId: CallId('c1'), - content: [{ type: 'image', url: 'https://x/y.png' }], + content: [{ type: 'reasoning', text: 'private' }], isError: false, }))[0] expect((update as { content: unknown[] }).content).toEqual([]) diff --git a/packages/ui/acp/tests/turns.spec.ts b/packages/ui/acp/tests/turns.spec.ts index 494bbb8741..e182d5f2bf 100644 --- a/packages/ui/acp/tests/turns.spec.ts +++ b/packages/ui/acp/tests/turns.spec.ts @@ -66,7 +66,10 @@ describe('acp bridge — turn outcomes', () => { const toolCalls = harness.updates.filter(u => u.sessionUpdate === 'tool_call') const toolUpdates = harness.updates.filter(u => u.sessionUpdate === 'tool_call_update') expect(toolCalls).toHaveLength(1) - expect(toolCalls[0]).toMatchObject({ toolCallId: 'c1', title: 'bash', kind: 'execute', status: 'in_progress' }) + // The inline stand-in declares no presentCall, so the generic fallback + // renders kind `other` (kinds are tool-owned; the bridge never sniffs the + // name — the REAL dsh-tool-bash test below covers the execute card). + expect(toolCalls[0]).toMatchObject({ toolCallId: 'c1', title: 'bash', kind: 'other', status: 'in_progress' }) expect(toolUpdates).toHaveLength(1) expect(toolUpdates[0]).toMatchObject({ toolCallId: 'c1', status: 'completed' }) diff --git a/packages/ui/app-boot/README.md b/packages/ui/app-boot/README.md new file mode 100644 index 0000000000..92fbc333be --- /dev/null +++ b/packages/ui/app-boot/README.md @@ -0,0 +1,15 @@ +# `@deepseek-ai/dsh-app-boot` + +Shared boot glue for the app bins ([`dsh-stdio-agent`](../stdio-agent/README.md), [`dsh-acp-agent`](../acp-agent/README.md)): each bin is a thin self-executing composition over these helpers, parameterized by its diagnostic prefix, so the loader-failure lore lives once — under the per-file coverage gate — instead of drifting between two published artifacts. + +| Export | Role | +|---|---| +| `resolveConfigPath(path, snapshotMode, cwd?)` | Absolute config path; `snapshotMode === 'replay'` swaps a `cordis.yml`/`.yaml` basename for its sibling `cordis.snapshot.yml` | +| `loadEnv(binName, dir?, warn?)` | Load the gitignored `.env` (Node `process.loadEnvFile`); absent file is fine, an unloadable one warns a single labelled line (default: stderr) | +| `installFailLoud(binName, proc?)` | Turn a post-`boot()` unhandled Loader rejection into one labelled stderr line + `exit(1)`; returns the uninstaller (for tests) | +| `assertEntriesLoaded(ctx, binName)` | Throw when a settled tree holds an enabled entry with no fiber (a plugin module that failed to import) | +| `boot(binName, absoluteConfigPath)` | Mount the Loader, include the config by absolute `file://` URL, await the whole tree, assert entries loaded, return the root context | + +Two failure classes the guards handle: `loader.await()` swallows init rejections (`Promise.allSettled`) — Node still exits non-zero on the resulting unhandled rejection, and `installFailLoud` replaces the noisy dump with one labelled line and a guaranteed `exit(1)`; a failed plugin IMPORT is only logged by the Loader (the process would otherwise exit 0 on a usable config typo), leaving a fiber-less entry that `assertEntriesLoaded` turns into a `boot()` rejection. + +Bare plugin specifiers in a config (`@deepseek-ai/dsh-*`) resolve through the cordis Loader's internal module loader, active only under `node --expose-internals`; the bins' subprocess smokes exercise that path, while this package's unit suite drives `boot()` in-process against configs with relative specifiers. diff --git a/packages/ui/app-boot/package.json b/packages/ui/app-boot/package.json new file mode 100644 index 0000000000..67b8b00dbc --- /dev/null +++ b/packages/ui/app-boot/package.json @@ -0,0 +1,34 @@ +{ + "name": "@deepseek-ai/dsh-app-boot", + "description": "Shared boot glue for the app bins: .env loading, fail-loud Loader guards, snapshot-aware config resolution, and the Loader boot sequence", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@cordisjs/plugin-include": "^1.0.4", + "@cordisjs/plugin-loader": "^1.0.0-rc.4", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@cordisjs/plugin-include": "workspace:^", + "@cordisjs/plugin-loader": "workspace:^", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/ui/app-boot/src/index.ts b/packages/ui/app-boot/src/index.ts new file mode 100644 index 0000000000..5515501188 --- /dev/null +++ b/packages/ui/app-boot/src/index.ts @@ -0,0 +1,154 @@ +/** + * Shared boot glue for the app bins (`dsh-stdio-agent`, `dsh-acp-agent`): load + * the gitignored `.env`, install the fail-loud Loader guards, resolve the + * config path (snapshot-aware), and drive the cordis Loader against a leaf + * `cordis.yml` until the whole tree has settled. Each bin stays a thin + * self-executing composition over these helpers, parameterized by its + * diagnostic prefix; the loader-failure lore lives here, once, under the + * per-file coverage gate. + * + * Two failure classes the guards handle: + * + * - `loader.await()` does NOT rethrow a load error (`EntryTree.await()` uses + * `Promise.allSettled`, which swallows rejections). A plugin whose + * `[Service.init]` throws surfaces as an unhandled rejection AFTER `boot()` + * resolves — Node's default handler already exits non-zero, and + * {@link installFailLoud} replaces the noisy dump with one labelled stderr + * line and a guaranteed `exit(1)`. + * - A plugin module that fails to IMPORT is caught and only LOGGED by the + * cordis Loader (`entry._init`), leaving the entry with no `fiber` and + * producing no rejection — the process would otherwise exit 0 with a usable + * config typo reported only as a log line; {@link assertEntriesLoaded} makes + * `boot()` reject on any such entry instead of returning a half-empty + * context. + * + * @module @deepseek-ai/dsh-app-boot + */ + +import { pathToFileURL } from 'node:url' +import { basename, dirname, resolve } from 'node:path' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' + +/** + * Resolve the config to boot, honoring snapshot REPLAY. Given the requested + * path, replay mode swaps a `cordis.yml` basename for `cordis.snapshot.yml` in + * the SAME directory (the keyless replay tree). Other modes — including no + * snapshot mode at all — use the path as-is. Returns an absolute path resolved + * from `cwd`. + */ +export function resolveConfigPath( + configPath: string, snapshotMode: string | undefined, cwd: string = process.cwd(), +): string { + const absolute = resolve(cwd, configPath) + if (snapshotMode !== 'replay') return absolute + const dir = dirname(absolute) + const replayName = basename(absolute).replace(/cordis\.ya?ml$/, 'cordis.snapshot.yml') + return resolve(dir, replayName) +} + +/** + * Load `DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL` from a gitignored `.env` in + * `dir` (Node native `process.loadEnvFile`). An absent file is fine — the + * environment may already carry the variables; the leaf `cordis.yml` reads + * them via the `!!js` tag. A present-but-unreadable `.env` is a real + * misconfiguration: surface it via `warn` (one line, default stderr) rather + * than silently running with the wrong environment. + */ +export function loadEnv( + binName: string, dir: string = process.cwd(), + warn: (line: string) => void = line => void process.stderr.write(line), +): void { + try { + process.loadEnvFile(resolve(dir, '.env')) + } catch (error) { + if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { + warn(`${binName}: failed to load .env: ${String(error)}\n`) + } + // ENOENT (no .env) is fine — rely on the ambient environment. + } +} + +/** + * The slice of `process` {@link installFailLoud} needs — injectable so tests + * exercise the handler without registering on (or exiting) the real process. + */ +export interface FailLoudProcess { + on(event: 'unhandledRejection', handler: (err: unknown) => void): unknown + off(event: 'unhandledRejection', handler: (err: unknown) => void): unknown + stderr: { write(chunk: string): unknown } + exit(code: number): void +} + +/** + * Make a load failure fail loud with a clear message on stderr. Covers the + * failure path {@link assertEntriesLoaded} cannot: an include whose + * `[Service.init]` throws (e.g. a config FILE that does not exist in a real + * directory) surfaces as an unhandled promise rejection AFTER `boot()` + * resolves. Node's default handler already exits non-zero on an unhandled + * rejection; this replaces the noisy stack dump with a single labelled line on + * STDERR (never stdout — for the ACP bin that channel carries JSON-RPC) and + * guarantees `exit(1)`. Install before `boot()`. Returns the uninstaller + * (tests use it; the bins run until exit and never do). + */ +export function installFailLoud(binName: string, proc: FailLoudProcess = process): () => void { + const handler = (err: unknown): void => { + proc.stderr.write(`${binName}: fatal load failure: ${err instanceof Error ? err.stack ?? err.message : String(err)}\n`) + proc.exit(1) + } + proc.on('unhandledRejection', handler) + return () => void proc.off('unhandledRejection', handler) +} + +/** + * After the tree settles, assert every loader entry actually started. A + * started entry has a `fiber`; an entry with `fiber === undefined` after the + * tree settled never loaded (its module failed to import), so throw and let + * `boot()` reject instead of returning a half-empty context. A `disabled` + * entry is the one legitimate fiber-less state: `Entry.refresh()` deliberately + * skips `init()` for it — a valid "plugin turned off" config, not a failed + * import — so it is excluded. + */ +export function assertEntriesLoaded(ctx: Context, binName: string): void { + const failed = [...ctx.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled) + if (failed.length > 0) { + const names = failed.map(entry => entry.options.name).join(', ') + throw new Error(`${binName}: plugin(s) failed to load: ${names} (see the error(s) logged above)`) + } +} + +/** + * Boot the Loader against `absoluteConfigPath` and return the root context + * once the whole tree has settled. The include is handed the config's ABSOLUTE + * `file://` URL as its `path`, so resolution never depends on `ctx.baseUrl` + * (an absolute URL ignores the base) and can never fall back to the cwd; + * `baseUrl` is still pinned to the config's directory so the config's OWN + * relative plugin/include paths resolve against it. + * + * The `await ctx.loader.await()` is load-bearing: `loader.create()` returns + * once the include ENTRY is registered, but the include then loads its child + * plugins asynchronously — without awaiting the tree, `boot()` would resolve + * while the app's plugins are still mounting, and a CLI process with no + * attached handles yet exits 0 silently. Failures surface two ways: an entry + * whose module failed to import is caught here by {@link assertEntriesLoaded} + * (this `boot()` rejects); an init that THROWS surfaces as an unhandled + * rejection caught by {@link installFailLoud} (installed by the bin first). + * + * Bare plugin specifiers in the config (`@deepseek-ai/dsh-*`, npm packages) + * are resolved by the cordis Loader's internal module loader, which is only + * active under `node --expose-internals`; a consumer running a built bin must + * pass that flag (or install the plugins where node hoists them). Relative + * specifiers resolve against the config directory with no flag. + */ +export async function boot(binName: string, absoluteConfigPath: string): Promise { + const ctx = new Context() + ctx.baseUrl = pathToFileURL(dirname(absoluteConfigPath)).href + '/' + await ctx.plugin(Loader) + await ctx.loader.create({ + name: '@cordisjs/plugin-include', + config: { path: pathToFileURL(absoluteConfigPath).href }, + }) + await ctx.loader.await() + assertEntriesLoaded(ctx, binName) + return ctx +} diff --git a/packages/ui/app-boot/tests/app-boot.spec.ts b/packages/ui/app-boot/tests/app-boot.spec.ts new file mode 100644 index 0000000000..510186ebb0 --- /dev/null +++ b/packages/ui/app-boot/tests/app-boot.spec.ts @@ -0,0 +1,178 @@ +import { mkdtempSync, mkdirSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join, resolve, sep } from 'node:path' +import { describe, expect, it, vi } from 'vitest' +import type { Context } from 'cordis' +import { + assertEntriesLoaded, boot, installFailLoud, loadEnv, resolveConfigPath, + type FailLoudProcess, +} from '../src/index.ts' + +const NAME = 'dsh-test-bin' + +const tmp = (): string => mkdtempSync(join(tmpdir(), 'dsh-app-boot-')) + +describe('resolveConfigPath', () => { + it('resolves relative to the given cwd outside replay mode', () => { + expect(resolveConfigPath('./cordis.yml', undefined, `${sep}base`)).toBe(resolve(`${sep}base`, 'cordis.yml')) + expect(resolveConfigPath('conf/app.yaml', 'record', `${sep}base`)).toBe(resolve(`${sep}base`, 'conf/app.yaml')) + }) + + it('swaps a cordis.yml/.yaml basename for cordis.snapshot.yml in replay mode', () => { + expect(resolveConfigPath('./cordis.yml', 'replay', `${sep}base`)).toBe(resolve(`${sep}base`, 'cordis.snapshot.yml')) + expect(resolveConfigPath('deep/cordis.yaml', 'replay', `${sep}base`)).toBe(resolve(`${sep}base`, 'deep/cordis.snapshot.yml')) + }) + + it('leaves a non-cordis basename alone in replay mode and defaults cwd to the process cwd', () => { + expect(resolveConfigPath('custom.yml', 'replay', `${sep}base`)).toBe(resolve(`${sep}base`, 'custom.yml')) + expect(resolveConfigPath('./x.yml', undefined)).toBe(resolve(process.cwd(), 'x.yml')) + }) +}) + +describe('loadEnv', () => { + it('loads variables from .env in the given dir', () => { + const dir = tmp() + writeFileSync(join(dir, '.env'), 'DSH_APP_BOOT_SPEC_VAR=loaded\n') + const warn = vi.fn() + loadEnv(NAME, dir, warn) + expect(process.env['DSH_APP_BOOT_SPEC_VAR']).toBe('loaded') + expect(warn).not.toHaveBeenCalled() + delete process.env['DSH_APP_BOOT_SPEC_VAR'] + }) + + it('stays silent when no .env exists (ambient environment wins)', () => { + const warn = vi.fn() + loadEnv(NAME, tmp(), warn) + expect(warn).not.toHaveBeenCalled() + }) + + it('warns (labelled, single line) when .env exists but cannot be loaded', () => { + const dir = tmp() + mkdirSync(join(dir, '.env')) // a directory named .env: present, unreadable as a file + const warn = vi.fn() + loadEnv(NAME, dir, warn) + expect(warn).toHaveBeenCalledTimes(1) + expect(warn.mock.calls[0]?.[0]).toMatch(new RegExp(`^${NAME}: failed to load \\.env: `)) + }) + + it('defaults dir to the process cwd and warn to a stderr write', () => { + const dir = tmp() + writeFileSync(join(dir, '.env'), 'DSH_APP_BOOT_SPEC_DEFAULTS=yes\n') + const previous = process.cwd() + process.chdir(dir) + try { + loadEnv(NAME) // happy path: the default warn sink is never invoked + } finally { + process.chdir(previous) + } + expect(process.env['DSH_APP_BOOT_SPEC_DEFAULTS']).toBe('yes') + delete process.env['DSH_APP_BOOT_SPEC_DEFAULTS'] + // The default warn sink itself: point it at a broken .env with stderr + // spied, so the arrow body runs without polluting the test output. + const broken = tmp() + mkdirSync(join(broken, '.env')) + const write = vi.spyOn(process.stderr, 'write').mockImplementation(() => true) + let written: string[] + try { + loadEnv(NAME, broken) + written = write.mock.calls.map(call => String(call[0])) + } finally { + write.mockRestore() + } + expect(written).toHaveLength(1) + expect(written[0]).toContain(`${NAME}: failed to load .env: `) + }) +}) + +describe('installFailLoud', () => { + function fakeProc(): FailLoudProcess & { handlers: Array<(err: unknown) => void>; written: string[]; exits: number[] } { + const handlers: Array<(err: unknown) => void> = [] + const written: string[] = [] + const exits: number[] = [] + return { + handlers, written, exits, + on: (_event, handler) => { handlers.push(handler) }, + off: (_event, handler) => { handlers.splice(handlers.indexOf(handler), 1) }, + stderr: { write: (chunk: string) => { written.push(chunk) } }, + exit: (code: number) => { exits.push(code) }, + } + } + + it('writes one labelled line with the stack and exits 1 on an Error rejection', () => { + const proc = fakeProc() + installFailLoud(NAME, proc) + const error = new Error('boom') + proc.handlers[0]!(error) + expect(proc.written[0]).toContain(`${NAME}: fatal load failure: `) + expect(proc.written[0]).toContain(error.stack) + expect(proc.exits).toEqual([1]) + }) + + it('stringifies a non-Error rejection and an Error without a stack falls back to its message', () => { + const proc = fakeProc() + installFailLoud(NAME, proc) + proc.handlers[0]!('plain failure') + expect(proc.written[0]).toContain('plain failure') + const stackless = new Error('no stack') + delete (stackless as { stack?: string }).stack + proc.handlers[0]!(stackless) + expect(proc.written[1]).toContain('no stack') + expect(proc.exits).toEqual([1, 1]) + }) + + it('returns an uninstaller that removes the handler (and defaults to the real process)', () => { + const proc = fakeProc() + const uninstall = installFailLoud(NAME, proc) + expect(proc.handlers).toHaveLength(1) + uninstall() + expect(proc.handlers).toHaveLength(0) + // Default-proc arm: install on the real process, then immediately uninstall + // so the suite leaks no handler and can never exit the runner. + const before = process.listenerCount('unhandledRejection') + const uninstallReal = installFailLoud(NAME) + expect(process.listenerCount('unhandledRejection')).toBe(before + 1) + uninstallReal() + expect(process.listenerCount('unhandledRejection')).toBe(before) + }) +}) + +describe('assertEntriesLoaded', () => { + const ctxWith = (entries: Array<{ fiber?: unknown; disabled?: boolean; options: { name?: string } }>): Context => + ({ loader: { entries: () => entries } }) as unknown as Context + + it('passes when every enabled entry has a fiber', () => { + expect(() => { assertEntriesLoaded(ctxWith([ + { fiber: {}, options: { name: 'a' } }, + { disabled: true, options: { name: 'off' } }, + ]), NAME) }).not.toThrow() + }) + + it('throws naming every enabled fiber-less entry', () => { + expect(() => { assertEntriesLoaded(ctxWith([ + { fiber: {}, options: { name: 'ok' } }, + { options: { name: 'broken-a' } }, + { options: { name: 'broken-b' } }, + ]), NAME) }).toThrow(`${NAME}: plugin(s) failed to load: broken-a, broken-b`) + }) +}) + +describe('boot', () => { + it('boots a leaf config through the real Loader and settles the tree', async () => { + const dir = tmp() + writeFileSync(join(dir, 'noop.mjs'), 'export const name = "noop"\nexport function apply() {}\n') + writeFileSync(join(dir, 'cordis.yml'), '- id: noop\n name: ./noop.mjs\n') + const ctx = await boot(NAME, join(dir, 'cordis.yml')) + try { + const entries = [...ctx.loader.entries()] + expect(entries.some(entry => entry.options.name === './noop.mjs' && entry.fiber !== undefined)).toBe(true) + } finally { + await ctx.fiber.dispose() + } + }) + + it('rejects (never exits 0 half-empty) when a config names a plugin that cannot be imported', async () => { + const dir = tmp() + writeFileSync(join(dir, 'cordis.yml'), '- id: ghost\n name: ./missing.mjs\n') + await expect(boot(NAME, join(dir, 'cordis.yml'))).rejects.toThrow(`${NAME}: plugin(s) failed to load: ./missing.mjs`) + }) +}) diff --git a/packages/ui/app-boot/tsconfig.json b/packages/ui/app-boot/tsconfig.json new file mode 100644 index 0000000000..3171312de4 --- /dev/null +++ b/packages/ui/app-boot/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/loader" + }, + { + "path": "../../../vendor/include" + } + ] +} diff --git a/packages/ui/stdio-agent/package.json b/packages/ui/stdio-agent/package.json index b2dfa47891..0dffe49211 100644 --- a/packages/ui/stdio-agent/package.json +++ b/packages/ui/stdio-agent/package.json @@ -32,6 +32,7 @@ "peerDependencies": { "@cordisjs/plugin-include": "^1.0.4", "@cordisjs/plugin-loader": "^1.0.0-rc.4", + "@deepseek-ai/dsh-app-boot": "^0.0.1", "@cordisjs/plugin-logger-console": "^1.0.0", "@deepseek-ai/dsh-agent": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", @@ -44,6 +45,7 @@ "devDependencies": { "@cordisjs/plugin-include": "workspace:^", "@cordisjs/plugin-loader": "workspace:^", + "@deepseek-ai/dsh-app-boot": "workspace:^", "@cordisjs/plugin-logger-console": "workspace:^", "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", diff --git a/packages/ui/stdio-agent/src/bin.ts b/packages/ui/stdio-agent/src/bin.ts index a07bb2e600..7056486996 100644 --- a/packages/ui/stdio-agent/src/bin.ts +++ b/packages/ui/stdio-agent/src/bin.ts @@ -2,139 +2,24 @@ /** * The `dsh-stdio-agent` bin: boot a Cordis app from a leaf `cordis.yml` that * loads the {@link @deepseek-ai/dsh-stdio-agent} app plugin (plus a backend LLM - * adapter and a bash executor). Owns the boot glue the three `examples/*` once - * duplicated in their `start.ts`: load the gitignored repo-root `.env`, then - * drive the cordis Loader against the config path (default `./cordis.yml`). + * adapter and a bash executor). The boot glue — `.env` loading, the fail-loud + * Loader guards, the settle-the-tree boot sequence — lives in + * {@link @deepseek-ai/dsh-app-boot}, shared with the ACP bin. * - * Usage: `dsh-stdio-agent [path-to-cordis.yml]`. The `demo:echo` / `demo:repl` - * scripts invoke it with the example's config. + * Usage: `dsh-stdio-agent [path-to-cordis.yml]` (default `./cordis.yml`). The + * `demo:echo` / `demo:repl` scripts invoke it with the example's config. * * @module @deepseek-ai/dsh-stdio-agent/bin */ -import { pathToFileURL } from 'node:url' -import { dirname, resolve } from 'node:path' -import { Context } from 'cordis' -import Loader from '@cordisjs/plugin-loader' +import { boot, installFailLoud, loadEnv, resolveConfigPath } from '@deepseek-ai/dsh-app-boot' -/** - * Load `DEEPSEEK_API_KEY` / `DEEPSEEK_BASE_URL` from a gitignored `.env` in the - * CURRENT WORKING DIRECTORY (Node native `process.loadEnvFile`). An absent file - * is fine — the environment may already carry the variables; the leaf - * `cordis.yml` reads them via the `!!js` tag. A present-but-unreadable/malformed - * `.env` is a real misconfiguration: surface it on stderr rather than silently - * running with the wrong environment. The mock-model demo (echo) ships no key - * and simply has no `.env`. - */ -function loadEnv(): void { - try { - process.loadEnvFile(resolve(process.cwd(), '.env')) - } catch (error) { - if ((error as NodeJS.ErrnoException | null)?.code !== 'ENOENT') { - process.stderr.write(`dsh-stdio-agent: failed to load .env: ${String(error)}\n`) - } - // ENOENT (no .env) is fine — rely on the ambient environment. - } -} +const NAME = 'dsh-stdio-agent' -/** - * Make a load failure fail loud with a clear message on stderr. Covers the - * failure path the entry-tree check below cannot: when the include's - * `[Service.init]` throws (e.g. a config FILE that does not exist in a real - * directory), the cordis Loader surfaces it as an unhandled promise rejection - * AFTER `boot()` has resolved — `loader.await()` does NOT rethrow it, because - * `EntryTree.await()` uses `Promise.allSettled`, which swallows rejections. - * Node's default handler already exits non-zero on an unhandled rejection, so - * this does not change the exit code; it replaces Node's noisy stack dump with a - * single labelled line and guarantees `process.exit(1)`. Install before `boot()`. - */ -export function installFailLoud(): void { - process.on('unhandledRejection', (err: unknown) => { - process.stderr.write(`dsh-stdio-agent: fatal load failure: ${err instanceof Error ? err.stack ?? err.message : String(err)}\n`) - process.exit(1) - }) -} - -/** - * After the tree settles, assert every loader entry actually started. This is - * the load-bearing guard against the SILENT-exit-0 bug: when a plugin module - * fails to IMPORT (e.g. a config path in a non-existent directory, so the include - * plugin itself cannot be resolved), the cordis Loader catches the import error - * and only LOGS it (`entry._init`), leaving the entry with no `fiber` and - * producing no rejection — so the process would otherwise exit 0 with a usable - * config typo reported only as a log line. A started entry has a `fiber`; an - * entry with `fiber === undefined` after the tree settled never loaded. Throw on - * any such entry so `boot()` rejects (and the top-level `await` fails the process - * non-zero) instead of returning a half-empty context. - * - * A `disabled` entry is the one legitimate fiber-less state: `Entry.refresh()` - * deliberately skips `init()` for it, so it settles without a fiber by design. - * That is a valid config (a consumer turning an optional plugin off), not a - * failed import — exclude it so the guard catches only real load failures. - */ -function assertEntriesLoaded(ctx: Context): void { - const failed = [...ctx.loader.entries()].filter(entry => entry.fiber === undefined && !entry.disabled) - if (failed.length > 0) { - const names = failed.map(entry => entry.options.name).join(', ') - throw new Error(`dsh-stdio-agent: plugin(s) failed to load: ${names} (see the error(s) logged above)`) - } -} - -/** - * Boot the Loader against `configPath` (resolved from the CWD). The include is - * handed the config's ABSOLUTE `file://` URL as its `path`, so resolution never - * depends on `ctx.baseUrl` (an absolute URL ignores the base) and can never fall - * back to the cwd. `baseUrl` is still pinned to the config's directory so the - * config's OWN relative plugin/include paths (e.g. `./src/mock-llm.ts`) resolve - * against it. Returns the root context once the whole tree has settled. - * - * The `await ctx.loader.await()` is load-bearing: `loader.create()` returns once - * the include ENTRY is registered, but the include then loads its child plugins - * asynchronously. Without awaiting the tree, `boot()` (and `main()`) would - * resolve while the app plugins — the stdin reader, the agent loop — are still - * mounting, and a CLI process with no attached handles yet exits 0 silently. - * Awaiting the tree keeps the process alive until the app's handles are attached. - * - * `loader.await()` does NOT, however, rethrow load errors (`EntryTree.await()` - * uses `Promise.allSettled`), so failures are surfaced two ways: a plugin that - * fails to IMPORT leaves an entry with no fiber, caught here by - * {@link assertEntriesLoaded} (this `boot()` rejects); a plugin whose init - * THROWS surfaces as an unhandled rejection caught by {@link installFailLoud} - * (installed by `main()` before this runs). Together they make any load failure - * exit non-zero with a clear message. - * - * Bare plugin specifiers in the config (`@deepseek-ai/dsh-*`, npm packages) are - * resolved by the cordis Loader's internal module loader, which is only active - * under `node --expose-internals` (the flag the `demo:echo`/`demo:repl` scripts - * pass). Without it the Loader falls back to resolving relative to its own module - * and cannot find the config's plugins, so a consumer running the built bin must - * pass `--expose-internals` (or install the plugins where node hoists them). - */ -export async function boot(configPath: string): Promise { - const absolute = resolve(process.cwd(), configPath) - const ctx = new Context() - ctx.baseUrl = pathToFileURL(dirname(absolute)).href + '/' - await ctx.plugin(Loader) - await ctx.loader.create({ - name: '@cordisjs/plugin-include', - config: { path: pathToFileURL(absolute).href }, - }) - await ctx.loader.await() - assertEntriesLoaded(ctx) - return ctx -} - -/** - * Entry point: install the fail-loud guard, load `.env`, then boot the config - * named on argv (default `./cordis.yml`). Awaited at the module top level by the - * published bin (`#!/usr/bin/env node` shebang via the package's `bin` field). - */ -export async function main(argv: string[] = process.argv.slice(2)): Promise { - installFailLoud() - loadEnv() - await boot(argv[0] ?? './cordis.yml') -} - -/* v8 ignore start -- top-level CLI invocation; the testable core is boot()/main(), driven by the keyless Loader-path smoke */ -await main() +/* v8 ignore start -- thin self-executing composition over the unit-tested + dsh-app-boot helpers; exercised end-to-end by the keyless Loader-path and + built-bin smokes */ +installFailLoud(NAME) +loadEnv(NAME) +await boot(NAME, resolveConfigPath(process.argv[2] ?? './cordis.yml', undefined)) /* v8 ignore stop */ diff --git a/packages/ui/stdio-agent/tests/built-bin.e2e.ts b/packages/ui/stdio-agent/tests/built-bin.e2e.ts index 30bc952673..cda8b41b1f 100644 --- a/packages/ui/stdio-agent/tests/built-bin.e2e.ts +++ b/packages/ui/stdio-agent/tests/built-bin.e2e.ts @@ -35,7 +35,7 @@ const stdioBin = join(repoRoot, 'packages/ui/stdio-agent/lib/bin.js') const dshPackages = [ 'core/agent-core', 'core/agent', 'core/session', 'core/system-prompt', 'core/tools', 'core/agent-loop', 'llm/llm', 'bash/bash', 'bash/bash-local', - 'bash/tool-bash', 'support/invariants', + 'bash/tool-bash', 'support/invariants', 'ui/app-boot', 'session-persistence/session-persistence', 'session-persistence/session-persistence-jsonl', 'ui/stdio-agent', ] diff --git a/packages/ui/stdio-agent/tsconfig.json b/packages/ui/stdio-agent/tsconfig.json index 948813c370..8b76352d11 100644 --- a/packages/ui/stdio-agent/tsconfig.json +++ b/packages/ui/stdio-agent/tsconfig.json @@ -17,6 +17,9 @@ { "path": "../../../vendor/loader" }, + { + "path": "../app-boot" + }, { "path": "../../../vendor/logger-console" }, diff --git a/packages/web/tool-web/README.md b/packages/web/tool-web/README.md index f0bd66cdb9..d8a2e266a9 100644 --- a/packages/web/tool-web/README.md +++ b/packages/web/tool-web/README.md @@ -28,4 +28,4 @@ Each tool is registered independently; a product that wants only one disables th Tool registration follows product **enablement**, not backend availability. A tool stays visible even when its selected provider is missing, misconfigured, ambiguous, or temporarily unavailable; the seam resolves the provider at execution time and execution fails with a structured `WebError` (e.g. `WEB_PROVIDER_UNAVAILABLE`, `WEB_PROVIDER_AMBIGUOUS`), which `ToolRegistry.execute()` turns into an error tool result the model can read and hooks/UI can route on. This keeps the model schema stable without making plugin load order, credential state, or HMR timing part of the model-facing contract. To remove a web tool entirely, disable it here in config. -The tool reads only the aggregated `ctx.web.searchStatus()` / `fetchStatus()` for diagnostics — never each provider's `status()` directly — so provider selection has one owner. +The tool never calls a provider's `status()` and never enumerates providers — its only execution path is `ctx.web.search()` / `ctx.web.fetch()`, and provider unavailability reaches it as the structured `WebError` codes selection throws at execution time. Provider selection stays entirely inside the seam, with one owner. diff --git a/packages/web/tool-web/tests/tool-web.spec.ts b/packages/web/tool-web/tests/tool-web.spec.ts index c253bebfef..2347f6a8a8 100644 --- a/packages/web/tool-web/tests/tool-web.spec.ts +++ b/packages/web/tool-web/tests/tool-web.spec.ts @@ -188,9 +188,12 @@ describe('tool-web registration', () => { }) it('registers web_search even when no provider is available (schema follows enablement, not availability)', async () => { - const { fiber, ctx } = await mountTools() + const { fiber, ctx, call } = await mountTools() expect(ctx.tools.schemas().map(s => s.name)).toContain('web_search') - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'none' }) + // No provider is registered: the schema stays visible and execution reports + // the structured unavailability instead. + const out = await call('web_search', { query: 'q' }) + expect(out.error?.code).toBe('WEB_PROVIDER_UNAVAILABLE') await fiber.dispose() }) diff --git a/packages/web/web-fetch-local/tests/fetch-local.spec.ts b/packages/web/web-fetch-local/tests/fetch-local.spec.ts index 27ed991c08..7d4c4e683b 100644 --- a/packages/web/web-fetch-local/tests/fetch-local.spec.ts +++ b/packages/web/web-fetch-local/tests/fetch-local.spec.ts @@ -377,9 +377,11 @@ describe('web-fetch-local plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID }) const fiber = await ctx.plugin(fetchPlugin, {}) - expect(ctx.web.fetchStatus()).toEqual({ available: true, providerId: LOCAL_FETCH_PROVIDER_ID }) + await expect(ctx.web.fetch({ url: `${base}/` })) + .resolves.toMatchObject({ providerId: LOCAL_FETCH_PROVIDER_ID, statusCode: 200 }) await fiber.dispose() - expect(ctx.web.fetchStatus()).toEqual({ available: false, reason: 'configured-missing' }) + await expect(ctx.web.fetch({ url: `${base}/` })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' })) }) it('has no default export (namespace plugin export shape)', () => { @@ -418,7 +420,8 @@ describe('web-fetch-local plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { fetchProvider: LOCAL_FETCH_PROVIDER_ID }) const fiber = await ctx.plugin(fetchPlugin, { maxRedirects: 0 }) - expect(ctx.web.fetchStatus()).toEqual({ available: true, providerId: LOCAL_FETCH_PROVIDER_ID }) + await expect(ctx.web.fetch({ url: `${base}/` })) + .resolves.toMatchObject({ providerId: LOCAL_FETCH_PROVIDER_ID, statusCode: 200 }) await fiber.dispose() }) }) diff --git a/packages/web/web-search-deepseek/tests/deepseek.spec.ts b/packages/web/web-search-deepseek/tests/deepseek.spec.ts index ef688b7ad2..0faab1f35a 100644 --- a/packages/web/web-search-deepseek/tests/deepseek.spec.ts +++ b/packages/web/web-search-deepseek/tests/deepseek.spec.ts @@ -264,12 +264,14 @@ describe('DeepSeekSearchProvider error handling', () => { describe('web-search-deepseek plugin registration', () => { it('registers the provider into ctx.web (HMR-safe)', async () => { + vi.stubGlobal('fetch', vi.fn(async () => jsonResponse(searchResponse()))) const ctx = new Context() await ctx.plugin(WebService, { searchProvider: DEEPSEEK_PROVIDER_ID }) const fiber = await ctx.plugin(deepseekPlugin, { apiKey: 'ds-key' }) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: DEEPSEEK_PROVIDER_ID }) + await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: DEEPSEEK_PROVIDER_ID }) await fiber.dispose() - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'configured-missing' }) + await expect(ctx.web.search({ query: 'q' })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' })) }) it('rejects maxTokens: 0 at plugin construction', async () => { @@ -315,13 +317,14 @@ describe('web-search-deepseek plugin registration', () => { }) it('boots over ctx.web through the unwrapped module without an inject error', async () => { + vi.stubGlobal('fetch', vi.fn(async () => jsonResponse(searchResponse()))) const ctx = new Context() await ctx.plugin(WebService, { searchProvider: DEEPSEEK_PROVIDER_ID }) const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(deepseekPlugin) as Parameters[0] // A collapsed export shape (dropped inject) would throw "without inject" here. const fiber = await ctx.plugin(unwrapped, { apiKey: 'ds-key' }) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: DEEPSEEK_PROVIDER_ID }) + await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: DEEPSEEK_PROVIDER_ID }) await fiber.dispose() }) @@ -334,7 +337,6 @@ describe('web-search-deepseek plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { searchProvider: DEEPSEEK_PROVIDER_ID }) const fiber = await ctx.plugin(deepseekPlugin, {}) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: DEEPSEEK_PROVIDER_ID }) await ctx.web.search({ query: 'q' }) const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit] expect(url).toBe('https://api.deepseek.com/anthropic/v1/messages') @@ -354,7 +356,8 @@ describe('web-search-deepseek plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { searchProvider: DEEPSEEK_PROVIDER_ID }) await ctx.plugin(deepseekPlugin, {}) - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'configured-unavailable' }) + await expect(ctx.web.search({ query: 'q' })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE' })) } finally { if (prev !== undefined) process.env.DEEPSEEK_API_KEY = prev } diff --git a/packages/web/web-search-exa/tests/exa.spec.ts b/packages/web/web-search-exa/tests/exa.spec.ts index 9cf31332f5..204b18f702 100644 --- a/packages/web/web-search-exa/tests/exa.spec.ts +++ b/packages/web/web-search-exa/tests/exa.spec.ts @@ -205,12 +205,14 @@ describe('ExaSearchProvider error handling', () => { describe('web-search-exa plugin registration', () => { it('registers the provider into ctx.web (HMR-safe)', async () => { + vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ results: [] }))) const ctx = new Context() await ctx.plugin(WebService, { searchProvider: EXA_PROVIDER_ID }) const fiber = await ctx.plugin(exaPlugin, { apiKey: 'exa-key' }) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: EXA_PROVIDER_ID }) + await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: EXA_PROVIDER_ID }) await fiber.dispose() - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'configured-missing' }) + await expect(ctx.web.search({ query: 'q' })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' })) }) it('has no default export (namespace plugin export shape)', () => { @@ -238,7 +240,6 @@ describe('web-search-exa plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { searchProvider: EXA_PROVIDER_ID }) const fiber = await ctx.plugin(exaPlugin, {}) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: EXA_PROVIDER_ID }) await ctx.web.search({ query: 'q' }) const [url] = fetchMock.mock.calls[0] as unknown as [string] expect(url).toBe('https://api.exa.ai/search') @@ -256,7 +257,8 @@ describe('web-search-exa plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { searchProvider: EXA_PROVIDER_ID }) await ctx.plugin(exaPlugin, {}) - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'configured-unavailable' }) + await expect(ctx.web.search({ query: 'q' })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE' })) } finally { if (prev !== undefined) process.env.EXA_API_KEY = prev } diff --git a/packages/web/web-search-perplexity/tests/perplexity.spec.ts b/packages/web/web-search-perplexity/tests/perplexity.spec.ts index 70a9a4c98b..df8a98f003 100644 --- a/packages/web/web-search-perplexity/tests/perplexity.spec.ts +++ b/packages/web/web-search-perplexity/tests/perplexity.spec.ts @@ -186,12 +186,14 @@ describe('PerplexitySearchProvider error handling', () => { describe('web-search-perplexity plugin registration', () => { it('registers the provider into ctx.web (HMR-safe)', async () => { + vi.stubGlobal('fetch', vi.fn(async () => jsonResponse({ choices: [{ message: { content: 'a' } }], citations: [] }))) const ctx = new Context() await ctx.plugin(WebService, { searchProvider: PERPLEXITY_PROVIDER_ID }) const fiber = await ctx.plugin(perplexityPlugin, { apiKey: 'pplx-key' }) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: PERPLEXITY_PROVIDER_ID }) + await expect(ctx.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: PERPLEXITY_PROVIDER_ID }) await fiber.dispose() - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'configured-missing' }) + await expect(ctx.web.search({ query: 'q' })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_MISSING' })) }) it('has no default export (namespace plugin export shape)', () => { @@ -219,7 +221,6 @@ describe('web-search-perplexity plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { searchProvider: PERPLEXITY_PROVIDER_ID }) const fiber = await ctx.plugin(perplexityPlugin, {}) - expect(ctx.web.searchStatus()).toEqual({ available: true, providerId: PERPLEXITY_PROVIDER_ID }) await ctx.web.search({ query: 'q' }) const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit] expect(url).toBe('https://api.perplexity.ai/chat/completions') @@ -238,7 +239,8 @@ describe('web-search-perplexity plugin registration', () => { const ctx = new Context() await ctx.plugin(WebService, { searchProvider: PERPLEXITY_PROVIDER_ID }) await ctx.plugin(perplexityPlugin, {}) - expect(ctx.web.searchStatus()).toEqual({ available: false, reason: 'configured-unavailable' }) + await expect(ctx.web.search({ query: 'q' })) + .rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_CONFIGURED_UNAVAILABLE' })) } finally { if (prev !== undefined) process.env.PERPLEXITY_API_KEY = prev } diff --git a/packages/web/web/README.md b/packages/web/web/README.md index 9b9e2ca333..fe5f1e650e 100644 --- a/packages/web/web/README.md +++ b/packages/web/web/README.md @@ -18,8 +18,7 @@ Search and fetch share no request schema and no business logic, but they are del | Member | Semantics | |---|---| -| `registerSearchProvider(provider)` / `registerFetchProvider(provider)` | Register a backend. Throws `WebError` `WEB_DUPLICATE_PROVIDER` on a duplicate id within that capability kind. Returns a disposer; emits `web/providers-change` on register and on dispose. Disposed with the calling fiber. | -| `searchStatus()` / `fetchStatus()` | Derived (never stored) `WebCapabilityStatus`: whether the capability has a selected usable provider, or the broad category it fails in. Diagnostics + execution-resolution input. | +| `registerSearchProvider(provider)` / `registerFetchProvider(provider)` | Register a backend. Throws `WebError` `WEB_DUPLICATE_PROVIDER` on a duplicate id within that capability kind. Returns a disposer. Disposed with the calling fiber. | | `search(request, exec?)` | Resolve the search provider and run one search. Enforces `request.maxResults` on the result (truncates `sources[]`, sets `truncated`). Throws `WebError` when the capability cannot run. | | `fetch(request, exec?)` | Resolve the fetch provider and retrieve one URL. A non-2xx response is a result, not a throw. Throws `WebError` for failures to safely retrieve or represent the resource. | @@ -27,18 +26,18 @@ Providers register **capabilities**, not tools. `dsh-tool-web` is the only owner ## Selection -Selection never depends on registration, config, or HMR order. A capability has an explicit provider id (config `searchProvider`/`fetchProvider`, or env `$DSH_WEB_SEARCH_PROVIDER`/`$DSH_WEB_FETCH_PROVIDER` feeding the same fields), or auto-selects when exactly one usable provider is registered: +Selection never depends on registration, config, or HMR order. A capability has an explicit provider id (config `searchProvider`/`fetchProvider`, or env `$DSH_WEB_SEARCH_PROVIDER`/`$DSH_WEB_FETCH_PROVIDER` feeding the same fields), or auto-selects when exactly one usable provider is registered. `search()`/`fetch()` resolve the provider at execution time: -| Situation | `WebCapabilityStatus` | Execution | -|---|---|---| -| configured id registered and `status().available` | `available` for it | runs | -| configured id not registered | `configured-missing` | `WEB_PROVIDER_CONFIGURED_MISSING` | -| configured id registered but unavailable | `configured-unavailable` | `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` | -| no id, exactly one registered usable provider | `available` for it | runs | -| no id, no usable provider | `none` | `WEB_PROVIDER_UNAVAILABLE` | -| no id, multiple usable providers | `ambiguous` | `WEB_PROVIDER_AMBIGUOUS` | +| Situation | Execution | +|---|---| +| configured id registered and `status().available` | runs that provider | +| configured id not registered | `WEB_PROVIDER_CONFIGURED_MISSING` | +| configured id registered but unavailable | `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` | +| no id, exactly one registered usable provider | runs it | +| no id, no usable provider | `WEB_PROVIDER_UNAVAILABLE` | +| no id, multiple usable providers | `WEB_PROVIDER_AMBIGUOUS` | -`WebCapabilityStatus` carries only `available` + a `reason` discriminant (plus the winning `providerId` on the available branch). The branchable per-reason detail lives in the thrown `WebError`, which is the surface callers route on — so the same fact never gets two homes that can disagree. A provider's own `status()` is a cheap local check (credential presence, parseable config) and **must not make network calls**; `dsh-tool-web` reads only the aggregated `searchStatus()`/`fetchStatus()`, never each provider's `status()` directly. +The failure branches throw `WebError`, whose structured code (plus message detail — the missing id, the ambiguous candidate set) is the surface callers route on. A provider's own `status()` is a cheap local check (credential presence, parseable config) that feeds this execution-time selection and **must not make network calls**; `dsh-tool-web` never calls a provider's `status()` — it executes through `ctx.web.search()`/`fetch()` and routes on the thrown codes, so provider selection has one owner. ## Vocabulary diff --git a/packages/web/web/src/index.ts b/packages/web/web/src/index.ts index 76c8065b30..5da736a85a 100644 --- a/packages/web/web/src/index.ts +++ b/packages/web/web/src/index.ts @@ -3,15 +3,14 @@ * execution surface for two capabilities — search and fetch. Provider packages * register concrete backends with `registerSearchProvider` / * `registerFetchProvider`; the model-facing consumer - * (`@deepseek-ai/dsh-tool-web`) reads capability status and executes through - * `search()` / `fetch()`. + * (`@deepseek-ai/dsh-tool-web`) executes through `search()` / `fetch()` and + * routes on the structured {@link WebError} codes selection throws. * * The registry half stays close to `LlmService`: a `Map` per * capability kind, register methods that return disposers, duplicate ids that * throw, and execution-time resolution that throws when the selected provider is - * absent or unusable. On top of that sits one small selection-status layer so - * diagnostics and execution can explain why a capability can or cannot run, - * independent of registration order. + * absent or unusable — with selection rules that never depend on registration + * order. * * @module @deepseek-ai/dsh-web */ @@ -19,7 +18,6 @@ import { Context, Service } from 'cordis' import z from 'schemastery' import type { - WebCapabilityStatus, WebExecContext, WebFetchProvider, WebFetchRequest, @@ -35,7 +33,6 @@ export { WebError, } from './types.ts' export type { - WebCapabilityStatus, WebExecContext, WebFetchBody, WebFetchProvider, @@ -52,21 +49,9 @@ declare module 'cordis' { interface Context { web: WebService } - - interface Events { - /** - * Fired after the provider registry changes — a search or fetch provider was - * registered or disposed. Carries no payload and no capability graph: it - * means only "the provider registry changed; observers may recompute status - * from `ctx.web`". `searchStatus()` / `fetchStatus()` stay derived, not - * stored. - * @mode emit - */ - 'web/providers-change'(this: WebService): void - } } -/** Selection inputs shared by the status query and execution resolution. */ +/** Selection inputs for execution-time provider resolution. */ interface Selection

{ /** The configured provider id for this capability, if any. */ readonly configuredId?: string @@ -90,17 +75,14 @@ export interface WebServiceConfig { /** * The web access service. Registered as `ctx.web` (one instance per context). * - * Selection semantics (identical for status and execution, never order- - * dependent): + * Selection semantics (resolved at execution time, never order-dependent): * - A configured id that is registered and `status().available` → that provider. - * - A configured id not registered → `configured-missing` / - * `WEB_PROVIDER_CONFIGURED_MISSING`. - * - A configured id registered but unavailable → `configured-unavailable` / + * - A configured id not registered → `WEB_PROVIDER_CONFIGURED_MISSING`. + * - A configured id registered but unavailable → * `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`. * - No id configured, exactly one registered usable provider → that provider. - * - No id configured, multiple usable providers → `ambiguous` / - * `WEB_PROVIDER_AMBIGUOUS`. - * - No id configured, no usable provider → `none` / `WEB_PROVIDER_UNAVAILABLE`. + * - No id configured, multiple usable providers → `WEB_PROVIDER_AMBIGUOUS`. + * - No id configured, no usable provider → `WEB_PROVIDER_UNAVAILABLE`. */ export class WebService extends Service { /** @@ -126,9 +108,8 @@ export class WebService extends Service { /** * Register a search provider. Throws {@link WebError} `WEB_DUPLICATE_PROVIDER` - * if its id is already registered for search. Returns a disposer; emits - * `web/providers-change` after a successful register and again on dispose. - * Disposed with the calling fiber. + * if its id is already registered for search. Returns a disposer; disposed + * with the calling fiber. * @param provider - the provider; its `id` is the registry key. * @returns the disposer that unregisters the provider. */ @@ -138,9 +119,8 @@ export class WebService extends Service { /** * Register a fetch provider. Throws {@link WebError} `WEB_DUPLICATE_PROVIDER` - * if its id is already registered for fetch. Returns a disposer; emits - * `web/providers-change` after a successful register and again on dispose. - * Disposed with the calling fiber. + * if its id is already registered for fetch. Returns a disposer; disposed + * with the calling fiber. * @param provider - the provider; its `id` is the registry key. * @returns the disposer that unregisters the provider. */ @@ -152,45 +132,15 @@ export class WebService extends Service { if (store.has(provider.id)) { throw new WebError(`a web provider with id "${provider.id}" is already registered`, 'WEB_DUPLICATE_PROVIDER') } - const dispose = this.ctx.effect(function* (this: WebService) { + const dispose = this.ctx.effect(function* () { store.set(provider.id, provider) - // Yield the rollback BEFORE emitting `web/providers-change`: the generator - // effect collects each yielded disposer before the next step runs, so a - // throwing change listener removes the just-added provider instead of - // leaking it into the registry. - yield () => { - store.delete(provider.id) - this.ctx.emit('web/providers-change') - } - this.ctx.emit('web/providers-change') - }.bind(this), 'web.registerProvider()') + yield () => store.delete(provider.id) + }, 'web.registerProvider()') // ctx.effect's disposer returns Promise; our disposer API is // synchronous fire-and-forget — discard the (always-resolved) promise. return () => void dispose() } - /** - * Search-capability selection status, derived live (never stored). - * @returns which provider would serve a search right now, or why none would. - */ - searchStatus(): WebCapabilityStatus { - return resolveStatus({ - providers: this.searchProviders, - ...this.searchProviderId !== undefined ? { configuredId: this.searchProviderId } : {}, - }) - } - - /** - * Fetch-capability selection status, derived live (never stored). - * @returns which provider would serve a fetch right now, or why none would. - */ - fetchStatus(): WebCapabilityStatus { - return resolveStatus({ - providers: this.fetchProviders, - ...this.fetchProviderId !== undefined ? { configuredId: this.fetchProviderId } : {}, - }) - } - /** * Run one search through the selected provider. Resolves the provider at call * time with the selection rules above; throws {@link WebError} when the @@ -231,27 +181,7 @@ interface ResolvableProvider { status(): WebProviderStatus } -/** Compute the capability status from configured id + registered providers. */ -function resolveStatus

(selection: Selection

): WebCapabilityStatus { - const { configuredId, providers } = selection - if (configuredId !== undefined) { - const provider = providers.get(configuredId) - if (!provider) return { available: false, reason: 'configured-missing' } - if (!provider.status().available) return { available: false, reason: 'configured-unavailable' } - return { available: true, providerId: configuredId } - } - const usable = [...providers.values()].filter(provider => provider.status().available) - const [single] = usable - if (single === undefined) return { available: false, reason: 'none' } - if (usable.length > 1) return { available: false, reason: 'ambiguous' } - return { available: true, providerId: single.id } -} - -/** - * Resolve the selected provider or throw the matching {@link WebError}. Shares - * the selection rules with {@link resolveStatus} so status and execution can - * never disagree. - */ +/** Resolve the selected provider or throw the matching {@link WebError}. */ function resolveProvider

(selection: Selection

): P { const { configuredId, providers } = selection if (configuredId !== undefined) { diff --git a/packages/web/web/src/types.ts b/packages/web/web/src/types.ts index 6f85787d1b..f4cda691b8 100644 --- a/packages/web/web/src/types.ts +++ b/packages/web/web/src/types.ts @@ -1,8 +1,8 @@ /** * Vocabulary for the web capability seam (`ctx.web`): the search/fetch * request/result shapes providers produce and consumers format, the provider - * and capability status discriminants selection reports, the execution-control - * context, and the typed error taxonomy. + * status discriminant selection reads, the execution-control context, and the + * typed error taxonomy. * * These types are shared by every provider backend * (`@deepseek-ai/dsh-web-search-exa`, `@deepseek-ai/dsh-web-search-perplexity`, @@ -128,25 +128,15 @@ export type WebFetchBody = /** * Whether one concrete provider implementation is usable, by cheap local checks * only (credential presence, parseable endpoint config). A provider `status()` - * must NOT make network calls. It is an input to selection, not a health system. + * must NOT make network calls. It is an input to execution-time selection, not + * a health system: `WebService.search()`/`fetch()` read it to pick a usable + * provider, and selection failure surfaces as the structured {@link WebError} + * codes callers route on. */ export type WebProviderStatus = | { readonly available: true } | { readonly available: false; readonly reason: 'missing-credential' | 'misconfigured' } -/** - * Whether a capability (search or fetch) has a selected usable provider, or the - * broad category in which selection fails. Intentionally small: it carries the - * winning `providerId` on the available branch (so diagnostics can report which - * provider won) but NOT the per-reason payload (the missing id, the ambiguous - * candidate set). That branchable detail lives in the {@link WebError} thrown at - * execution time — the surface callers route on — so the same fact does not get - * two homes that can disagree. - */ -export type WebCapabilityStatus = - | { readonly available: true; readonly providerId: string } - | { readonly available: false; readonly reason: 'none' | 'configured-missing' | 'configured-unavailable' | 'ambiguous' } - /** * A search-capable backend. Registered with `ctx.web.registerSearchProvider`. * `id` is a stable string, unique within the search capability kind. diff --git a/packages/web/web/tests/web.spec.ts b/packages/web/web/tests/web.spec.ts index e97630ebab..8189e342da 100644 --- a/packages/web/web/tests/web.spec.ts +++ b/packages/web/web/tests/web.spec.ts @@ -1,4 +1,4 @@ -import { describe, expect, it, vi } from 'vitest' +import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import WebService, { WebError, @@ -42,18 +42,14 @@ async function mountWeb(config: ConstructorParameters[1] = {} } describe('WebService registration', () => { - it('registers and disposes a search provider, emitting providers-change each way', async () => { - const { ctx, web } = await mountWeb() - const changed = vi.fn() - ctx.on('web/providers-change', changed) + it('registers a search provider and unregisters it via the returned disposer', async () => { + const { web } = await mountWeb() const dispose = web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) - expect(changed).toHaveBeenCalledTimes(1) - expect(web.searchStatus()).toEqual({ available: true, providerId: 'exa' }) + await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'exa' }) dispose() - expect(changed).toHaveBeenCalledTimes(2) - expect(web.searchStatus()).toEqual({ available: false, reason: 'none' }) + await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_UNAVAILABLE' })) }) it('throws WEB_DUPLICATE_PROVIDER on a duplicate search id', async () => { @@ -69,88 +65,14 @@ describe('WebService registration', () => { expect(() => web.registerFetchProvider(makeFetchProvider('shared', available, fetchResult('shared')))).not.toThrow() }) - it('rolls back a registration when a providers-change listener throws', async () => { - const { ctx, web } = await mountWeb() - ctx.on('web/providers-change', () => { throw new Error('listener boom') }) - expect(() => web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa'))))) - .toThrow('listener boom') - // The throwing listener must not leave the provider in the registry. - expect(web.searchStatus()).toEqual({ available: false, reason: 'none' }) - }) - it('disposes provider registrations when the contributing fiber is disposed (HMR safety)', async () => { const { ctx, web } = await mountWeb() const fiber = await ctx.plugin(Object.assign((inner: Context) => { inner.web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) }, { inject: ['web'] })) - expect(web.searchStatus()).toEqual({ available: true, providerId: 'exa' }) + await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'exa' }) await fiber.dispose() - expect(web.searchStatus()).toEqual({ available: false, reason: 'none' }) - }) -}) - -describe('WebService selection status', () => { - it('reports none when nothing is registered', async () => { - const { web } = await mountWeb() - expect(web.searchStatus()).toEqual({ available: false, reason: 'none' }) - expect(web.fetchStatus()).toEqual({ available: false, reason: 'none' }) - }) - - it('auto-selects the single usable provider when no id is configured', async () => { - const { web } = await mountWeb() - web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) - expect(web.searchStatus()).toEqual({ available: true, providerId: 'exa' }) - }) - - it('reports ambiguous when multiple usable providers exist and none is configured', async () => { - const { web } = await mountWeb() - web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) - web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) - expect(web.searchStatus()).toEqual({ available: false, reason: 'ambiguous' }) - }) - - it('ignores unusable providers when auto-selecting', async () => { - const { web } = await mountWeb() - web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) - web.registerSearchProvider(makeSearchProvider('perplexity', unavailable, () => Promise.resolve(searchResult('perplexity')))) - expect(web.searchStatus()).toEqual({ available: true, providerId: 'exa' }) - }) - - it('reports none when providers exist but none are usable', async () => { - const { web } = await mountWeb() - web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) - expect(web.searchStatus()).toEqual({ available: false, reason: 'none' }) - }) - - it('honors a configured id over a different registered provider', async () => { - const { web } = await mountWeb({ searchProvider: 'perplexity' }) - web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) - web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) - expect(web.searchStatus()).toEqual({ available: true, providerId: 'perplexity' }) - }) - - it('reports configured-missing when the configured id is not registered', async () => { - const { web } = await mountWeb({ searchProvider: 'perplexity' }) - web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) - expect(web.searchStatus()).toEqual({ available: false, reason: 'configured-missing' }) - }) - - it('reports configured-unavailable when the configured id is registered but unusable', async () => { - const { web } = await mountWeb({ searchProvider: 'exa' }) - web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) - expect(web.searchStatus()).toEqual({ available: false, reason: 'configured-unavailable' }) - }) - - it('does not let registration order change auto-selection', async () => { - const a = await mountWeb() - a.web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) - a.web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) - expect(a.web.searchStatus()).toEqual({ available: true, providerId: 'perplexity' }) - - const b = await mountWeb() - b.web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) - b.web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) - expect(b.web.searchStatus()).toEqual({ available: true, providerId: 'perplexity' }) + await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_UNAVAILABLE' })) }) }) @@ -160,6 +82,12 @@ describe('WebService execution resolution', () => { await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_UNAVAILABLE' })) }) + it('throws WEB_PROVIDER_UNAVAILABLE when providers exist but none are usable', async () => { + const { web } = await mountWeb() + web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) + await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_UNAVAILABLE' })) + }) + it('throws WEB_PROVIDER_CONFIGURED_MISSING for an unregistered configured id', async () => { const { web } = await mountWeb({ searchProvider: 'perplexity' }) web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) @@ -179,6 +107,32 @@ describe('WebService execution resolution', () => { await expect(web.search({ query: 'q' })).rejects.toThrow(expect.objectContaining({ code: 'WEB_PROVIDER_AMBIGUOUS' })) }) + it('runs the configured provider even when another usable provider is registered', async () => { + const { web } = await mountWeb({ searchProvider: 'perplexity' }) + web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) + web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) + await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'perplexity' }) + }) + + it('ignores unusable providers when auto-selecting', async () => { + const { web } = await mountWeb() + web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve(searchResult('exa')))) + web.registerSearchProvider(makeSearchProvider('perplexity', unavailable, () => Promise.resolve(searchResult('perplexity')))) + await expect(web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'exa' }) + }) + + it('does not let registration order change auto-selection', async () => { + const a = await mountWeb() + a.web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) + a.web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) + await expect(a.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'perplexity' }) + + const b = await mountWeb() + b.web.registerSearchProvider(makeSearchProvider('perplexity', available, () => Promise.resolve(searchResult('perplexity')))) + b.web.registerSearchProvider(makeSearchProvider('exa', unavailable, () => Promise.resolve(searchResult('exa')))) + await expect(b.web.search({ query: 'q' })).resolves.toMatchObject({ providerId: 'perplexity' }) + }) + it('runs the selected provider and returns its result', async () => { const { web } = await mountWeb() web.registerSearchProvider(makeSearchProvider('exa', available, () => Promise.resolve( diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 503814d60d..f02d014c04 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -882,6 +882,9 @@ importers: '@deepseek-ai/dsh-agent-core': specifier: workspace:^ version: link:../../core/agent-core + '@deepseek-ai/dsh-app-boot': + specifier: workspace:^ + version: link:../app-boot '@deepseek-ai/dsh-session-persistence-jsonl': specifier: workspace:^ version: link:../../session-persistence/session-persistence-jsonl @@ -892,6 +895,18 @@ importers: specifier: ^3.17.0 version: 3.18.0 + packages/ui/app-boot: + devDependencies: + '@cordisjs/plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@cordisjs/plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + cordis: + specifier: ^4.0.0-rc.6 + version: 4.0.0-rc.6(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader) + packages/ui/stdio-agent: devDependencies: '@cordisjs/plugin-include': @@ -909,6 +924,9 @@ importers: '@deepseek-ai/dsh-agent-core': specifier: workspace:^ version: link:../../core/agent-core + '@deepseek-ai/dsh-app-boot': + specifier: workspace:^ + version: link:../app-boot '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm diff --git a/scripts/doc-typecheck.ts b/scripts/doc-typecheck.ts index 7422d6ac77..c6a5510948 100644 --- a/scripts/doc-typecheck.ts +++ b/scripts/doc-typecheck.ts @@ -9,13 +9,15 @@ * opts out with an explicit ` ```ts ignore-check ` info string — the opt-out * is visible in the source, and this script reports the ratio so the escape * hatch can't quietly become the norm. A third info string, - * doc-typecheck.ts recognizes two more fence variants and skips both (each is a - * separately-checked category, not an unchecked sketch, so neither counts in the - * opt-out ratio): ` ```ts type-equiv ` is a verbatim source-type paste that - * `scripts/verify-type-equiv.ts` drift-checks, and ` ```ts cordis-catalog ` is a + * doc-typecheck.ts recognizes three more fence variants and skips all three (each + * is a separately-checked category, not an unchecked sketch, so none counts in + * the opt-out ratio): ` ```ts type-equiv ` is a verbatim source-type paste that + * `scripts/verify-type-equiv.ts` drift-checks, ` ```ts cordis-catalog ` is a * generated event/service signature fragment in the cordis catalog (a bare * signature is not standalone-compilable; the catalog is generated and frozen by - * `scripts/gen-cordis-catalog.ts` + its `--check` freshness gate). + * `scripts/gen-cordis-catalog.ts` + its `--check` freshness gate), and + * ` ```ts persistence-catalog ` is a generated log-event payload fragment in the + * persistence catalog (same reasoning, frozen by `scripts/gen-persistence-catalog.ts`). * * Run: `tsx scripts/doc-typecheck.ts`. */ @@ -43,8 +45,12 @@ const root = resolve(import.meta.dirname, '..') * (a bare signature fragment has no imports and does not stand alone) and * EXCLUDED from the opt-out ratio: the catalog is generated and frozen by * `scripts/gen-cordis-catalog.ts` + its `--check` freshness gate. + * - `persistence-catalog` (` ```ts persistence-catalog `) — a generated + * log-event payload fragment in the persistence catalog. Same treatment for + * the same reason; frozen by `scripts/gen-persistence-catalog.ts` + its + * `--check` freshness gate. */ -type BlockKind = 'check' | 'ignore' | 'type-equiv' | 'cordis-catalog' +type BlockKind = 'check' | 'ignore' | 'type-equiv' | 'cordis-catalog' | 'persistence-catalog' /** One extracted code block. */ interface Block { @@ -55,7 +61,8 @@ interface Block { code: string } -/** Extract every ts / ts ignore-check / ts type-equiv / ts cordis-catalog block from one Markdown file. */ +/** Extract every ts / ts ignore-check / ts type-equiv / ts cordis-catalog / + * ts persistence-catalog block from one Markdown file. */ function extractBlocks(absPath: string): Block[] { const text = readFileSync(absPath, 'utf8') const lines = text.split('\n') @@ -82,7 +89,8 @@ function extractBlocks(absPath: string): Block[] { : info === 'ts ignore-check' ? 'ignore' : info === 'ts type-equiv' ? 'type-equiv' : info === 'ts cordis-catalog' ? 'cordis-catalog' - : null + : info === 'ts persistence-catalog' ? 'persistence-catalog' + : null if (kind) open = { line: i + 1, kind, body: [] } }) return blocks @@ -131,11 +139,11 @@ files.sort() const all = files.flatMap(extractBlocks) const checked = all.filter(b => b.kind === 'check') const ignored = all.filter(b => b.kind === 'ignore') -// `type-equiv` and `cordis-catalog` blocks are verified elsewhere -// (verify-type-equiv.ts and the gen-cordis-catalog `--check` freshness gate), -// not here: neither compiled nor counted toward the opt-out ratio (each is a -// separate fully-checked category, not an unchecked sketch). The ratio's -// denominator is therefore the compile-eligible blocks only. +// `type-equiv`, `cordis-catalog`, and `persistence-catalog` blocks are verified +// elsewhere (verify-type-equiv.ts and each catalog generator's `--check` +// freshness gate), not here: neither compiled nor counted toward the opt-out +// ratio (each is a separate fully-checked category, not an unchecked sketch). +// The ratio's denominator is therefore the compile-eligible blocks only. const ratioDenominator = checked.length + ignored.length if (checked.length === 0) { @@ -171,7 +179,7 @@ try { const ratio = ignored.length / ratioDenominator const skipped = all.length - ratioDenominator - console.log(`doc-typecheck: ${checked.length} block(s) compiled, ${ignored.length} ignored (${(ratio * 100).toFixed(0)}% opt-out), ${skipped} type-equiv/cordis-catalog (checked elsewhere).`) + console.log(`doc-typecheck: ${checked.length} block(s) compiled, ${ignored.length} ignored (${(ratio * 100).toFixed(0)}% opt-out), ${skipped} type-equiv/catalog (checked elsewhere).`) // Guard against the escape hatch becoming the norm. if (ratioDenominator >= 4 && ratio > 0.5) { console.error(`doc-typecheck: too many blocks opt out of checking (${ignored.length}/${ratioDenominator}). Make them compile or delete them.`) diff --git a/scripts/gen-persistence-catalog.ts b/scripts/gen-persistence-catalog.ts new file mode 100644 index 0000000000..6d1988beb7 --- /dev/null +++ b/scripts/gen-persistence-catalog.ts @@ -0,0 +1,456 @@ +/** + * Generate (and verify) the persistence log event catalog in + * docs/persistence-catalog/log-events.md. + * + * The catalog is the ON-DISK-vocabulary reference: every event type that can + * appear in a session's durable event log — every member of the + * merge-extensible `SessionEventMap`, across the owning declaration in + * `@deepseek-ai/dsh-session` and every plugin declaration merge. It complements + * the cordis events/services catalog (the live bus wiring — a log event is NOT + * a cordis event; it reaches listeners via the single `session/event` emit) and + * the core-data-structures session page (the `SessionEvent` envelope and + * derivation semantics): this page is the RECORDS a persisted log can contain. + * + * `tsx scripts/gen-persistence-catalog.ts` → write the catalog + * `tsx scripts/gen-persistence-catalog.ts --check` → exit 1 if the committed + * file is stale (CI / + * pre-push gate) + * + * Like its AST sibling `gen-cordis-catalog.ts` (and unlike the boot-based + * `gen-tool-catalog.ts`), this is a pure source pass: every log event is a + * string-literal-named property with a static type annotation, so the AST is + * the whole truth and a brand-new event (core or merged) appears in the next + * regenerate — an un-regenerated file fails `--check`. The walk enforces JSDoc + * COMPLETENESS on the whole vocabulary: every member carries description prose + * (it becomes the catalog entry), and an `@mode` tag on a member is a hard + * error — dispatch modes belong to cordis bus events, and a log event has none + * (see docs/rfc/implemented/process/2026-07-04-persistence-log-catalog.md). + * Structural holes are hard errors for the same reason: a member that is not a + * property signature with an explicit payload type, an `extends` clause on a + * declaration, a top-level `interface SessionEventMap` that is not the single + * exported declaration in the owning package, and a duplicate declaration of + * one event would each let something join (or impersonate) + * `keyof SessionEventMap` without a truthful catalog row. Violations aggregate + * into ONE error listing every offender. + * + * The surface/log-only badge is parsed from the `SurfaceEventType` union in the + * owning package (never hand-listed here), and every union member must name a + * collected event — a stale union member is a hard error. + * + * Payload fences use the ` ```ts persistence-catalog ` info string: + * doc-typecheck recognizes it and skips compilation (a bare payload fragment is + * not standalone-compilable), excluded from the opt-out ratio. + */ + +import { globSync, readFileSync, writeFileSync } from 'node:fs' +import { resolve } from 'node:path' +import ts from 'typescript' + +const root = resolve(import.meta.dirname, '..') +const OUT = 'docs/persistence-catalog/log-events.md' + +/** The fenced-block info string for generated payload blocks (skipped by + * doc-typecheck, since a bare payload fragment is not standalone-compilable). */ +const FENCE = 'ts persistence-catalog' + +/** The package whose module id plugin merges augment (`declare module '…'`). */ +const SESSION_MODULE = '@deepseek-ai/dsh-session' + +/** + * Cross-link map: a type name that appears in a payload → the + * core-data-structures page that documents it (path relative to OUT's folder). + * Hand-curated and catalog-owned, same policy as the cordis catalog's map: each + * name resolves to exactly one PRIMARY page. A payload type with no + * core-data-structures home (e.g. `HookDialect`, documented in its package) + * simply gets no link. + */ +const LINK_MAP: Record = { + CallId: 'core.md', + ContentBlock: 'core.md', + MessageSource: 'core.md', + StreamChunk: 'llm-streaming.md', + TokenUsage: 'llm-streaming.md', + TodoItem: 'session.md', + TurnTrigger: 'session.md', + TurnEndReason: 'session.md', +} + +/** One log event, extracted from a `SessionEventMap` declaration. */ +export interface LogEventEntry { + /** Scoped name, e.g. `turn/start`. */ + name: string + /** The scope prefix, e.g. `turn` (everything before the first `/`). */ + scope: string + /** Payload type text (the member's type annotation, whitespace-collapsed). */ + payload: string + /** Description prose (the member's JSDoc), one line per paragraph. */ + doc: string + /** Source pointer `packages/…/file.ts:line` of the declaration. */ + source: string +} + +/** A {@link LogEventEntry} plus its surface-eligibility badge. */ +export interface AnnotatedLogEventEntry extends LogEventEntry { + /** Whether the type is a `SurfaceEventType` member (may carry `surfaceOp`). */ + surface: boolean +} + +/** Repo-relative source pointer `file:line` for a node's first character. */ +function pointer(rel: string, sf: ts.SourceFile, node: ts.Node): string { + const { line } = sf.getLineAndCharacterOfPosition(node.getStart(sf)) + return `${rel}:${line + 1}` +} + +const printer = ts.createPrinter({ removeComments: true }) + +/** + * One-line payload text for a member's type annotation. Printed through the + * TypeScript printer (not sliced from source text): the printer emits `;` + * member separators regardless of how the source separated them, so a + * multi-line newline-separated type literal still collapses to a VALID + * single-line fragment. The trailing `;` the printer puts before every `}` is + * dropped to match the repo's inline-literal style. + */ +function payloadText(type: ts.TypeNode, sf: ts.SourceFile): string { + return printer.printNode(ts.EmitHint.Unspecified, type, sf) + .replace(/\s+/g, ' ') + .replace(/;\s*\}/g, ' }') + .trim() +} + +/** The raw `/** … *​/` JSDoc block immediately preceding a node, or '' if none. */ +function rawJsDoc(text: string, node: ts.Node): string { + const ranges = ts.getLeadingCommentRanges(text, node.getFullStart()) ?? [] + const jsdoc = ranges.filter(r => text.slice(r.pos, r.pos + 3) === '/**').at(-1) + return jsdoc ? text.slice(jsdoc.pos, jsdoc.end) : '' +} + +/** + * Parse a raw JSDoc block into description prose, flagging whether any `@mode` + * tag is present (forbidden on log events). Output obeys the repo's markdown + * conventions so the generated file passes verify-md-wrap: each prose paragraph + * collapses to ONE physical line, and a `-` bullet list is preserved with each + * item on its own single line (continuation lines folded in). `{@link Foo}` + * unwraps to `Foo`. Description prose ends at the FIRST block tag (standard + * JSDoc semantics): tag lines and their continuation lines are never prose. + */ +function parseJsDoc(raw: string): { doc: string; hasMode: boolean } { + const inner = raw + .replace(/^\/\*\*/, '') + .replace(/\*\/$/, '') + .split('\n') + .map(l => l.replace(/^\s*\*?\s?/, '').replace(/\s+$/, '')) + let hasMode = false + let inTags = false + const blocks: string[] = [] + let para: string[] = [] + let list: string[] = [] + let item: string[] = [] + const join = (parts: string[]): string => parts.join(' ').replace(/\s+/g, ' ').trim() + const flushItem = (): void => { + if (item.length) list.push(join(item)) + item = [] + } + const flushList = (): void => { + flushItem() + if (list.length) blocks.push(list.join('\n')) // one block, items on own lines + list = [] + } + const flushPara = (): void => { + flushList() + if (para.length) blocks.push(join(para)) + para = [] + } + for (const line of inner) { + // Tag detection runs on the trimmed line: the normalization above strips at + // most one post-`*` space, so an extra-indented `* @mode` still reaches + // here with leading whitespace and must not leak into prose. + const tagLine = line.trimStart() + if (/^@mode\b/.test(tagLine)) { hasMode = true; flushPara(); inTags = true; continue } + if (tagLine.startsWith('@')) { flushPara(); inTags = true; continue } + if (inTags) continue // block-tag territory: continuations are never prose + if (line.trim() === '') { flushPara(); continue } + if (/^-\s+/.test(line)) { + // A list item starts: a pending paragraph (e.g. an intro line directly + // above the list, no blank between) flushes FIRST so it renders above. + flushItem() + if (para.length) { blocks.push(join(para)); para = [] } + item.push(line) + continue + } + if (item.length) { item.push(line); continue } // continuation of current item + para.push(line) + } + flushPara() + const doc = blocks.join('\n\n').replace(/\{@link\s+([^}]+)\}/g, '$1').trim() + return { doc, hasMode } +} + +/** + * Throw one aggregate error for every completeness violation a walk collected. + * Aggregation is deliberate: a remediation pass sees the whole list at once + * instead of replaying the gate once per offender. + */ +function reportViolations(violations: string[]): void { + if (violations.length === 0) return + throw new Error( + `gen-persistence-catalog: ${violations.length} JSDoc completeness violation(s):\n` + + violations.map(v => ` ${v}`).join('\n'), + ) +} + +/** + * Every `interface SessionEventMap` declaration in a source file: the owning + * top-level declaration (in `@deepseek-ai/dsh-session`) and any declaration + * merge inside a `declare module '@deepseek-ai/dsh-session'` block. Both forms + * declare members of the SAME merged interface, so both are catalogued + * uniformly. `topLevel` distinguishes the owning form so the caller can verify + * it actually lives in the owning package — an unrelated local interface that + * happens to share the name must not be catalogued as the on-disk vocabulary. + */ +function sessionEventMapDecls(sf: ts.SourceFile): { decl: ts.InterfaceDeclaration; topLevel: boolean }[] { + const decls: { decl: ts.InterfaceDeclaration; topLevel: boolean }[] = [] + for (const stmt of sf.statements) { + if (ts.isInterfaceDeclaration(stmt) && stmt.name.text === 'SessionEventMap') decls.push({ decl: stmt, topLevel: true }) + if (ts.isModuleDeclaration(stmt) && ts.isStringLiteral(stmt.name) && stmt.name.text === SESSION_MODULE + && stmt.body && ts.isModuleBlock(stmt.body)) { + for (const inner of stmt.body.statements) { + if (ts.isInterfaceDeclaration(inner) && inner.name.text === 'SessionEventMap') decls.push({ decl: inner, topLevel: false }) + } + } + } + return decls +} + +/** + * The npm package name owning a `packages///…` source file, read + * from that package's manifest — or null when the manifest is missing or + * unparseable (the caller treats null as "ownership unverifiable"). + */ +function packageNameFor(rel: string, scanRoot: string): string | null { + const dir = rel.split('/').slice(0, 3).join('/') + try { + const manifest = JSON.parse(readFileSync(resolve(scanRoot, dir, 'package.json'), 'utf8')) as { name?: string } + return typeof manifest.name === 'string' ? manifest.name : null + } catch { + // Missing or malformed package.json — every real workspace package has one, + // so this only arises in stripped-down fixture trees; either way ownership + // cannot be verified and the caller reports the declaration. + return null + } +} + +/** + * Walk every `SessionEventMap` declaration (the owning interface plus every + * plugin declaration merge) and extract its events, hard-erroring (aggregated) + * on any completeness violation: a member without description prose, an + * `@mode` tag (a category error — log events have no dispatch mode), a member + * that is not a property signature with an explicit payload type, a + * non-literal member name, an `extends` clause (inherited keys would join + * `keyof SessionEventMap` without a catalog row), a top-level declaration that + * is not the single exported one in the owning package, or the same event + * declared twice. + * `scanRoot` defaults to the repo root; tests pass a fixture dir. + */ +export function collectLogEvents(scanRoot: string = root): LogEventEntry[] { + const entries: LogEventEntry[] = [] + const violations: string[] = [] + const seen = new Map() + let owningDecl: string | null = null + for (const rel of globSync('packages/*/*/src/**/*.ts', { cwd: scanRoot }).sort()) { + const abs = resolve(scanRoot, rel) + const text = readFileSync(abs, 'utf8') + if (!text.includes('SessionEventMap')) continue + const sf = ts.createSourceFile(abs, text, ts.ScriptTarget.Latest, true) + for (const { decl, topLevel } of sessionEventMapDecls(sf)) { + const declSrc = pointer(rel, sf, decl) + if (topLevel) { + // The top-level form is the OWNING vocabulary, and it has exactly one + // home: the single EXPORTED declaration in the owning package. A + // same-named interface anywhere else — another package, a non-exported + // local, a second exported copy — is a different type that must not be + // catalogued as on-disk events. + const pkg = packageNameFor(rel, scanRoot) + if (pkg !== SESSION_MODULE) { + violations.push(`top-level interface SessionEventMap (${declSrc}) is outside ${SESSION_MODULE} (package ${pkg ?? 'unknown'}). Rename the interface, or contribute events via declare module '${SESSION_MODULE}'.`) + continue + } + const exported = decl.modifiers?.some(m => m.kind === ts.SyntaxKind.ExportKeyword) ?? false + if (!exported) { + violations.push(`top-level interface SessionEventMap (${declSrc}) is not exported; the owning vocabulary is the single exported declaration — rename a local helper interface.`) + continue + } + if (owningDecl) { + violations.push(`top-level interface SessionEventMap (${declSrc}) is already declared at ${owningDecl}; the owning vocabulary has exactly one home.`) + continue + } + owningDecl = declSrc + } + if (decl.heritageClauses?.length) { + violations.push(`SessionEventMap declaration (${declSrc}) uses extends; inherited keys would join keyof SessionEventMap without a catalog row — declare event members directly.`) + } + for (const member of decl.members) { + const src = pointer(rel, sf, member) + if (!ts.isPropertySignature(member) || !member.type) { + // A method-form or type-less member still joins `keyof SessionEventMap`, + // so skipping it silently would be exactly the undocumented-event hole + // this catalog exists to close. + const label = (member as { name?: ts.Node }).name?.getText(sf) ?? member.getText(sf).replace(/\s+/g, ' ') + violations.push(`SessionEventMap member ${label} (${src}) is not a property signature with an explicit payload type; declare every log event as 'scope/name': .`) + continue + } + if (!ts.isStringLiteral(member.name)) { + violations.push(`log event at ${src} has a non-literal name; the catalog needs string-literal event names.`) + continue + } + const name = member.name.text + const where = `log event '${name}' (${src})` + const prior = seen.get(name) + if (prior) { + violations.push(`${where} is already declared at ${prior}; an event type has exactly one declaration.`) + continue + } + seen.set(name, src) + const payload = payloadText(member.type, sf) + const { doc, hasMode } = parseJsDoc(rawJsDoc(text, member)) + if (hasMode) { + violations.push(`${where} carries an @mode tag, but a log event has no dispatch mode (it is not a cordis bus event — it rides the 'session/event' emit). Remove the tag.`) + } + if (!doc) { + violations.push(`${where} has no description prose. Say what the event records and what its payload means — the JSDoc becomes the catalog entry.`) + } + entries.push({ name, scope: name.split('/')[0] ?? name, payload, doc, source: src }) + } + } + } + reportViolations(violations) + return entries +} + +/** + * Parse the `SurfaceEventType` union — the surface-eligible subset of event + * types — from source. Hard-errors when the alias is missing, declared more + * than once, or contains a non-string-literal member: the badge derivation + * relies on the union being a closed set of literal event names. + * `scanRoot` defaults to the repo root; tests pass a fixture dir. + */ +export function collectSurfaceEventTypes(scanRoot: string = root): string[] { + const found: { names: string[]; source: string }[] = [] + for (const rel of globSync('packages/*/*/src/**/*.ts', { cwd: scanRoot }).sort()) { + const abs = resolve(scanRoot, rel) + const text = readFileSync(abs, 'utf8') + if (!text.includes('SurfaceEventType')) continue + const sf = ts.createSourceFile(abs, text, ts.ScriptTarget.Latest, true) + for (const stmt of sf.statements) { + if (!ts.isTypeAliasDeclaration(stmt) || stmt.name.text !== 'SurfaceEventType') continue + const src = pointer(rel, sf, stmt) + const members = ts.isUnionTypeNode(stmt.type) ? [...stmt.type.types] : [stmt.type] + const names: string[] = [] + for (const m of members) { + if (ts.isLiteralTypeNode(m) && ts.isStringLiteral(m.literal)) names.push(m.literal.text) + else throw new Error(`gen-persistence-catalog: SurfaceEventType (${src}) has a non-string-literal member; the badge derivation needs a closed literal union.`) + } + found.push({ names, source: src }) + } + } + const only = found[0] + if (!only) throw new Error('gen-persistence-catalog: no SurfaceEventType union found under packages/*/*/src.') + if (found.length > 1) throw new Error(`gen-persistence-catalog: SurfaceEventType is declared more than once (${found.map(f => f.source).join(', ')}); the surface subset has exactly one owner.`) + return only.names +} + +/** + * Attach the surface/log-only badge to each event. Hard-errors when a + * `SurfaceEventType` union member names no collected event — a stale union + * member would otherwise silently badge nothing. + */ +export function annotateSurface(events: LogEventEntry[], surfaceTypes: string[]): AnnotatedLogEventEntry[] { + const names = new Set(events.map(e => e.name)) + const stale = surfaceTypes.filter(t => !names.has(t)) + if (stale.length > 0) { + throw new Error(`gen-persistence-catalog: SurfaceEventType member(s) ${stale.map(t => `'${t}'`).join(', ')} name no declared log event (stale union member?).`) + } + const surface = new Set(surfaceTypes) + return events.map(e => ({ ...e, surface: surface.has(e.name) })) +} + +/** Render the cross-link "Types:" line for a payload, or '' if none apply. */ +function typeLinks(payload: string): string { + const seen = new Set() + for (const name of Object.keys(LINK_MAP)) { + if (new RegExp(`\\b${name}\\b`).test(payload)) seen.add(name) + } + if (seen.size === 0) return '' + const links = [...seen].sort().map(n => `[${n}](../core-data-structures/${LINK_MAP[n]})`) + return `Types: ${links.join(' · ')}` +} + +/** Render one log event entry. */ +function renderEvent(e: AnnotatedLogEventEntry): string[] { + const out = [`#### \`${e.name}\` — ${e.surface ? 'surface' : 'log-only'}`, ''] + if (e.doc) out.push(e.doc, '') + out.push('```' + FENCE, `'${e.name}': ${e.payload}`, '```', '') + const links = typeLinks(e.payload) + if (links) out.push(links, '') + out.push(`Source: [\`${e.source}\`](../../${e.source.split(':')[0]})`, '') + return out +} + +/** Render the full catalog (pure, deterministic given the collected inputs). */ +export function render(events: AnnotatedLogEventEntry[]): string { + const lines: string[] = [ + '', + '', + '# Persistence Log Event Catalog', + '', + 'Every event type that can appear in a session\'s durable event log: each member of the merge-extensible `SessionEventMap` — the owning vocabulary in `@deepseek-ai/dsh-session` plus every plugin declaration merge in this repo — with the payload it carries, its surface badge, and the declaration it comes from. It complements [session.md](../core-data-structures/session.md) (the `SessionEvent` envelope, surface list, and `deriveMessages()` projection), [persistence.md](../core-data-structures/persistence.md) (how the log is made durable), and the [cordis catalog](../cordis-catalog/events-and-services.md) (the live bus wiring — a log event is NOT a cordis event; it reaches listeners via the single `session/event` emit).', + '', + 'This file is GENERATED from source (`scripts/gen-persistence-catalog.ts`) and verified fresh by `pnpm run verify-persistence-catalog` (part of `doc-sync`) — do not edit it by hand. Payload blocks use a `ts persistence-catalog` fence (skipped by doc-typecheck, since a bare payload fragment is not standalone-compilable). Type names in a payload link to the page that documents them. See [the persistence-log-catalog RFC](../rfc/implemented/process/2026-07-04-persistence-log-catalog.md).', + '', + 'The on-disk envelope around every payload is `SessionEvent` — `type`, monotonic `seq`, epoch-ms `time`, the `data` documented here, plus `surfaceOp`/`sourceEventSeqs` on **surface** events only ([envelope](../core-data-structures/session.md#sessioneventt--one-log-entry)). **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`), and the whole format is pinned at `SESSION_FORMAT_VERSION = 0` — pre-release, no compatibility implied ([the version stance](../core-data-structures/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further event types, which are outside this catalog by construction.', + '', + '## Events', + '', + ] + const scopes = [...new Set(events.map(e => e.scope))].sort() + for (const scope of scopes) { + lines.push(`### \`${scope}/*\``, '') + for (const e of events.filter(x => x.scope === scope).sort((a, b) => a.name.localeCompare(b.name))) { + lines.push(...renderEvent(e)) + } + } + return lines.join('\n') +} + +/** CLI entry: default writes the catalog, `--check` fails if the committed copy + * is stale. Guarded behind an entry-point check so importing this module for + * tests neither regenerates the committed file nor calls process.exit. */ +function main(): void { + const content = render(annotateSurface(collectLogEvents(), collectSurfaceEventTypes())) + if (process.argv.includes('--check')) { + let committed: string | null = null + try { + committed = readFileSync(resolve(root, OUT), 'utf8') + } catch { + // Only ENOENT (not yet generated) is expected; a present-but-unreadable + // file is not a state this repo produces. Either way the remedy is the + // same — regenerate — so treat a read failure as "stale". + committed = null + } + if (committed === content) { + console.log(`gen-persistence-catalog: ${OUT} is up to date.`) + process.exit(0) + } + console.error(`gen-persistence-catalog: ${OUT} is stale. Run \`pnpm run gen-persistence-catalog\` and commit ${OUT}.`) + process.exit(1) + } + + writeFileSync(resolve(root, OUT), content) + console.log(`gen-persistence-catalog: wrote ${OUT}.`) +} + +// Run only when invoked as a script, not when imported by a test. +if (process.argv[1] && import.meta.filename === resolve(process.argv[1])) { + main() +} diff --git a/scripts/gen-tool-catalog.ts b/scripts/gen-tool-catalog.ts index e42754bf53..1176002a45 100644 --- a/scripts/gen-tool-catalog.ts +++ b/scripts/gen-tool-catalog.ts @@ -254,7 +254,6 @@ export async function collectToolCatalog(packages: ToolPackage[] = TOOL_PACKAGES function renderTool(schema: ToolSchema, source: string): string[] { const out = [`### \`${schema.name}\``, ''] if (schema.description) out.push(schema.description, '') - if (schema.strict !== undefined) out.push(`Strict: \`${String(schema.strict)}\``, '') out.push('```json', JSON.stringify(schema.parameters, null, 2), '```', '') out.push(`Source: [\`${source}\`](../../${source})`, '') return out diff --git a/scripts/translation-pairing.manifest.json b/scripts/translation-pairing.manifest.json index 8c2708d48d..c9a88aa4e0 100644 --- a/scripts/translation-pairing.manifest.json +++ b/scripts/translation-pairing.manifest.json @@ -11,6 +11,7 @@ "docs/module-graph.md", "docs/cordis-catalog/", "docs/tool-catalog/", + "docs/persistence-catalog/", "docs/i18n/terminology.md" ] } diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 0aca732e03..b1c3163782 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -18,6 +18,7 @@ { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "StreamChunk", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "TokenUsage", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "ContentBlockMap", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "AppIdentity", "source": "packages/llm/llm/src/attribution.ts" }, { "doc": "docs/core-data-structures/session.md", "symbol": "SessionEventMap", "source": "packages/core/session/src/types.ts" }, { "doc": "docs/core-data-structures/session.md", "symbol": "TodoItem", "source": "packages/core/session/src/types.ts" }, @@ -76,7 +77,6 @@ { "doc": "docs/core-data-structures/web.md", "symbol": "WebFetchRequest", "source": "packages/web/web/src/types.ts" }, { "doc": "docs/core-data-structures/web.md", "symbol": "WebFetchResult", "source": "packages/web/web/src/types.ts" }, { "doc": "docs/core-data-structures/web.md", "symbol": "WebFetchBody", "source": "packages/web/web/src/types.ts" }, - { "doc": "docs/core-data-structures/web.md", "symbol": "WebProviderStatus", "source": "packages/web/web/src/types.ts" }, - { "doc": "docs/core-data-structures/web.md", "symbol": "WebCapabilityStatus", "source": "packages/web/web/src/types.ts" } + { "doc": "docs/core-data-structures/web.md", "symbol": "WebProviderStatus", "source": "packages/web/web/src/types.ts" } ] } diff --git a/tsconfig.build.json b/tsconfig.build.json index 477c68d31e..b6ba7901f2 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -41,6 +41,7 @@ { "path": "./packages/support/invariants" }, { "path": "./packages/ui/acp" }, { "path": "./packages/ui/acp-agent" }, + { "path": "./packages/ui/app-boot" }, { "path": "./packages/ui/stdio-agent" }, { "path": "./packages/support/llm-replay" }, { "path": "./packages/subagent/subagent" }, diff --git a/tsconfig.json b/tsconfig.json index 7d1209b42e..9cd7aa8a6d 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -52,6 +52,7 @@ { "path": "./packages/support/invariants" }, { "path": "./packages/ui/acp" }, { "path": "./packages/ui/acp-agent" }, + { "path": "./packages/ui/app-boot" }, { "path": "./packages/ui/stdio-agent" }, { "path": "./packages/support/llm-replay" }, { "path": "./packages/subagent/subagent" },