From 11773170f09d0e2261a326cc611b058687b05498 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 21 Jun 2026 00:36:27 +0800 Subject: [PATCH 01/15] docs: propose unstable LLM API recovery RFC --- docs/rfc/README.md | 1 + .../2026-06-21-unstable-llm-api-recovery.md | 154 ++++++++++++++++++ 2 files changed, 155 insertions(+) create mode 100644 docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md diff --git a/docs/rfc/README.md b/docs/rfc/README.md index 3778aaf4ad..64b5743433 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -65,6 +65,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Extract a generic long-running tool runtime](proposed/architecture/2026-06-20-generic-long-running-tool-runtime.md) | 2026-06-20 | | [Extract example apps into packages](proposed/architecture/2026-06-20-extract-example-app-packages.md) | 2026-06-20 | | [Branded IDs everywhere they belong](proposed/architecture/2026-06-20-branded-ids.md) | 2026-06-20 | +| [Treat unstable LLM APIs as a first-class failure mode](proposed/architecture/2026-06-21-unstable-llm-api-recovery.md) | 2026-06-21 | ### Process diff --git a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md new file mode 100644 index 0000000000..8a498e2e19 --- /dev/null +++ b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md @@ -0,0 +1,154 @@ +# RFC: Treat unstable LLM APIs as a first-class failure mode + +Status: proposed + +## Problem + +LLM APIs are not a stable local function call. They rate-limit, overload, return 5xx/502/503 from gateways, close streaming sockets before `[DONE]`, emit malformed or provider-specific error payloads, hang mid-stream, surface SDK errors as in-band events, and sometimes require a delayed retry using `Retry-After`. The harness currently contains good error containment, but it does not yet treat this API instability as a first-class design problem. + +`dsh-llm` defines two sanctioned adapter failure paths - throw from `stream()` or end with `finish { kind: 'error' | 'aborted' }` - and the agent loop translates both into a failed step instead of logging a fake completed assistant message. That was the right MVP containment baseline, documented in [the architecture](../../../architecture.md) and reinforced by [the twin-adapter RFC](../../implemented/architecture/2026-06-13-twin-llm-adapters.md). It is not enough for an agent that depends on an unstable remote model API for every turn. + +The audit found four load-bearing gaps. + +- `LlmError` and `FinishReasonMap.error` carry only `message`, `code`, and sometimes HTTP `status` ([packages/llm/llm/src/index.ts](../../../../packages/llm/llm/src/index.ts), [packages/llm/llm/src/types.ts](../../../../packages/llm/llm/src/types.ts)). A caller cannot reliably tell "retry this after 800 ms on the same endpoint", "fail over to another route for the same model", "ask the user for credentials", "never retry because the request is invalid", "provider truncated the stream after committed output", or "adapter protocol bug" without provider-specific heuristics. +- The `llm/stream` waterfall is documented as the place for retry/routing/caching, but its value is one committed `AsyncIterable`. A listener can technically catch an API error and call `next()` again, but once it has yielded any chunks to the agent loop those chunks are already appended as `assistant/chunk` events and emitted to UI. Retrying after that point would concatenate chunks from two provider attempts into one model step, corrupting replay and the user transcript. +- The adapter registry is one adapter per model name. That makes "logical model" and "concrete API route" the same thing, so the service has no vocabulary for "same model through another endpoint or SDK", "same provider region with different health", "fallback route with compatible capabilities", provider request ids, or per-route backoff state. +- Crash recovery and LLM API recovery are easy to conflate. `dsh-session` repairs an interrupted durable log by closing an open turn and synthesizing missing tool results ([packages/core/session/src/repair.ts](../../../../packages/core/session/src/repair.ts)); that preserves already-written work after a process crash. It does not make a failed provider attempt safe to retry, discard, replay, or fail over. + +The result is a system that can survive an unstable LLM API without killing the loop, but cannot make principled recovery decisions. For a coding agent, that is underdesigned: ordinary provider turbulence should not require every UI or product plugin to reinvent retries around an unsafe stream boundary. + +## Proposal + +Introduce an LLM-call v2 contract centered on API-instability recovery: classify provider/API failures, separate provider attempts from committed model output, route logical models through recoverable API routes, and make conservative retry/failover the default behavior in `dsh-llm`. Because the harness is unreleased, this should be a breaking cleanup rather than a compatibility layer around the underspecified v1 surface. + +### 1. Replace flat error codes with a serializable `LlmFailure` + +Keep `HarnessError` as the common thrown-error base, but make LLM failures carry a structured, JSON-serializable payload. `code` remains a stable leaf label for logs and provider-specific matching; retry/failover policy branches on the structured fields. + +```ts ignore-check +type LlmFailureClass = + | 'auth' + | 'rate-limit' + | 'quota' + | 'invalid-request' + | 'unsupported' + | 'timeout' + | 'transport' + | 'provider-overloaded' + | 'provider-unavailable' + | 'provider-bug' + | 'protocol' + | 'safety' + | 'aborted' + | 'unknown' + +type LlmFailurePhase = + | 'request-build' + | 'connect' + | 'response-headers' + | 'stream' + | 'finish' + +interface LlmFailure { + message: string + code: string + class: LlmFailureClass + phase: LlmFailurePhase + retryable: boolean + failover: 'never' | 'same-model' | 'compatible-model' + partialOutput: 'none' | 'uncommitted' | 'committed' + provider?: string + routeId?: string + model?: string + wireModel?: string + status?: number + retryAfterMs?: number + requestId?: string +} +``` + +`LlmError` should carry `failure: LlmFailure`; `FinishReasonMap.error` should carry the same payload instead of a parallel `{ message, code? }` shape. The agent loop should persist the serializable failure fields in `session error` and `turn/end { kind: 'error' }`, while the thrown `LlmError` may still carry a non-serializable `cause` chain for local debugging. + +Adapters are responsible for faithful provider/API classification at their boundary: HTTP status, `Retry-After`, provider request id headers, SDK error type, timeout vs caller abort, malformed SSE, missing `[DONE]`, unknown finish reason, and unsupported local request shape. The current pi-ai adapter's regex over message text is acceptable only as a temporary fallback when the SDK hides the real status; the adapter should prefer structured SDK/provider fields when available. + +### 2. Split provider attempts from committed model output + +Make "attempt" a first-class boundary below `ctx.llm.stream()`. An adapter streams one provider API attempt. The LLM service runs zero or more attempts according to recovery policy and yields only committed output to the agent loop. + +The important invariant: chunks from a failed attempt must never be silently spliced together with chunks from a later attempt as one assistant step. Recovery must choose one of these paths instead: + +- **Retry before commit.** If an API attempt fails before any chunks are committed to the loop, the service may retry or fail over and hide the failed attempt from `assistant/chunk` history, while recording attempt diagnostics separately. +- **Commit and stop retrying.** Once chunks are committed to the loop, the attempt owns the visible step. If it later fails, the step fails with `partialOutput: 'committed'`; automatic retry is not allowed unless a later RFC designs an explicit continuation/repair protocol. +- **Buffered recovery mode.** A caller or policy may choose to buffer an entire attempt until it reaches `finish`, then yield the winning attempt's chunks. This improves retryability against flaky APIs at the cost of live token streaming and should be a deliberate mode, not an accidental side effect. + +This likely means replacing the single overloaded `llm/stream` waterfall with narrower hooks: one around a single provider attempt, one around recovery policy decisions, and one around the committed stream. Names are implementation details for the follow-up PR, but the semantics are not: plugins must be able to wrap "one API attempt" without pretending they can safely retry already-committed chunks. + +### 3. Route logical models through recoverable API routes + +Separate the logical model a caller requests from the concrete provider route that serves an attempt. Replace "one adapter per model name" with route registration, for example: + +```ts ignore-check +ctx.llm.registerRoute({ + routeId: 'deepseek-direct:deepseek-v4-flash', + model: 'deepseek-v4-flash', + wireModel: 'deepseek-v4-flash', + provider: 'deepseek', + adapter, + priority: 0, + capabilities: { tools: true, reasoning: true, images: false, prefill: false }, +}) +``` + +`GenerateOptions.model` remains the logical model. The service resolves it to a route for each API attempt, records the route in failure/attempt diagnostics, and can retry on the same route or fail over to another route with compatible capabilities. Duplicate model names become normal; duplicate route ids are the conflict. This is the smallest vocabulary that can express direct endpoint vs SDK-backed endpoint, regional endpoints, and future fallback models without making every caller own routing. + +### 4. Put default API recovery policy in `dsh-llm` + +Adapters should not perform hidden SDK retries unless those retries are surfaced as attempts with classified failures. The service owns the default policy so every consumer gets the same behavior and the same audit trail. + +Default policy should be conservative: + +- Retry transient API failures (`rate-limit`, `timeout`, `transport`, `provider-overloaded`, `provider-unavailable`) only before committed output. +- Honor `retryAfterMs`, otherwise use bounded exponential backoff with jitter. +- Treat 429/408/409/425/500/502/503/504 and connection resets as potentially recoverable unless the provider payload says otherwise; treat 400/401/403, unsupported local options, caller abort, and adapter protocol bugs as non-retryable. +- Fail over only when the failure says failover is safe and the candidate route advertises compatible capabilities for the request (`tools`, reasoning passback, images, prefill, stop sequences, strict tools). +- Share the caller's `AbortSignal` across the whole recovered call, and expose per-attempt timeouts as explicit policy. A stuck stream must time out in a controlled way instead of hanging the turn forever. +- Bound attempts by count and elapsed time, with clear failure reporting when the budget is exhausted. + +The policy should be configurable through a typed service option and an event/waterfall seam so product plugins can tighten or loosen it, but the default must be safe enough that a basic agent does not need a custom retry plugin to survive ordinary 429/5xx/connectivity noise. + +### 5. Record API attempt diagnostics without polluting derived history + +The session log should be able to explain what happened during a recovered model call without feeding failed attempts back to the model as assistant output. Add turn-enclosed, derive-skipped diagnostics for LLM API attempts, or an equivalent trace surface if we decide session events should stay conversation-only. The data must be JSON-serializable and include attempt number, route id, failure payload, backoff, provider request id, and whether any chunks were committed. + +`assistant/chunk` remains the replay source for the committed attempt only. Hidden failed attempts are diagnostics, not model history. A stream that fails after committed chunks remains replayable as a thrown stream through the existing `llm-replay` sidecar mechanism, but the failure payload should become structured rather than `{ message, code, status? }`. + +## Out of scope + +This RFC does not propose silent mid-stream continuation after user-visible output. That requires a separate model-history design: either provider-supported prefill/continuation, a recovery prompt that explicitly shows the partial assistant output, or a UI affordance that marks the partial answer as failed and asks the model to continue in a new step. Splicing two API attempts into one assistant message is rejected. + +This RFC also does not solve semantic model-output repair: malformed tool-call JSON, refusal handling, or content-filter fallbacks. Those may use the same failure vocabulary later, but they are higher-level agent behaviors, not unstable-API recovery. + +## Acceptance criteria + +- `LlmError` and in-band finish errors carry one structured `LlmFailure` payload; the agent loop persists that payload's serializable fields in error turn data. +- Recovery policy can distinguish retry, failover, credential/user-action, unsupported request, caller abort, adapter/protocol bug, and post-commit partial stream failure without parsing message text. +- The LLM service has an explicit API-attempt boundary; no retry path can append chunks from two provider attempts as one committed assistant step. +- The route registry allows multiple concrete API routes for one logical model and records the selected route on attempts/failures. +- Default recovery retries transient pre-commit failures with bounded backoff, honors provider retry-after hints, times out stuck streams, disables hidden SDK retries or surfaces them as attempts, and never retries after committed chunks without an explicit continuation design. +- Unit tests cover thrown errors and finish-error chunks through the real agent loop, retry-before-first-chunk, failover to a second compatible route, retry budget exhaustion, abort during backoff, stream timeout, and the "partial chunks then failure does not retry/splice" invariant. +- Adapter tests classify representative HTTP statuses, retry-after headers, request ids, malformed/truncated SSE streams, SDK in-stream errors, caller aborts, and unsupported options into `LlmFailure`. +- Snapshot/replay support can faithfully represent a recovered call and a post-commit thrown stream without losing the structured failure payload. +- Docs updated in the same change: [the architecture LLM section](../../../architecture.md), [the LLM adapter cookbook](../../../cookbook/adding-an-llm-adapter.md), and the LLM package READMEs. + +## Risks / what we give up + +- **More surface area in the LLM core.** API recovery adds policy, route state, attempt diagnostics, and tests. That complexity belongs in `dsh-llm` because every consumer otherwise reinvents it around the same unsafe stream boundary. +- **Some modes reduce live streaming.** Buffered recovery trades first-token latency for safe retry. That should be opt-in or policy-driven; the default can still stream eagerly while retrying only before commit. +- **Breaking adapter churn.** Existing adapters will change from "stream chunks or throw a flat `LlmError`" to "stream one classified API attempt." Pre-release rules favor the correct seam over shims. +- **Route compatibility is easy to overclaim.** A route must advertise concrete capabilities, and failover must check the request actually fits them. "Same model name" is not enough when one route lacks strict tools, reasoning passback, images, stop sequences, or prefill. + +## Related + +- Builds on [Provider-neutral content-block vocabulary](../../implemented/architecture/2026-06-11-content-block-vocabulary.md): the content vocabulary stays provider-neutral; this adds a provider-neutral failure/recovery vocabulary beside it. +- Revises the scope implied by [Two LLM adapters as a design-verification twin](../../implemented/architecture/2026-06-13-twin-llm-adapters.md): the twin validated chunk shape and error delivery paths, but it also exposed that delivery paths are not enough for unstable API recovery. +- Extends [Structured error taxonomy](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md): `HarnessError.code` was the foundation; LLM API recovery needs a richer payload because retry/failover policy cannot safely branch on one flat string. From e141ffa27f9c10e8a5157b81805fdd4236991904 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 21 Jun 2026 10:02:35 +0800 Subject: [PATCH 02/15] docs: tighten LLM recovery RFC scope --- .../2026-06-21-unstable-llm-api-recovery.md | 123 ++++++++++++------ 1 file changed, 86 insertions(+), 37 deletions(-) diff --git a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md index 8a498e2e19..f5f7659c41 100644 --- a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md +++ b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md @@ -6,26 +6,25 @@ Status: proposed LLM APIs are not a stable local function call. They rate-limit, overload, return 5xx/502/503 from gateways, close streaming sockets before `[DONE]`, emit malformed or provider-specific error payloads, hang mid-stream, surface SDK errors as in-band events, and sometimes require a delayed retry using `Retry-After`. The harness currently contains good error containment, but it does not yet treat this API instability as a first-class design problem. -`dsh-llm` defines two sanctioned adapter failure paths - throw from `stream()` or end with `finish { kind: 'error' | 'aborted' }` - and the agent loop translates both into a failed step instead of logging a fake completed assistant message. That was the right MVP containment baseline, documented in [the architecture](../../../architecture.md) and reinforced by [the twin-adapter RFC](../../implemented/architecture/2026-06-13-twin-llm-adapters.md). It is not enough for an agent that depends on an unstable remote model API for every turn. +`dsh-llm` defines two sanctioned adapter failure paths - throw from `stream()` or end with `finish { kind: 'error' | 'aborted' }` - and downstream consumers are expected to treat both as failed model calls. That was the right MVP containment baseline, documented in [the architecture](../../../architecture.md) and reinforced by [the twin-adapter RFC](../../implemented/architecture/2026-06-13-twin-llm-adapters.md). It is not enough for callers that depend on an unstable remote model API for every turn, and it forces every caller to understand two failure delivery mechanisms. -The audit found four load-bearing gaps. +The audit found three load-bearing gaps. - `LlmError` and `FinishReasonMap.error` carry only `message`, `code`, and sometimes HTTP `status` ([packages/llm/llm/src/index.ts](../../../../packages/llm/llm/src/index.ts), [packages/llm/llm/src/types.ts](../../../../packages/llm/llm/src/types.ts)). A caller cannot reliably tell "retry this after 800 ms on the same endpoint", "fail over to another route for the same model", "ask the user for credentials", "never retry because the request is invalid", "provider truncated the stream after committed output", or "adapter protocol bug" without provider-specific heuristics. -- The `llm/stream` waterfall is documented as the place for retry/routing/caching, but its value is one committed `AsyncIterable`. A listener can technically catch an API error and call `next()` again, but once it has yielded any chunks to the agent loop those chunks are already appended as `assistant/chunk` events and emitted to UI. Retrying after that point would concatenate chunks from two provider attempts into one model step, corrupting replay and the user transcript. +- The `llm/stream` waterfall is documented as the place for retry/routing/caching, but its value is one raw `AsyncIterable`. A listener can technically catch an API error and call `next()` again, but once it has yielded chunks, callers may already have rendered them, buffered them as output, or executed side effects based on completed tool calls. Retrying after that point can concatenate chunks from two provider responses into one apparent model output. The current surface has no canonical way to say "the tokens you saw were tentative; this response timed out, so discard them and restart." - The adapter registry is one adapter per model name. That makes "logical model" and "concrete API route" the same thing, so the service has no vocabulary for "same model through another endpoint or SDK", "same provider region with different health", "fallback route with compatible capabilities", provider request ids, or per-route backoff state. -- Crash recovery and LLM API recovery are easy to conflate. `dsh-session` repairs an interrupted durable log by closing an open turn and synthesizing missing tool results ([packages/core/session/src/repair.ts](../../../../packages/core/session/src/repair.ts)); that preserves already-written work after a process crash. It does not make a failed provider attempt safe to retry, discard, replay, or fail over. -The result is a system that can survive an unstable LLM API without killing the loop, but cannot make principled recovery decisions. For a coding agent, that is underdesigned: ordinary provider turbulence should not require every UI or product plugin to reinvent retries around an unsafe stream boundary. +The result is a package that can surface an unstable LLM API failure, but cannot make principled recovery decisions for its callers. Ordinary provider turbulence should not require every consumer to reinvent retries around an unsafe stream boundary. ## Proposal -Introduce an LLM-call v2 contract centered on API-instability recovery: classify provider/API failures, separate provider attempts from committed model output, route logical models through recoverable API routes, and make conservative retry/failover the default behavior in `dsh-llm`. Because the harness is unreleased, this should be a breaking cleanup rather than a compatibility layer around the underspecified v1 surface. +Introduce an LLM-call v2 contract centered on API-instability recovery: classify provider/API failures, separate provider responses from committed model output, route logical models through recoverable API routes, and make conservative retry/failover the default behavior in `dsh-llm`. Because the harness is unreleased, this should be a breaking cleanup rather than a compatibility layer around the underspecified v1 surface. ### 1. Replace flat error codes with a serializable `LlmFailure` Keep `HarnessError` as the common thrown-error base, but make LLM failures carry a structured, JSON-serializable payload. `code` remains a stable leaf label for logs and provider-specific matching; retry/failover policy branches on the structured fields. -```ts ignore-check +```ts type LlmFailureClass = | 'auth' | 'rate-limit' @@ -67,25 +66,46 @@ interface LlmFailure { } ``` -`LlmError` should carry `failure: LlmFailure`; `FinishReasonMap.error` should carry the same payload instead of a parallel `{ message, code? }` shape. The agent loop should persist the serializable failure fields in `session error` and `turn/end { kind: 'error' }`, while the thrown `LlmError` may still carry a non-serializable `cause` chain for local debugging. +`LlmError` should carry `failure: LlmFailure`; `FinishReasonMap.error` should carry the same payload instead of a parallel `{ message, code? }` shape. Adapter-thrown errors and in-band finish errors are input forms to the recovery layer. The public lifecycle stream should convert classified LLM API failures into terminal lifecycle events rather than requiring callers to catch thrown provider errors. The failure payload is serializable so callers can log or persist it if they choose, while the internal thrown `LlmError` may still carry a non-serializable `cause` chain for local debugging. Adapters are responsible for faithful provider/API classification at their boundary: HTTP status, `Retry-After`, provider request id headers, SDK error type, timeout vs caller abort, malformed SSE, missing `[DONE]`, unknown finish reason, and unsupported local request shape. The current pi-ai adapter's regex over message text is acceptable only as a temporary fallback when the SDK hides the real status; the adapter should prefer structured SDK/provider fields when available. -### 2. Split provider attempts from committed model output +### 2. Split provider responses from committed model output -Make "attempt" a first-class boundary below `ctx.llm.stream()`. An adapter streams one provider API attempt. The LLM service runs zero or more attempts according to recovery policy and yields only committed output to the agent loop. +Make "response" a first-class boundary in `ctx.llm.stream()`. An adapter streams one provider API response. The LLM service runs zero or more responses according to recovery policy and exposes one canonical response-lifecycle stream to consumers. Convenience APIs may expose a committed-or-failed result for simple callers, but that result must be derived from the lifecycle stream, not a parallel contract. -The important invariant: chunks from a failed attempt must never be silently spliced together with chunks from a later attempt as one assistant step. Recovery must choose one of these paths instead: +The primary response id is generated by the harness before the adapter call starts. Provider response ids and request ids are metadata attached when known; they are not the primary key because providers may omit them, report them only after the stream starts, reuse them in surprising ways, or fail before one exists. -- **Retry before commit.** If an API attempt fails before any chunks are committed to the loop, the service may retry or fail over and hide the failed attempt from `assistant/chunk` history, while recording attempt diagnostics separately. -- **Commit and stop retrying.** Once chunks are committed to the loop, the attempt owns the visible step. If it later fails, the step fails with `partialOutput: 'committed'`; automatic retry is not allowed unless a later RFC designs an explicit continuation/repair protocol. -- **Buffered recovery mode.** A caller or policy may choose to buffer an entire attempt until it reaches `finish`, then yield the winning attempt's chunks. This improves retryability against flaky APIs at the cost of live token streaming and should be a deliberate mode, not an accidental side effect. +The important invariant: chunks from a failed response must never be silently spliced together with chunks from a later response as one apparent model result. Token deltas from a response are tentative until that response reaches a committing terminal finish (`stop`, `tool-calls`, or `max-tokens`). The lifecycle stream must be able to report that tentative tokens were shown live, then discarded because the response timed out, disconnected, or otherwise failed before commit. -This likely means replacing the single overloaded `llm/stream` waterfall with narrower hooks: one around a single provider attempt, one around recovery policy decisions, and one around the committed stream. Names are implementation details for the follow-up PR, but the semantics are not: plugins must be able to wrap "one API attempt" without pretending they can safely retry already-committed chunks. +The event vocabulary should keep the familiar `assistant/chunk` concept but stop pretending every chunk is already final output. A possible spelling is: + +```ts ignore-check +type LlmStreamEvent = + | { type: 'response/start'; responseId: ResponseId; responseIndex: number; routeId: string } + | { type: 'assistant/chunk'; responseId: ResponseId; commitment: 'uncommitted'; chunk: StreamChunk } + | { type: 'response/interrupted'; responseId: ResponseId; failure: LlmFailure; scheduledRetryMs?: number } + | { type: 'response/failed'; responseId?: ResponseId; failure: LlmFailure } + | { type: 'response/committed'; responseId: ResponseId; message: Message; finish: FinishReason; usage?: TokenUsage } + +type GenerateOutcome = + | { type: 'committed'; responseId: ResponseId; message: Message; finish: FinishReason; usage?: TokenUsage } + | { type: 'failed'; responseId?: ResponseId; failure: LlmFailure } +``` + +The implementation may choose the exact names, but the type shape should make the state transition obvious: assistant chunks start uncommitted, then the enclosing response becomes interrupted/discarded, failed, or committed. + +- **Lifecycle assistant chunks.** The lifecycle stream should make the old ambiguity explicit: these events are assistant chunk messages, but each one belongs to a response and has a commitment state. Most arrive as uncommitted live UI state; a response that fails before commit marks them interrupted/discarded, and a response that reaches a committing finish lets the UI mark that response committed. +- **Retry before commit.** If an API response fails before a committing finish, the service may retry or fail over and exclude the failed response from the committed result, while surfacing response diagnostics separately. +- **Commit on terminal finish.** Once a response reaches a committing finish, the response owns the visible result. `dsh-llm` emits a `response/committed` event carrying the fully assembled assistant `Message`, final `FinishReason`, usage, and response metadata. Callers that persist messages, execute tool calls, or otherwise take side effects should use this committed event rather than rebuilding output from lifecycle chunks. +- **Terminal failure.** If recovery reaches a non-retryable failure, is aborted by the caller, or otherwise stops without a committing finish, `dsh-llm` emits `response/failed` carrying the final `LlmFailure` and ends the lifecycle stream normally. Throwing is reserved for defects outside the classified LLM API failure contract. +- **Fail after commit.** If a later failure is ever observable after commit, the lifecycle reports `response/failed` with `partialOutput: 'committed'`; automatic retry is not allowed unless a later RFC designs an explicit continuation/repair protocol. + +This means replacing the single overloaded raw-chunk `llm/stream` waterfall with a lifecycle stream and narrower hooks: one around a single provider response, one around recovery policy decisions, and one around convenience APIs that only expose the terminal outcome. Names are implementation details for the follow-up PR, but the semantics are not: plugins must be able to wrap "one API response" without pretending they can safely retry already-committed chunks. ### 3. Route logical models through recoverable API routes -Separate the logical model a caller requests from the concrete provider route that serves an attempt. Replace "one adapter per model name" with route registration, for example: +Separate the logical model a caller requests from the concrete provider route that serves a response. Replace "one adapter per model name" with route registration, for example: ```ts ignore-check ctx.llm.registerRoute({ @@ -99,52 +119,81 @@ ctx.llm.registerRoute({ }) ``` -`GenerateOptions.model` remains the logical model. The service resolves it to a route for each API attempt, records the route in failure/attempt diagnostics, and can retry on the same route or fail over to another route with compatible capabilities. Duplicate model names become normal; duplicate route ids are the conflict. This is the smallest vocabulary that can express direct endpoint vs SDK-backed endpoint, regional endpoints, and future fallback models without making every caller own routing. +`GenerateOptions.model` remains the logical model. The service resolves it to a route for each API response, records the route in failure/response diagnostics, and can retry on the same route or fail over to another route with compatible capabilities. Duplicate model names become normal; duplicate route ids are the conflict. This is the smallest vocabulary that can express direct endpoint vs SDK-backed endpoint, regional endpoints, and future fallback models without making every caller own routing. ### 4. Put default API recovery policy in `dsh-llm` -Adapters should not perform hidden SDK retries unless those retries are surfaced as attempts with classified failures. The service owns the default policy so every consumer gets the same behavior and the same audit trail. +Adapters should not perform hidden SDK retries unless those retries are surfaced as response lifecycle events with classified failures. The service owns the default policy so every consumer gets the same behavior and the same audit trail. Default policy should be conservative: -- Retry transient API failures (`rate-limit`, `timeout`, `transport`, `provider-overloaded`, `provider-unavailable`) only before committed output. -- Honor `retryAfterMs`, otherwise use bounded exponential backoff with jitter. +- Retry transient API failures (`rate-limit`, `timeout`, `transport`, `provider-overloaded`, `provider-unavailable`) only before committed output, and keep retrying until the caller aborts or the failure class changes to a non-retryable one. +- Honor `retryAfterMs` up to `maxRetryDelayMs`, otherwise use bounded exponential backoff with jitter. The same cap applies to both provider-supplied retry hints and ordinary exponential backoff; diagnostics record whether the delay source was `provider-retry-after` or `exponential-backoff`. - Treat 429/408/409/425/500/502/503/504 and connection resets as potentially recoverable unless the provider payload says otherwise; treat 400/401/403, unsupported local options, caller abort, and adapter protocol bugs as non-retryable. - Fail over only when the failure says failover is safe and the candidate route advertises compatible capabilities for the request (`tools`, reasoning passback, images, prefill, stop sequences, strict tools). -- Share the caller's `AbortSignal` across the whole recovered call, and expose per-attempt timeouts as explicit policy. A stuck stream must time out in a controlled way instead of hanging the turn forever. -- Bound attempts by count and elapsed time, with clear failure reporting when the budget is exhausted. +- Share the caller's `AbortSignal` across the whole recovered call, and expose per-response timeouts as explicit policy. A stuck stream must time out in a controlled way instead of hanging the turn forever. +- Surface every retry decision to the UI with retry count, backoff delay, route, and failure summary, so an actively watching user can tell the agent is waiting on provider capacity instead of frozen. -The policy should be configurable through a typed service option and an event/waterfall seam so product plugins can tighten or loosen it, but the default must be safe enough that a basic agent does not need a custom retry plugin to survive ordinary 429/5xx/connectivity noise. +The policy should be configurable through a typed service option and an event/waterfall seam so product plugins can adjust timing/backoff details, but the default retry posture is not opt-in: a basic agent should keep recovering from retryable 429/5xx/connectivity noise until cancelled. -### 5. Record API attempt diagnostics without polluting derived history +The zero-config defaults should be sensible production behavior, not placeholders: -The session log should be able to explain what happened during a recovered model call without feeding failed attempts back to the model as assistant output. Add turn-enclosed, derive-skipped diagnostics for LLM API attempts, or an equivalent trace surface if we decide session events should stay conversation-only. The data must be JSON-serializable and include attempt number, route id, failure payload, backoff, provider request id, and whether any chunks were committed. +```ts +const defaultLlmRecoveryConfig = { + maxResponses: 'unbounded', + maxElapsedMs: 'unbounded', + connectTimeoutMs: 15_000, + responseHeaderTimeoutMs: 60_000, + streamIdleTimeoutMs: 5 * 60_000, + initialBackoffMs: 200, + maxRetryDelayMs: 10 * 60_000, + jitterRatio: 0.1, +} +``` -`assistant/chunk` remains the replay source for the committed attempt only. Hidden failed attempts are diagnostics, not model history. A stream that fails after committed chunks remains replayable as a thrown stream through the existing `llm-replay` sidecar mechanism, but the failure payload should become structured rather than `{ message, code, status? }`. +The retry-delay cap is deliberate. The survey found mixed precedent: Codex parses retry delays out of streamed OpenAI rate-limit error messages and uses that requested delay, but Codex also has finite stream retry counts; the official OpenAI and Anthropic TypeScript SDKs parse `retry-after-ms`, `Retry-After` seconds, and `Retry-After` dates and then sleep for the provider-specified duration; the official OpenAI and Anthropic Python SDKs only honor `Retry-After` when it is greater than zero and at most 60 seconds, otherwise falling back to ordinary exponential backoff. Because this RFC's default retry posture is unbounded, blindly honoring a multi-hour provider delay can make the agent look dead, while ignoring the hint entirely can retry too aggressively. The service should therefore record both `providerRetryAfterMs` and `scheduledRetryMs`, cap the scheduled sleep at `maxRetryDelayMs`, and surface that choice to the UI. + +The service cannot reliably infer whether the user is actively watching or away from the keyboard, so the default should not fail a retryable model call merely because a short interactive budget expired. A clear UI can make long waits tolerable even in interactive sessions: "retried 8 times; next retry in 10 minutes" is better than silently failing recoverable provider turbulence and asking the user to resubmit. + +### 5. Define the caller contract, not the product transcript + +This RFC is deliberately about the `dsh-llm` API and how callers use it, not about the final transcript/event architecture of the product. The LLM package should guarantee these caller-visible semantics: + +- `ctx.llm.stream()` is the live response-lifecycle API. It reports response starts, uncommitted assistant chunks, retries/backoff, interruptions/discards, terminal failures, and the one committed result. +- `response/committed` is the only event that makes model output safe for history or side effects. It carries the assembled `Message`, finish reason, usage, response id, route metadata, and provider ids known to the service. +- `response/failed` is the terminal event for classified failures. Callers should not need `try`/`catch` to learn that a provider was rate-limited, unavailable, misconfigured, aborted, or otherwise unable to produce a committed response. +- Response diagnostics are JSON-serializable so callers can store, display, or ignore them. The LLM package does not decide whether those diagnostics become session events, agent events, telemetry rows, or UI-only state. +- Convenience helpers such as `generate()` return a terminal union (`committed` or `failed`) rather than throwing for classified LLM failures. They must be derived from the lifecycle stream so recovery semantics stay single-sourced. + +The session log shape, agent event taxonomy, ACP rendering, snapshot/replay fixtures, and whether live uncommitted chunks are ever durably recorded are downstream integration decisions. This RFC should constrain them only by the LLM API contract above. ## Out of scope -This RFC does not propose silent mid-stream continuation after user-visible output. That requires a separate model-history design: either provider-supported prefill/continuation, a recovery prompt that explicitly shows the partial assistant output, or a UI affordance that marks the partial answer as failed and asks the model to continue in a new step. Splicing two API attempts into one assistant message is rejected. +This RFC does not propose silent mid-stream continuation after user-visible output. That requires a separate model-history design: either provider-supported prefill/continuation, a recovery prompt that explicitly shows the partial assistant output, or a UI affordance that marks the partial answer as failed and asks the model to continue in a new step. Splicing two API responses into one assistant message is rejected. This RFC also does not solve semantic model-output repair: malformed tool-call JSON, refusal handling, or content-filter fallbacks. Those may use the same failure vocabulary later, but they are higher-level agent behaviors, not unstable-API recovery. +This RFC does not decide whether product UIs consume LLM lifecycle events directly, through agent events, or through session events. It also does not decide which response diagnostics belong in the durable session log. Those decisions belong in narrower integration RFCs once the `dsh-llm` contract exists. + ## Acceptance criteria -- `LlmError` and in-band finish errors carry one structured `LlmFailure` payload; the agent loop persists that payload's serializable fields in error turn data. +- Adapter-thrown `LlmError`s and in-band finish errors carry one structured, JSON-serializable `LlmFailure` payload. - Recovery policy can distinguish retry, failover, credential/user-action, unsupported request, caller abort, adapter/protocol bug, and post-commit partial stream failure without parsing message text. -- The LLM service has an explicit API-attempt boundary; no retry path can append chunks from two provider attempts as one committed assistant step. -- The route registry allows multiple concrete API routes for one logical model and records the selected route on attempts/failures. -- Default recovery retries transient pre-commit failures with bounded backoff, honors provider retry-after hints, times out stuck streams, disables hidden SDK retries or surfaces them as attempts, and never retries after committed chunks without an explicit continuation design. -- Unit tests cover thrown errors and finish-error chunks through the real agent loop, retry-before-first-chunk, failover to a second compatible route, retry budget exhaustion, abort during backoff, stream timeout, and the "partial chunks then failure does not retry/splice" invariant. +- `ctx.llm.stream()` exposes the response lifecycle as the canonical stream, including terminal `response/failed` events for classified failures; convenience APIs are derived views for callers that only want the terminal outcome. +- `response/committed` carries the assembled assistant `Message`; callers do not need to rebuild committed output from lifecycle chunks. +- `generate()` returns a committed/failed union derived from the lifecycle stream, rather than throwing for classified LLM failures. +- The LLM service has an explicit API-response boundary; no retry path can present output from two provider responses as one committed assistant result. +- The route registry allows multiple concrete API routes for one logical model and records the selected route on responses/failures. +- Default recovery retries transient pre-commit failures with bounded backoff, honors provider retry-after hints, times out stuck streams, disables hidden SDK retries or surfaces them as response lifecycle events, and never retries after committed chunks without an explicit continuation design. +- Unit tests cover thrown errors and finish-error chunks through `dsh-llm`, retry-before-first-commit, failover to a second compatible route, unbounded retry status/backoff visibility, abort during backoff, stream timeout, and the "partial chunks then failure does not retry/splice" invariant. - Adapter tests classify representative HTTP statuses, retry-after headers, request ids, malformed/truncated SSE streams, SDK in-stream errors, caller aborts, and unsupported options into `LlmFailure`. -- Snapshot/replay support can faithfully represent a recovered call and a post-commit thrown stream without losing the structured failure payload. -- Docs updated in the same change: [the architecture LLM section](../../../architecture.md), [the LLM adapter cookbook](../../../cookbook/adding-an-llm-adapter.md), and the LLM package READMEs. +- Docs updated in the same change: [the architecture LLM section](../../../architecture.md), [the LLM adapter cookbook](../../../cookbook/adding-an-llm-adapter.md), and the LLM package README. ## Risks / what we give up -- **More surface area in the LLM core.** API recovery adds policy, route state, attempt diagnostics, and tests. That complexity belongs in `dsh-llm` because every consumer otherwise reinvents it around the same unsafe stream boundary. -- **Some modes reduce live streaming.** Buffered recovery trades first-token latency for safe retry. That should be opt-in or policy-driven; the default can still stream eagerly while retrying only before commit. -- **Breaking adapter churn.** Existing adapters will change from "stream chunks or throw a flat `LlmError`" to "stream one classified API attempt." Pre-release rules favor the correct seam over shims. +- **More surface area in the LLM core.** API recovery adds policy, route state, response diagnostics, and tests. That complexity belongs in `dsh-llm` because every consumer otherwise reinvents it around the same unsafe stream boundary. +- **Committed result lags live UI.** Safe recovery means callers cannot treat streamed tokens as final assistant output until the response commits. UIs can still stream eagerly from lifecycle assistant chunks, but side-effecting consumers must wait for `response/committed`. +- **Breaking adapter churn.** Existing adapters will change from "stream chunks or throw a flat `LlmError`" to "stream one classified API response." Pre-release rules favor the correct seam over shims. - **Route compatibility is easy to overclaim.** A route must advertise concrete capabilities, and failover must check the request actually fits them. "Same model name" is not enough when one route lacks strict tools, reasoning passback, images, stop sequences, or prefill. ## Related From 133f37584f1e9ad13796a0cd3f7c11b6d143f931 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 21 Jun 2026 10:23:16 +0800 Subject: [PATCH 03/15] docs: reconcile LLM recovery RFC with sibling PRs --- .../2026-06-21-unstable-llm-api-recovery.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md index f5f7659c41..1f8f300509 100644 --- a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md +++ b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md @@ -88,7 +88,7 @@ type LlmStreamEvent = | { type: 'response/failed'; responseId?: ResponseId; failure: LlmFailure } | { type: 'response/committed'; responseId: ResponseId; message: Message; finish: FinishReason; usage?: TokenUsage } -type GenerateOutcome = +type LlmCallOutcome = | { type: 'committed'; responseId: ResponseId; message: Message; finish: FinishReason; usage?: TokenUsage } | { type: 'failed'; responseId?: ResponseId; failure: LlmFailure } ``` @@ -121,6 +121,10 @@ ctx.llm.registerRoute({ `GenerateOptions.model` remains the logical model. The service resolves it to a route for each API response, records the route in failure/response diagnostics, and can retry on the same route or fail over to another route with compatible capabilities. Duplicate model names become normal; duplicate route ids are the conflict. This is the smallest vocabulary that can express direct endpoint vs SDK-backed endpoint, regional endpoints, and future fallback models without making every caller own routing. +The route registry must keep the lifecycle guarantees of the current adapter registry: `registerRoute()` is effect-scoped, returns a disposer, and has an HMR-safety test proving disposal removes the route. It should not preserve `llm/adapter-change`; if [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82) lands first, that event is already gone, and the route registry should not reintroduce it without a concrete consumer. + +The new ids should follow the branded-id policy. `ResponseId`, `RouteId`, and the logical/wire model ids cross package boundaries and are easy to swap accidentally, so the implementation should deliberately brand or explicitly decline to brand each one in line with `2026-06-20-branded-ids` and its implementation stack ([PR #84](https://github.com/deepseek-ai/deepseek-harness/pull/84)). + ### 4. Put default API recovery policy in `dsh-llm` Adapters should not perform hidden SDK retries unless those retries are surfaced as response lifecycle events with classified failures. The service owns the default policy so every consumer gets the same behavior and the same audit trail. @@ -163,9 +167,9 @@ This RFC is deliberately about the `dsh-llm` API and how callers use it, not abo - `response/committed` is the only event that makes model output safe for history or side effects. It carries the assembled `Message`, finish reason, usage, response id, route metadata, and provider ids known to the service. - `response/failed` is the terminal event for classified failures. Callers should not need `try`/`catch` to learn that a provider was rate-limited, unavailable, misconfigured, aborted, or otherwise unable to produce a committed response. - Response diagnostics are JSON-serializable so callers can store, display, or ignore them. The LLM package does not decide whether those diagnostics become session events, agent events, telemetry rows, or UI-only state. -- Convenience helpers such as `generate()` return a terminal union (`committed` or `failed`) rather than throwing for classified LLM failures. They must be derived from the lifecycle stream so recovery semantics stay single-sourced. +- Any assembled convenience helper that survives or is reintroduced returns a terminal union (`committed` or `failed`) rather than throwing for classified LLM failures. It must be derived from the lifecycle stream so recovery semantics stay single-sourced. -The session log shape, agent event taxonomy, ACP rendering, snapshot/replay fixtures, and whether live uncommitted chunks are ever durably recorded are downstream integration decisions. This RFC should constrain them only by the LLM API contract above. +The session log shape, agent event taxonomy, ACP rendering, snapshot/replay fixtures, and whether live uncommitted chunks are ever durably recorded are downstream integration decisions. So is the fate of today's assembled public helper methods: [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82) implements the proposed removal of `generate()`, `streamBlocks()`, `GenerateResult`, and `llm/generate`, and this RFC should not resurrect them without a real caller. This RFC should constrain downstream work only by the LLM API contract above. ## Out of scope @@ -181,9 +185,11 @@ This RFC does not decide whether product UIs consume LLM lifecycle events direct - Recovery policy can distinguish retry, failover, credential/user-action, unsupported request, caller abort, adapter/protocol bug, and post-commit partial stream failure without parsing message text. - `ctx.llm.stream()` exposes the response lifecycle as the canonical stream, including terminal `response/failed` events for classified failures; convenience APIs are derived views for callers that only want the terminal outcome. - `response/committed` carries the assembled assistant `Message`; callers do not need to rebuild committed output from lifecycle chunks. -- `generate()` returns a committed/failed union derived from the lifecycle stream, rather than throwing for classified LLM failures. +- Any assembled convenience API that survives or is reintroduced returns a committed/failed union derived from the lifecycle stream, rather than throwing for classified LLM failures. - The LLM service has an explicit API-response boundary; no retry path can present output from two provider responses as one committed assistant result. - The route registry allows multiple concrete API routes for one logical model and records the selected route on responses/failures. +- `registerRoute()` is effect-scoped, returns a disposer, and has an HMR-safety test proving route cleanup; `llm/adapter-change` is not reintroduced unless a concrete consumer needs it. +- New LLM ids are deliberately branded or explicitly left unbranded according to the branded-id policy, with `ResponseId`, `RouteId`, and logical/wire model ids decided together. - Default recovery retries transient pre-commit failures with bounded backoff, honors provider retry-after hints, times out stuck streams, disables hidden SDK retries or surfaces them as response lifecycle events, and never retries after committed chunks without an explicit continuation design. - Unit tests cover thrown errors and finish-error chunks through `dsh-llm`, retry-before-first-commit, failover to a second compatible route, unbounded retry status/backoff visibility, abort during backoff, stream timeout, and the "partial chunks then failure does not retry/splice" invariant. - Adapter tests classify representative HTTP statuses, retry-after headers, request ids, malformed/truncated SSE streams, SDK in-stream errors, caller aborts, and unsupported options into `LlmFailure`. @@ -201,3 +207,5 @@ This RFC does not decide whether product UIs consume LLM lifecycle events direct - Builds on [Provider-neutral content-block vocabulary](../../implemented/architecture/2026-06-11-content-block-vocabulary.md): the content vocabulary stays provider-neutral; this adds a provider-neutral failure/recovery vocabulary beside it. - Revises the scope implied by [Two LLM adapters as a design-verification twin](../../implemented/architecture/2026-06-13-twin-llm-adapters.md): the twin validated chunk shape and error delivery paths, but it also exposed that delivery paths are not enough for unstable API recovery. - Extends [Structured error taxonomy](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md): `HarnessError.code` was the foundation; LLM API recovery needs a richer payload because retry/failover policy cannot safely branch on one flat string. +- Coordinates with [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82), which implements the `drop-unconsumed-llm-assembled-surfaces` and `drop-unconsumed-llm-adapter-change-event` simplification RFCs. If that PR lands first, this RFC starts from a narrower `dsh-llm`: no `generate()`, no `streamBlocks()`, no `GenerateResult`, no `llm/generate`, and no `llm/adapter-change`. Recovery should build on that baseline rather than revive removed convenience or change-notification surfaces speculatively. +- Coordinates with [PR #84](https://github.com/deepseek-ai/deepseek-harness/pull/84), which implements the branded-id RFC. The new response/route/model ids introduced here are exactly the sort of cross-boundary ids that need an explicit branding decision before implementation. From 5716fab2c5499ec2f4bbed80eb64bc6f1f82b6bf Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Sun, 21 Jun 2026 10:27:53 +0800 Subject: [PATCH 04/15] docs: note in-flight RFC implementation stack --- .../architecture/2026-06-21-unstable-llm-api-recovery.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md index 1f8f300509..c1fc8703c8 100644 --- a/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md +++ b/docs/rfc/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md @@ -171,6 +171,8 @@ This RFC is deliberately about the `dsh-llm` API and how callers use it, not abo The session log shape, agent event taxonomy, ACP rendering, snapshot/replay fixtures, and whether live uncommitted chunks are ever durably recorded are downstream integration decisions. So is the fate of today's assembled public helper methods: [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82) implements the proposed removal of `generate()`, `streamBlocks()`, `GenerateResult`, and `llm/generate`, and this RFC should not resurrect them without a real caller. This RFC should constrain downstream work only by the LLM API contract above. +The current simplification stack was checked while drafting this proposal. [PR #83](https://github.com/deepseek-ai/deepseek-harness/pull/83) and [PR #85](https://github.com/deepseek-ai/deepseek-harness/pull/85) do not change this RFC's LLM API assumptions. [PR #86](https://github.com/deepseek-ai/deepseek-harness/pull/86) does matter for later integration because it folds durable token usage onto `assistant/message` and operational errors onto `turn/end.reason`; if it lands first, the LLM recovery implementation should still stop at the `dsh-llm` lifecycle contract here and let the agent/session layer decide how committed usage and terminal failures map onto those load-bearing product events. + ## Out of scope This RFC does not propose silent mid-stream continuation after user-visible output. That requires a separate model-history design: either provider-supported prefill/continuation, a recovery prompt that explicitly shows the partial assistant output, or a UI affordance that marks the partial answer as failed and asks the model to continue in a new step. Splicing two API responses into one assistant message is rejected. @@ -208,4 +210,6 @@ This RFC does not decide whether product UIs consume LLM lifecycle events direct - Revises the scope implied by [Two LLM adapters as a design-verification twin](../../implemented/architecture/2026-06-13-twin-llm-adapters.md): the twin validated chunk shape and error delivery paths, but it also exposed that delivery paths are not enough for unstable API recovery. - Extends [Structured error taxonomy](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md): `HarnessError.code` was the foundation; LLM API recovery needs a richer payload because retry/failover policy cannot safely branch on one flat string. - Coordinates with [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82), which implements the `drop-unconsumed-llm-assembled-surfaces` and `drop-unconsumed-llm-adapter-change-event` simplification RFCs. If that PR lands first, this RFC starts from a narrower `dsh-llm`: no `generate()`, no `streamBlocks()`, no `GenerateResult`, no `llm/generate`, and no `llm/adapter-change`. Recovery should build on that baseline rather than revive removed convenience or change-notification surfaces speculatively. +- Is orthogonal to [PR #81](https://github.com/deepseek-ai/deepseek-harness/pull/81), which proposes provider-request app attribution headers. Recovery route metadata and provider request construction can carry attribution policy later, but this RFC does not define request headers. - Coordinates with [PR #84](https://github.com/deepseek-ai/deepseek-harness/pull/84), which implements the branded-id RFC. The new response/route/model ids introduced here are exactly the sort of cross-boundary ids that need an explicit branding decision before implementation. +- Coordinates with [PR #86](https://github.com/deepseek-ai/deepseek-harness/pull/86), which implements the `collapse-trace-only-session-events` simplification RFC. If that stack lands first, downstream recovery integration should map committed usage and terminal error facts onto the surviving load-bearing session events instead of reintroducing standalone trace-only records from inside `dsh-llm`. From 277b58088f68b10cac0b9b6b1010f2ea7dfe7a0b Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 00:28:51 +0800 Subject: [PATCH 05/15] docs: narrow LLM request recovery proposal --- ...2026-06-21-bounded-llm-request-recovery.md | 147 ++++++++++++ .../2026-06-21-unstable-llm-api-recovery.md | 219 ------------------ 2 files changed, 147 insertions(+), 219 deletions(-) create mode 100644 .agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md delete mode 100644 .agents/notes/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md diff --git a/.agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md b/.agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md new file mode 100644 index 0000000000..e57cffe687 --- /dev/null +++ b/.agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md @@ -0,0 +1,147 @@ +# Agent Note: Bounded recovery for transient LLM request failures + +Status: proposed + +## Problem + +`dsh-llm` can report provider failures either by throwing during adapter dispatch or iteration or by ending with `finish { kind: 'error' | 'aborted' }`. The final adapter boundary tags thrown failures so `dsh-agent-loop` can distinguish them from middleware and result-processing defects, and the loop normalizes both delivery forms into `agent/request-error` after closing the failed step. The default decision is `fail`; `dsh-compact-basic` is the only shipped recovery listener, and it retries a canonical context-window overflow only after compaction proves that the durable surface shrank. + +That boundary is already safe for another request attempt. Raw `assistant/chunk` events carry the failed `turn` and `step`, message derivation ignores them unless a successful `assistant/message` cites them, tool calls are dispatched only after a successful terminal finish and assembly, and a retry opens a new numbered step from the durable log. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate. + +Three narrower gaps remain. + +- Provider failures retain only a message and usually a code. HTTP status, retry delay, and provider request id are discarded or recoverable only through provider-specific error objects, so generic recovery cannot make or explain a decision without parsing text. +- Retry ownership differs by adapter. The hand-written DeepSeek adapter makes one attempt, while pi-ai profiles can enable opaque library retries. Combining hidden transport retries with a future `agent/request-error` listener would multiply attempts and omit intermediate failures from the session log. +- A recovered failure has no durable status fact. The failed step and chunks remain reconstructable, but an observer cannot tell whether the agent is deliberately backing off, for how long, or why. A long silent wait looks like a stalled loop. + +The goal is bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer. + +## Proposal + +### Preserve failure facts without embedding policy + +Add one JSON-serializable `LlmFailure` payload to `@deepseek-ai/dsh-llm`: + +```ts ignore-check +type ProviderRequestId = Branded<'ProviderRequestId'> + +interface LlmFailure { + message: string + code: string + status?: number + retryAfterMs?: number + requestId?: ProviderRequestId +} +``` + +`code` remains the provider-neutral machine-routing taxonomy established by `HarnessError`; the new fields are observations from the provider boundary. `ProviderRequestId` is owned and constructed by `dsh-llm`, then serializes as its provider-issued string. The payload deliberately has no `retryable`, `failover`, `partialOutput`, provider, model, phase, or route id fields. Retryability belongs to policy, provider/model are already in the durable request header, and partial output is derived from the failed step's `assistant/chunk` events. + +`LlmError` carries `failure: LlmFailure` and preserves `failure.code === error.code`. `FinishReasonMap.error` and `FinishReasonMap.aborted` carry the same payload instead of parallel failure shapes. An adapter-thrown `Error` keeps its exact object identity: the final-adapter scope associates the normalized facts with that object in call-local sidecar state and rethrows it unchanged; a non-`Error` throw is wrapped as today. `llmFailureOf(stream, error)` retrieves those facts alongside the existing provenance check, while an in-band finish without an error object becomes a new `LlmError`. This preserves listeners that key on error type or identity while giving all final-adapter failures, including unknown SDK exceptions, an `UNKNOWN` terminal payload. + +The agent loop keeps `RequestError` as that exact error object and passes `LlmFailure` as a separate argument to `agent/request-error`; it does not mutate possibly frozen third-party errors. It also uses the payload when converting an in-band finish and when recording an unrecovered `turn/end.reason`. + +Adapters extract structured facts before falling back to message inspection. They validate HTTP status, parse `Retry-After` seconds or dates into a positive finite millisecond delay, brand the provider request id when exposed, and distinguish their own timeout from the caller's abort. Provider-specific codes and messages may refine a mapping, but no recovery listener parses them. + +The initial shared transient-code set is intentionally small: the adapters' existing `RATE_LIMIT` and `SERVER` mappings plus explicit `TIMEOUT` and `TRANSPORT` codes for the two missing remote-failure families. Authentication, quota, invalid request, context overflow, protocol, abort, and unknown failures keep distinct stable codes and are not transient by default. Adding a code requires adapter fixtures and a documented policy decision; it does not require expanding a second failure-class enum. + +### Put retry policy on the existing failed-step seam + +Add a function plugin, `@deepseek-ai/dsh-llm-retry`, that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow. + +Replace the scalar `retryAttempt` argument with the current `LlmFailure` and an immutable list of prior failures that led to another request attempt in this consecutive recovery sequence. `dsh-llm-retry` counts only prior failures whose codes are in its configured transient set, while `dsh-compact-basic` counts only prior context-overflow failures. A successful model request clears the history as it clears the current scalar. Alternating transient and context-overflow failures therefore consume their owning policy budgets independently; the maximum request count is one plus the sum of the finite budgets of the loaded recovery policies. + +The plugin resolves and validates this deployment configuration at load: + +```ts ignore-check +interface Config { + maxTransientRetries?: number + initialDelayMs?: number + maxDelayMs?: number + jitterRatio?: number + retryableCodes?: string[] +} +``` + +The defaults are two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the four transient codes above. The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47). Two retries mean at most three provider requests when no other recovery policy applies. `maxTransientRetries` is a non-negative integer, delays are positive finite numbers with `initialDelayMs <= maxDelayMs`, `jitterRatio` is in `[0, 1]`, and codes are non-empty and unique. These are Cordis config fields rather than hidden constants so deployments can choose different cost and latency budgets. + +For an eligible failure with budget remaining, the one-based transient retry count uses bounded exponential backoff. A valid provider `retryAfterMs` replaces exponential backoff only when it does not exceed `maxDelayMs`; a longer provider delay causes delegation instead of an earlier retry that violates the provider instruction. Local backoff multiplies by an injected random factor in `[1 - jitterRatio, 1 + jitterRatio]` and clamps the final value to `maxDelayMs`; provider delay is not jittered. + +The plugin owns a lifetime `AbortController` and tracks every active backoff callback. Each wait fuses the waterfall's turn signal with that lifetime signal. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; a captured callback whose lifetime signal aborts returns `fail` and can neither retry nor enter the rest of its captured waterfall after disposal. This makes HMR disposal quiescent even though Cordis has already captured the listener. + +Before sleeping, `dsh-llm-retry` appends one non-surface `llm/retry` session event containing the turn, failed step, one-based transient retry number, configured maximum, scheduled delay, and `LlmFailure`. The plugin owns the `SessionEventMap` augmentation; `dsh-session` remains generic persistence and does not absorb the optional policy's vocabulary. The event says what was scheduled, not that the next request completed; cancellation during the delay is subsequently visible on `turn/end`. The event ships only with a production renderer and replay/snapshot coverage, because its purpose is operational state rather than trace collection. + +The listener calls `next()` for a non-transient code, an exhausted policy budget, or an over-cap provider delay. This preserves composition with context-overflow recovery and later policy plugins. It returns `{ action: 'retry' }` only after the delay completes under both signals; turn cancellation and plugin disposal return `fail`, after which the loop's cancellation/disposal checks remain authoritative. + +The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot CLI, and ACP example compositions use the same bounded policy. Library consumers retain explicit plugin composition: omitting the plugin leaves `agent/request-error` at its current fail default. + +### Make one layer own visible attempts + +Adapters perform one provider request per `stream()` call. The pi-ai adapter removes public `maxRetries` and `maxRetryDelayMs` profile fields and disables library retries; the hand-written adapter keeps its current single-attempt behavior. This prevents an SDK budget from multiplying the agent budget and ensures every transient retry is represented by a closed failed step plus `llm/retry`. + +`ctx.llm.stream()` remains the raw one-attempt waterfall. Direct callers such as compaction summarization receive the structured failure but do not gain automatic retry, because they have no agent step boundary or general durable place to separate attempts. A future direct-call consumer may justify a buffering helper that retries only before emitting a chunk, but this proposal does not add one speculatively. + +### Bound stalled streams where they can be stopped + +Each adapter exposes a validated `streamIdleTimeoutMs` configuration field with the five-minute prior-art default cited above. The interval covers each outstanding iterator `next()` from demand to the next valid `StreamChunk`; time a consumer spends between `next()` calls is not provider idle time. + +Extend `@deepseek-ai/dsh-timeout` with a rearmable idle-watchdog primitive. One stable local `AbortController` is fused with the caller signal and passed to the transport for the whole adapter call; each outstanding `next()` arms the watchdog, resolution disarms it, and the next demand rearms it. Timeout aborts that stable controller with a capability-owned `TimeoutReason`, and `finally` clears the timer. The adapter classifies its watchdog as `TIMEOUT` and an earlier upstream abort as `ABORTED`. The existing one-shot `deadline()` is not presented as a sliding timer. + +The two adapters must prove termination at their actual boundaries. The hand-written adapter aborts its fetch/reader, and the pi-ai adapter maps the stable signal through the SDK only after a test proves the SDK stops the request. A timer that merely rejects a consumer promise while leaving the request running does not satisfy the contract. + +### Keep attempts separate in the existing log + +A failed attempt may leave `assistant/chunk` events in its closed step, but it never appends `assistant/message` and never dispatches a tool. A retry opens the next numbered step, reconstructs the request from the durable surface, and produces its own chunks. UIs may render live chunks while a step is open, then mark or clear that transient view when `llm/retry` identifies the failed step or `turn/end` records terminal failure; message derivation continues to ignore the failed chunks. + +If recovery is exhausted, the final failure is stored once on `turn/end.reason` with the structured facts. If transient recovery continues, `llm/retry` is the durable home for that attempt's failure and delay. No standalone final-error event or response-id vocabulary is added. + +## Out of scope + +- Automatic provider or model failover. Requests already select one explicit provider and model, and the provider registry deliberately has one adapter owner per provider. +- Retrying or continuing after a successful terminal finish, or splicing chunks from two attempts into one assistant message. +- Repairing malformed tool arguments, refusals, content filters, or other semantic model output. +- Unbounded retries, unattended retry-until-cancelled behavior, circuit breakers, shared provider health, or cross-agent retry budgets. +- Changing `llm/stream` into a response lifecycle or adding convenience generation APIs without a production consumer. + +## Alternatives considered + +- **Retry inside `llm/stream` or the provider SDK** — rejected because a raw stream has no durable attempt boundary after emitting chunks, hidden SDK retries multiply budgets, and neither path can record each failed attempt consistently. +- **Add response start, interrupted, discarded, failed, and committed events to `dsh-llm`** — rejected because the agent log already separates raw chunks, successful messages, and numbered attempts. A second state machine would duplicate ownership without enabling the bounded same-route retry. +- **Add logical routes, capability matrices, and failover selection** — rejected because current requests already name provider and model explicitly, one adapter owns each provider, and no current consumer requires automatic fallback or can prove semantic compatibility. +- **Put `retryable` or `failover` on `LlmFailure`** — rejected because adapters report facts while deployment policy decides action. The same 429 may be retried in an interactive bundle and rejected in a cost-capped batch. +- **Retry forever while the caller remains active** — rejected because it gives one request unbounded cost and latency. Visible status makes bounded waiting understandable; it does not make an unlimited budget safe. +- **Log retry status only through the process logger** — rejected because process logs do not reconstruct session behavior and cannot drive replayed UI state. +- **Keep only flat codes** — rejected because retry delay and provider request id are structured provider facts, and HTTP status is necessary for diagnosis when different wire failures share one stable code. + +## Acceptance criteria + +- `LlmFailure` is the single serializable payload for thrown, error-finish, and aborted-finish final-adapter failures; normalization preserves stable code, status, retry delay, branded provider request id, error cause, and caller-abort versus adapter-timeout classification where available. +- An adapter-thrown `Error` reaches `agent/request-error` as the exact same object while its sidecar `LlmFailure` reaches the adjacent argument; tests retain the existing identity assertion for extensible and frozen third-party errors. +- DeepSeek and pi-ai adapter tests cover representative 400, 401/403, 429, 5xx, connection, malformed/truncated stream, timeout, abort, retry-after seconds/date, request-id, and unknown-SDK-error paths without recovery policy parsing message text. +- Pi-ai performs one wire attempt per adapter call, and a wire-level test rejects any regression that silently restores SDK retries. +- `agent/request-error` carries current failure facts plus immutable prior-retried failure facts; a success clears that history, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets. +- `dsh-llm-retry` validates every config field at Loader startup, delegates all ineligible paths with `next()`, and makes at most `maxTransientRetries + 1` provider requests when no other policy applies. +- HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, emits no retry decision after disposal, and leaves no timer or promise alive. +- Pure unit tests cover transient-code selection, exponential backoff and jitter bounds, valid and over-cap `Retry-After`, exhausted budgets, deterministic timer/random seams, and abort during backoff. +- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new step, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery. +- The partial-chunk integration test proves failed chunks remain attributed to the failed step, no assistant message or tool side effect is committed for that step, and the successful retry has distinct provenance. +- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and has a production UI consumer with keyless ACP or TUI snapshot coverage for scheduled retry, cancellation during delay, and eventual success or exhaustion. +- Idle-watchdog tests prove the stable signal is rearmed only while `next()` is outstanding, disarmed during consumer think time and in `finally`, and classified separately from a total-call deadline and an earlier caller abort; adapter tests prove the signal stops the underlying request rather than merely detaching it. +- Direct `ctx.llm.stream()` callers remain single-attempt and receive the same structured failure facts. +- The architecture LLM section, the implemented request-recovery and timeout notes, agent-loop and adapter READMEs, package catalogs, example configuration, persistence catalog, and testing documentation are updated in the implementation change; all generated outputs and bilingual counterparts required by those files are refreshed together. + +## Risks + +- A retry can duplicate provider billing even when no chunk arrived; the finite attempt budget limits but cannot remove that risk. +- Provider SDKs may hide status or retry headers. Those adapters must use `UNKNOWN` or a stable coarse code rather than infer policy from fragile text. +- Durable retry events expand the session protocol and UI state machine. Shipping the event and its consumer together prevents an unused telemetry vocabulary, but later schema changes still require persistence and replay work. +- Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots must make the transition explicit. +- Adapter-local timeout enforcement can drift across transport libraries. Contract tests at the termination boundary are required for both implementations. +- Multiple recovery plugins add their finite budgets. Their classifiers should remain disjoint; an overlapping classifier is registration-order policy and must be documented and tested by the plugins that introduce it. + +## Related + +- [Structured error taxonomy](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md) owns stable machine-routable codes and cause chaining. +- [Reconstructable requests](../../implemented/architecture/2026-07-05-reconstructable-requests.md) makes provider/model and complete request inputs durable before dispatch. +- [Timeout deadline library](../../implemented/architecture/2026-07-06-timeout-deadline-library.md) separates shared deadline classification from capability-owned termination. +- [After-call compaction pressure and context-overflow recovery](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md) owns the current closed-step request-recovery seam and bounded overflow retry. +- [Provider-routed LLM adapters](../../implemented/architecture/2026-07-14-provider-routed-llm-adapters.md) owns explicit provider/model routing and the one-adapter-per-provider invariant. diff --git a/.agents/notes/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md b/.agents/notes/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md deleted file mode 100644 index 9e5ad308be..0000000000 --- a/.agents/notes/proposed/architecture/2026-06-21-unstable-llm-api-recovery.md +++ /dev/null @@ -1,219 +0,0 @@ -# Agent Note: Treat unstable LLM APIs as a first-class failure mode - -Status: proposed - -## Problem - -LLM APIs are not a stable local function call. They rate-limit, overload, return 5xx/502/503 from gateways, close streaming sockets before `[DONE]`, emit malformed or provider-specific error payloads, hang mid-stream, surface SDK errors as in-band events, and sometimes require a delayed retry using `Retry-After`. The harness currently contains good error containment, but it does not yet treat this API instability as a first-class design problem. - -`dsh-llm` defines two sanctioned adapter failure paths - throw from `stream()` or end with `finish { kind: 'error' | 'aborted' }` - and downstream consumers are expected to treat both as failed model calls. That was the right MVP containment baseline, documented in [the architecture](../../../../docs/architecture.md) and reinforced by [the twin-adapter RFC](../../implemented/architecture/2026-06-13-twin-llm-adapters.md). It is not enough for callers that depend on an unstable remote model API for every turn, and it forces every caller to understand two failure delivery mechanisms. - -The audit found three load-bearing gaps. - -- `LlmError` and `FinishReasonMap.error` carry only `message`, `code`, and sometimes HTTP `status` ([packages/llm/llm/src/index.ts](../../../../packages/llm/llm/src/index.ts), [packages/llm/llm/src/types.ts](../../../../packages/llm/llm/src/types.ts)). A caller cannot reliably tell "retry this after 800 ms on the same endpoint", "fail over to another route for the same model", "ask the user for credentials", "never retry because the request is invalid", "provider truncated the stream after committed output", or "adapter protocol bug" without provider-specific heuristics. -- The `llm/stream` waterfall is documented as the place for retry/routing/caching, but its value is one raw `AsyncIterable`. A listener can technically catch an API error and call `next()` again, but once it has yielded chunks, callers may already have rendered them, buffered them as output, or executed side effects based on completed tool calls. Retrying after that point can concatenate chunks from two provider responses into one apparent model output. The current surface has no canonical way to say "the tokens you saw were tentative; this response timed out, so discard them and restart." -- The adapter registry is one adapter per model name. That makes "logical model" and "concrete API route" the same thing, so the service has no vocabulary for "same model through another endpoint or SDK", "same provider region with different health", "fallback route with compatible capabilities", provider request ids, or per-route backoff state. - -The result is a package that can surface an unstable LLM API failure, but cannot make principled recovery decisions for its callers. Ordinary provider turbulence should not require every consumer to reinvent retries around an unsafe stream boundary. - -## Proposal - -Introduce an LLM-call v2 contract centered on API-instability recovery: classify provider/API failures, separate provider responses from committed model output, route logical models through recoverable API routes, and make conservative retry/failover the default behavior in `dsh-llm`. Because the harness is unreleased, this should be a breaking cleanup rather than a compatibility layer around the underspecified v1 surface. - -### 1. Replace flat error codes with a serializable `LlmFailure` - -Keep `HarnessError` as the common thrown-error base, but make LLM failures carry a structured, JSON-serializable payload. `code` remains a stable leaf label for logs and provider-specific matching; retry/failover policy branches on the structured fields. - -```ts -type LlmFailureClass = - | 'auth' - | 'rate-limit' - | 'quota' - | 'invalid-request' - | 'unsupported' - | 'timeout' - | 'transport' - | 'provider-overloaded' - | 'provider-unavailable' - | 'provider-bug' - | 'protocol' - | 'safety' - | 'aborted' - | 'unknown' - -type LlmFailurePhase = - | 'request-build' - | 'connect' - | 'response-headers' - | 'stream' - | 'finish' - -interface LlmFailure { - message: string - code: string - class: LlmFailureClass - phase: LlmFailurePhase - retryable: boolean - failover: 'never' | 'same-model' | 'compatible-model' - partialOutput: 'none' | 'uncommitted' | 'committed' - provider?: string - routeId?: string - model?: string - wireModel?: string - status?: number - retryAfterMs?: number - requestId?: string -} -``` - -`LlmError` should carry `failure: LlmFailure`; `FinishReasonMap.error` should carry the same payload instead of a parallel `{ message, code? }` shape. Adapter-thrown errors and in-band finish errors are input forms to the recovery layer. The public lifecycle stream should convert classified LLM API failures into terminal lifecycle events rather than requiring callers to catch thrown provider errors. The failure payload is serializable so callers can log or persist it if they choose, while the internal thrown `LlmError` may still carry a non-serializable `cause` chain for local debugging. - -Adapters are responsible for faithful provider/API classification at their boundary: HTTP status, `Retry-After`, provider request id headers, SDK error type, timeout vs caller abort, malformed SSE, missing `[DONE]`, unknown finish reason, and unsupported local request shape. The current pi-ai adapter's regex over message text is acceptable only as a temporary fallback when the SDK hides the real status; the adapter should prefer structured SDK/provider fields when available. - -### 2. Split provider responses from committed model output - -Make "response" a first-class boundary in `ctx.llm.stream()`. An adapter streams one provider API response. The LLM service runs zero or more responses according to recovery policy and exposes one canonical response-lifecycle stream to consumers. Convenience APIs may expose a committed-or-failed result for simple callers, but that result must be derived from the lifecycle stream, not a parallel contract. - -The primary response id is generated by the harness before the adapter call starts. Provider response ids and request ids are metadata attached when known; they are not the primary key because providers may omit them, report them only after the stream starts, reuse them in surprising ways, or fail before one exists. - -The important invariant: chunks from a failed response must never be silently spliced together with chunks from a later response as one apparent model result. Token deltas from a response are tentative until that response reaches a committing terminal finish (`stop`, `tool-calls`, or `max-tokens`). The lifecycle stream must be able to report that tentative tokens were shown live, then discarded because the response timed out, disconnected, or otherwise failed before commit. - -The event vocabulary should keep the familiar `assistant/chunk` concept but stop pretending every chunk is already final output. A possible spelling is: - -```ts ignore-check -type LlmStreamEvent = - | { type: 'response/start'; responseId: ResponseId; responseIndex: number; routeId: string } - | { type: 'assistant/chunk'; responseId: ResponseId; commitment: 'uncommitted'; chunk: StreamChunk } - | { type: 'response/interrupted'; responseId: ResponseId; failure: LlmFailure; scheduledRetryMs?: number } - | { type: 'response/failed'; responseId?: ResponseId; failure: LlmFailure } - | { type: 'response/committed'; responseId: ResponseId; message: Message; finish: FinishReason; usage?: TokenUsage } - -type LlmCallOutcome = - | { type: 'committed'; responseId: ResponseId; message: Message; finish: FinishReason; usage?: TokenUsage } - | { type: 'failed'; responseId?: ResponseId; failure: LlmFailure } -``` - -The implementation may choose the exact names, but the type shape should make the state transition obvious: assistant chunks start uncommitted, then the enclosing response becomes interrupted/discarded, failed, or committed. - -- **Lifecycle assistant chunks.** The lifecycle stream should make the old ambiguity explicit: these events are assistant chunk messages, but each one belongs to a response and has a commitment state. Most arrive as uncommitted live UI state; a response that fails before commit marks them interrupted/discarded, and a response that reaches a committing finish lets the UI mark that response committed. -- **Retry before commit.** If an API response fails before a committing finish, the service may retry or fail over and exclude the failed response from the committed result, while surfacing response diagnostics separately. -- **Commit on terminal finish.** Once a response reaches a committing finish, the response owns the visible result. `dsh-llm` emits a `response/committed` event carrying the fully assembled assistant `Message`, final `FinishReason`, usage, and response metadata. Callers that persist messages, execute tool calls, or otherwise take side effects should use this committed event rather than rebuilding output from lifecycle chunks. -- **Terminal failure.** If recovery reaches a non-retryable failure, is aborted by the caller, or otherwise stops without a committing finish, `dsh-llm` emits `response/failed` carrying the final `LlmFailure` and ends the lifecycle stream normally. Throwing is reserved for defects outside the classified LLM API failure contract. -- **Fail after commit.** If a later failure is ever observable after commit, the lifecycle reports `response/failed` with `partialOutput: 'committed'`; automatic retry is not allowed unless a later RFC designs an explicit continuation/repair protocol. - -This means replacing the single overloaded raw-chunk `llm/stream` waterfall with a lifecycle stream and narrower hooks: one around a single provider response, one around recovery policy decisions, and one around convenience APIs that only expose the terminal outcome. Names are implementation details for the follow-up PR, but the semantics are not: plugins must be able to wrap "one API response" without pretending they can safely retry already-committed chunks. - -### 3. Route logical models through recoverable API routes - -Separate the logical model a caller requests from the concrete provider route that serves a response. Replace "one adapter per model name" with route registration, for example: - -```ts ignore-check -ctx.llm.registerRoute({ - routeId: 'deepseek-direct:deepseek-v4-flash', - model: 'deepseek-v4-flash', - wireModel: 'deepseek-v4-flash', - provider: 'deepseek', - adapter, - priority: 0, - capabilities: { tools: true, reasoning: true, images: false, prefill: false }, -}) -``` - -`GenerateOptions.model` remains the logical model. The service resolves it to a route for each API response, records the route in failure/response diagnostics, and can retry on the same route or fail over to another route with compatible capabilities. Duplicate model names become normal; duplicate route ids are the conflict. This is the smallest vocabulary that can express direct endpoint vs SDK-backed endpoint, regional endpoints, and future fallback models without making every caller own routing. - -The route registry must keep the lifecycle guarantees of the current adapter registry: `registerRoute()` is effect-scoped, returns a disposer, and has an HMR-safety test proving disposal removes the route. It should not preserve `llm/adapter-change`; if [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82) lands first, that event is already gone, and the route registry should not reintroduce it without a concrete consumer. - -The new ids should follow the branded-id policy. `ResponseId`, `RouteId`, and the logical/wire model ids cross package boundaries and are easy to swap accidentally, so the implementation should deliberately brand or explicitly decline to brand each one in line with `2026-06-20-branded-ids` and its implementation stack ([PR #84](https://github.com/deepseek-ai/deepseek-harness/pull/84)). - -### 4. Put default API recovery policy in `dsh-llm` - -Adapters should not perform hidden SDK retries unless those retries are surfaced as response lifecycle events with classified failures. The service owns the default policy so every consumer gets the same behavior and the same audit trail. - -Default policy should be conservative: - -- Retry transient API failures (`rate-limit`, `timeout`, `transport`, `provider-overloaded`, `provider-unavailable`) only before committed output, and keep retrying until the caller aborts or the failure class changes to a non-retryable one. -- Honor `retryAfterMs` up to `maxRetryDelayMs`, otherwise use bounded exponential backoff with jitter. The same cap applies to both provider-supplied retry hints and ordinary exponential backoff; diagnostics record whether the delay source was `provider-retry-after` or `exponential-backoff`. -- Treat 429/408/409/425/500/502/503/504 and connection resets as potentially recoverable unless the provider payload says otherwise; treat 400/401/403, unsupported local options, caller abort, and adapter protocol bugs as non-retryable. -- Fail over only when the failure says failover is safe and the candidate route advertises compatible capabilities for the request (`tools`, reasoning passback, images, prefill, stop sequences, strict tools). -- Share the caller's `AbortSignal` across the whole recovered call, and expose per-response timeouts as explicit policy. A stuck stream must time out in a controlled way instead of hanging the turn forever. -- Surface every retry decision to the UI with retry count, backoff delay, route, and failure summary, so an actively watching user can tell the agent is waiting on provider capacity instead of frozen. - -The policy should be configurable through a typed service option and an event/waterfall seam so product plugins can adjust timing/backoff details, but the default retry posture is not opt-in: a basic agent should keep recovering from retryable 429/5xx/connectivity noise until cancelled. - -The zero-config defaults should be sensible production behavior, not placeholders: - -```ts -const defaultLlmRecoveryConfig = { - maxResponses: 'unbounded', - maxElapsedMs: 'unbounded', - connectTimeoutMs: 15_000, - responseHeaderTimeoutMs: 60_000, - streamIdleTimeoutMs: 5 * 60_000, - initialBackoffMs: 200, - maxRetryDelayMs: 10 * 60_000, - jitterRatio: 0.1, -} -``` - -The retry-delay cap is deliberate. The survey found mixed precedent: Codex parses retry delays out of streamed OpenAI rate-limit error messages and uses that requested delay, but Codex also has finite stream retry counts; the official OpenAI and Anthropic TypeScript SDKs parse `retry-after-ms`, `Retry-After` seconds, and `Retry-After` dates and then sleep for the provider-specified duration; the official OpenAI and Anthropic Python SDKs only honor `Retry-After` when it is greater than zero and at most 60 seconds, otherwise falling back to ordinary exponential backoff. Because this RFC's default retry posture is unbounded, blindly honoring a multi-hour provider delay can make the agent look dead, while ignoring the hint entirely can retry too aggressively. The service should therefore record both `providerRetryAfterMs` and `scheduledRetryMs`, cap the scheduled sleep at `maxRetryDelayMs`, and surface that choice to the UI. - -The service cannot reliably infer whether the user is actively watching or away from the keyboard, so the default should not fail a retryable model call merely because a short interactive budget expired. A clear UI can make long waits tolerable even in interactive sessions: "retried 8 times; next retry in 10 minutes" is better than silently failing recoverable provider turbulence and asking the user to resubmit. - -### 5. Define the caller contract, not the product transcript - -This RFC is deliberately about the `dsh-llm` API and how callers use it, not about the final transcript/event architecture of the product. The LLM package should guarantee these caller-visible semantics: - -- `ctx.llm.stream()` is the live response-lifecycle API. It reports response starts, uncommitted assistant chunks, retries/backoff, interruptions/discards, terminal failures, and the one committed result. -- `response/committed` is the only event that makes model output safe for history or side effects. It carries the assembled `Message`, finish reason, usage, response id, route metadata, and provider ids known to the service. -- `response/failed` is the terminal event for classified failures. Callers should not need `try`/`catch` to learn that a provider was rate-limited, unavailable, misconfigured, aborted, or otherwise unable to produce a committed response. -- Response diagnostics are JSON-serializable so callers can store, display, or ignore them. The LLM package does not decide whether those diagnostics become session events, agent events, telemetry rows, or UI-only state. -- Any assembled convenience helper that survives or is reintroduced returns a terminal union (`committed` or `failed`) rather than throwing for classified LLM failures. It must be derived from the lifecycle stream so recovery semantics stay single-sourced. - -The session log shape, agent event taxonomy, ACP rendering, snapshot/replay fixtures, and whether live uncommitted chunks are ever durably recorded are downstream integration decisions. So is the fate of today's assembled public helper methods: [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82) implements the proposed removal of `generate()`, `streamBlocks()`, `GenerateResult`, and `llm/generate`, and this RFC should not resurrect them without a real caller. This RFC should constrain downstream work only by the LLM API contract above. - -The current simplification stack was checked while drafting this proposal. [PR #83](https://github.com/deepseek-ai/deepseek-harness/pull/83) and [PR #85](https://github.com/deepseek-ai/deepseek-harness/pull/85) do not change this RFC's LLM API assumptions. [PR #86](https://github.com/deepseek-ai/deepseek-harness/pull/86) does matter for later integration because it folds durable token usage onto `assistant/message` and operational errors onto `turn/end.reason`; if it lands first, the LLM recovery implementation should still stop at the `dsh-llm` lifecycle contract here and let the agent/session layer decide how committed usage and terminal failures map onto those load-bearing product events. - -## Out of scope - -This RFC does not propose silent mid-stream continuation after user-visible output. That requires a separate model-history design: either provider-supported prefill/continuation, a recovery prompt that explicitly shows the partial assistant output, or a UI affordance that marks the partial answer as failed and asks the model to continue in a new step. Splicing two API responses into one assistant message is rejected. - -This RFC also does not solve semantic model-output repair: malformed tool-call JSON, refusal handling, or content-filter fallbacks. Those may use the same failure vocabulary later, but they are higher-level agent behaviors, not unstable-API recovery. - -This RFC does not decide whether product UIs consume LLM lifecycle events directly, through agent events, or through session events. It also does not decide which response diagnostics belong in the durable session log. Those decisions belong in narrower integration RFCs once the `dsh-llm` contract exists. - -## Alternatives considered - - - -## Acceptance criteria - -- Adapter-thrown `LlmError`s and in-band finish errors carry one structured, JSON-serializable `LlmFailure` payload. -- Recovery policy can distinguish retry, failover, credential/user-action, unsupported request, caller abort, adapter/protocol bug, and post-commit partial stream failure without parsing message text. -- `ctx.llm.stream()` exposes the response lifecycle as the canonical stream, including terminal `response/failed` events for classified failures; convenience APIs are derived views for callers that only want the terminal outcome. -- `response/committed` carries the assembled assistant `Message`; callers do not need to rebuild committed output from lifecycle chunks. -- Any assembled convenience API that survives or is reintroduced returns a committed/failed union derived from the lifecycle stream, rather than throwing for classified LLM failures. -- The LLM service has an explicit API-response boundary; no retry path can present output from two provider responses as one committed assistant result. -- The route registry allows multiple concrete API routes for one logical model and records the selected route on responses/failures. -- `registerRoute()` is effect-scoped, returns a disposer, and has an HMR-safety test proving route cleanup; `llm/adapter-change` is not reintroduced unless a concrete consumer needs it. -- New LLM ids are deliberately branded or explicitly left unbranded according to the branded-id policy, with `ResponseId`, `RouteId`, and logical/wire model ids decided together. -- Default recovery retries transient pre-commit failures with bounded backoff, honors provider retry-after hints, times out stuck streams, disables hidden SDK retries or surfaces them as response lifecycle events, and never retries after committed chunks without an explicit continuation design. -- Unit tests cover thrown errors and finish-error chunks through `dsh-llm`, retry-before-first-commit, failover to a second compatible route, unbounded retry status/backoff visibility, abort during backoff, stream timeout, and the "partial chunks then failure does not retry/splice" invariant. -- Adapter tests classify representative HTTP statuses, retry-after headers, request ids, malformed/truncated SSE streams, SDK in-stream errors, caller aborts, and unsupported options into `LlmFailure`. -- Docs updated in the same change: [the architecture LLM section](../../../../docs/architecture.md), [the LLM adapter cookbook](../../../../docs/cookbook/adding-an-llm-adapter.md), and the LLM package README. - -## Risks - -- **More surface area in the LLM core.** API recovery adds policy, route state, response diagnostics, and tests. That complexity belongs in `dsh-llm` because every consumer otherwise reinvents it around the same unsafe stream boundary. -- **Committed result lags live UI.** Safe recovery means callers cannot treat streamed tokens as final assistant output until the response commits. UIs can still stream eagerly from lifecycle assistant chunks, but side-effecting consumers must wait for `response/committed`. -- **Breaking adapter churn.** Existing adapters will change from "stream chunks or throw a flat `LlmError`" to "stream one classified API response." Pre-release rules favor the correct seam over shims. -- **Route compatibility is easy to overclaim.** A route must advertise concrete capabilities, and failover must check the request actually fits them. "Same model name" is not enough when one route lacks strict tools, reasoning passback, images, stop sequences, or prefill. - -## Related - -- Builds on [Provider-neutral content-block vocabulary](../../implemented/architecture/2026-06-11-content-block-vocabulary.md): the content vocabulary stays provider-neutral; this adds a provider-neutral failure/recovery vocabulary beside it. -- Revises the scope implied by [Two LLM adapters as a design-verification twin](../../implemented/architecture/2026-06-13-twin-llm-adapters.md): the twin validated chunk shape and error delivery paths, but it also exposed that delivery paths are not enough for unstable API recovery. -- Extends [Structured error taxonomy](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md): `HarnessError.code` was the foundation; LLM API recovery needs a richer payload because retry/failover policy cannot safely branch on one flat string. -- Coordinates with [PR #82](https://github.com/deepseek-ai/deepseek-harness/pull/82), which implements the `drop-unconsumed-llm-assembled-surfaces` and `drop-unconsumed-llm-adapter-change-event` simplification RFCs. If that PR lands first, this RFC starts from a narrower `dsh-llm`: no `generate()`, no `streamBlocks()`, no `GenerateResult`, no `llm/generate`, and no `llm/adapter-change`. Recovery should build on that baseline rather than revive removed convenience or change-notification surfaces speculatively. -- Is orthogonal to [PR #81](https://github.com/deepseek-ai/deepseek-harness/pull/81), which proposes provider-request app attribution headers. Recovery route metadata and provider request construction can carry attribution policy later, but this RFC does not define request headers. -- Coordinates with [PR #84](https://github.com/deepseek-ai/deepseek-harness/pull/84), which implements the branded-id RFC. The new response/route/model ids introduced here are exactly the sort of cross-boundary ids that need an explicit branding decision before implementation. -- Coordinates with [PR #86](https://github.com/deepseek-ai/deepseek-harness/pull/86), which implements the `collapse-trace-only-session-events` simplification RFC. If that stack lands first, downstream recovery integration should map committed usage and terminal error facts onto the surviving load-bearing session events instead of reintroducing standalone trace-only records from inside `dsh-llm`. From 3b0b0cefebb82901ed27ef64c0f4ff1d233ff54a Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 03:34:19 +0800 Subject: [PATCH 06/15] feat: implement bounded LLM request recovery --- ...2026-06-21-bounded-llm-request-recovery.md | 41 +- .../2026-07-06-timeout-deadline-library.md | 25 +- ...-14-provider-routed-llm-adapters.i18n.yaml | 4 +- ...2026-07-14-provider-routed-llm-adapters.md | 6 +- ...6-07-14-provider-routed-llm-adapters.zh.md | 6 +- docs/architecture.md | 8 +- docs/config-catalog.md | 52 +- docs/cordis-catalog/events.md | 17 +- docs/cordis-catalog/services.md | 2 +- docs/core-data-structures/core.md | 24 +- docs/core-data-structures/llm-streaming.md | 24 +- docs/core-data-structures/session.md | 10 +- docs/event-producer-consumer.md | 10 +- docs/module-graph.md | 24 +- docs/persistence-catalog.md | 48 +- docs/testing.md | 2 + .../snapshots/error-finish/session.jsonl | 2 +- .../error-finish/stdout.expected.jsonl | 1 + packages/compact/compact-basic/package.json | 1 + packages/compact/compact-basic/src/index.ts | 7 +- .../compact/compact-basic/src/summarizer.ts | 10 +- .../compact-basic/tests/compact-basic.spec.ts | 12 +- .../tests/compact-loop-repro.spec.ts | 102 +++- .../cordis/tool-cordis/src/api-catalog.ts | 16 +- packages/core/agent-loop/README.md | 3 +- packages/core/agent-loop/src/loop.ts | 52 +- .../tests/contract-regressions.spec.ts | 37 +- .../agent-loop/tests/coverage-edges.spec.ts | 7 +- .../agent-loop/tests/request-recovery.spec.ts | 91 +++- packages/core/agent/README.md | 2 +- packages/core/agent/src/types.ts | 7 +- packages/core/session/README.md | 6 +- packages/core/session/src/types.ts | 10 +- packages/examples/acp-demo/README.md | 1 + packages/examples/acp-demo/src/index.ts | 3 + packages/examples/agent-spine-demo/README.md | 9 +- .../examples/agent-spine-demo/package.json | 4 +- .../examples/agent-spine-demo/src/index.ts | 11 +- .../agent-spine-demo/tests/agent-core.spec.ts | 45 +- .../examples/agent-spine-demo/tsconfig.json | 3 + packages/examples/cli-demo/README.md | 3 +- packages/examples/cli-demo/src/cli.ts | 12 +- packages/examples/cli-demo/src/index.ts | 3 + packages/examples/cli-demo/tests/cli.spec.ts | 26 + packages/examples/stdio-demo/README.md | 1 + packages/examples/stdio-demo/src/index.ts | 3 + packages/llm/README.md | 3 +- packages/llm/llm-deepseek/README.md | 7 +- packages/llm/llm-deepseek/package.json | 2 + packages/llm/llm-deepseek/src/adapter.ts | 98 +++- packages/llm/llm-deepseek/src/index.ts | 7 +- packages/llm/llm-deepseek/src/translate.ts | 5 +- .../llm/llm-deepseek/tests/adapter.spec.ts | 198 +++++++- .../llm/llm-deepseek/tests/translate.spec.ts | 3 +- packages/llm/llm-deepseek/tsconfig.json | 3 + packages/llm/llm-pi-ai/README.md | 11 +- packages/llm/llm-pi-ai/package.json | 2 + packages/llm/llm-pi-ai/src/adapter.ts | 61 ++- packages/llm/llm-pi-ai/src/config.ts | 37 +- packages/llm/llm-pi-ai/src/stream.ts | 18 +- packages/llm/llm-pi-ai/tests/adapter.spec.ts | 159 +++++- packages/llm/llm-pi-ai/tests/convert.spec.ts | 50 +- .../llm/llm-pi-ai/tests/provider-apis.e2e.ts | 2 +- .../llm/llm-pi-ai/tests/sdk-options.spec.ts | 35 ++ packages/llm/llm-pi-ai/tsconfig.json | 3 + packages/llm/llm-retry/README.md | 39 ++ packages/llm/llm-retry/package.json | 47 ++ packages/llm/llm-retry/src/index.ts | 211 ++++++++ .../tests/loader-composition.spec.ts | 124 +++++ .../llm/llm-retry/tests/persistence.spec.ts | 57 +++ packages/llm/llm-retry/tests/retry.spec.ts | 453 ++++++++++++++++++ packages/llm/llm-retry/tsconfig.json | 33 ++ packages/llm/llm/README.md | 11 +- packages/llm/llm/src/adapter-failure.ts | 83 +++- packages/llm/llm/src/brand.ts | 15 +- packages/llm/llm/src/error.ts | 16 + packages/llm/llm/src/index.ts | 48 +- packages/llm/llm/src/types.ts | 20 +- packages/llm/llm/tests/properties.spec.ts | 5 +- packages/llm/llm/tests/service.spec.ts | 168 +++++++ .../llm-replay/tests/llm-replay.spec.ts | 2 +- packages/ui/acp/README.md | 4 +- packages/ui/acp/package.json | 2 + packages/ui/acp/src/index.ts | 22 +- packages/ui/acp/tests/harness.ts | 2 +- packages/ui/acp/tests/stream-update.spec.ts | 27 ++ packages/ui/acp/tests/turns.spec.ts | 9 + packages/ui/acp/tsconfig.json | 3 + packages/ui/stdio/README.md | 2 +- packages/ui/stdio/package.json | 2 + packages/ui/stdio/src/index.ts | 26 +- packages/ui/stdio/tests/stdio.spec.ts | 45 ++ packages/ui/stdio/tsconfig.json | 3 + packages/ui/tui/README.md | 2 +- packages/ui/tui/package.json | 2 + packages/ui/tui/src/index.ts | 75 ++- packages/ui/tui/tests/harness.ts | 5 +- .../snapshots/retry-cancelled.expected.txt | 48 ++ .../snapshots/retry-exhausted.expected.txt | 45 ++ .../snapshots/retry-recovered.expected.txt | 49 ++ .../snapshots/retry-scheduled.expected.txt | 45 ++ packages/ui/tui/tests/tui.snapshot.ts | 81 ++++ packages/ui/tui/tests/tui.spec.ts | 75 ++- packages/ui/tui/tsconfig.json | 3 + packages/util/timeout/README.md | 7 +- packages/util/timeout/src/index.ts | 76 +++ packages/util/timeout/tests/timeout.spec.ts | 87 +++- pnpm-lock.yaml | 67 +++ python/sdk-runtime/package.json | 1 + scripts/gen-cordis-catalog.ts | 1 + scripts/type-equiv.manifest.json | 2 + tsconfig.build.json | 1 + tsconfig.json | 1 + website/zh-CN/api/harness/events.md | 18 +- website/zh-CN/api/harness/llm.md | 10 +- 115 files changed, 3311 insertions(+), 366 deletions(-) rename .agents/notes/{proposed => implemented}/architecture/2026-06-21-bounded-llm-request-recovery.md (82%) create mode 100644 packages/llm/llm-pi-ai/tests/sdk-options.spec.ts create mode 100644 packages/llm/llm-retry/README.md create mode 100644 packages/llm/llm-retry/package.json create mode 100644 packages/llm/llm-retry/src/index.ts create mode 100644 packages/llm/llm-retry/tests/loader-composition.spec.ts create mode 100644 packages/llm/llm-retry/tests/persistence.spec.ts create mode 100644 packages/llm/llm-retry/tests/retry.spec.ts create mode 100644 packages/llm/llm-retry/tsconfig.json create mode 100644 packages/ui/tui/tests/snapshots/retry-cancelled.expected.txt create mode 100644 packages/ui/tui/tests/snapshots/retry-exhausted.expected.txt create mode 100644 packages/ui/tui/tests/snapshots/retry-recovered.expected.txt create mode 100644 packages/ui/tui/tests/snapshots/retry-scheduled.expected.txt diff --git a/.agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md similarity index 82% rename from .agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md rename to .agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md index e57cffe687..ad476b8b35 100644 --- a/.agents/notes/proposed/architecture/2026-06-21-bounded-llm-request-recovery.md +++ b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md @@ -1,6 +1,6 @@ # Agent Note: Bounded recovery for transient LLM request failures -Status: proposed +Status: implemented ## Problem @@ -8,19 +8,19 @@ Status: proposed That boundary is already safe for another request attempt. Raw `assistant/chunk` events carry the failed `turn` and `step`, message derivation ignores them unless a successful `assistant/message` cites them, tool calls are dispatched only after a successful terminal finish and assembly, and a retry opens a new numbered step from the durable log. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate. -Three narrower gaps remain. +The prior boundary left three narrower gaps. - Provider failures retain only a message and usually a code. HTTP status, retry delay, and provider request id are discarded or recoverable only through provider-specific error objects, so generic recovery cannot make or explain a decision without parsing text. -- Retry ownership differs by adapter. The hand-written DeepSeek adapter makes one attempt, while pi-ai profiles can enable opaque library retries. Combining hidden transport retries with a future `agent/request-error` listener would multiply attempts and omit intermediate failures from the session log. +- Retry ownership differs by adapter. The hand-written DeepSeek adapter makes one attempt, while pi-ai profiles can enable opaque library retries. Combining hidden transport retries with an `agent/request-error` listener would multiply attempts and omit intermediate failures from the session log. - A recovered failure has no durable status fact. The failed step and chunks remain reconstructable, but an observer cannot tell whether the agent is deliberately backing off, for how long, or why. A long silent wait looks like a stalled loop. The goal is bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer. -## Proposal +## Decision ### Preserve failure facts without embedding policy -Add one JSON-serializable `LlmFailure` payload to `@deepseek-ai/dsh-llm`: +`@deepseek-ai/dsh-llm` exports one JSON-serializable `LlmFailure` payload: ```ts ignore-check type ProviderRequestId = Branded<'ProviderRequestId'> @@ -46,9 +46,9 @@ The initial shared transient-code set is intentionally small: the adapters' exis ### Put retry policy on the existing failed-step seam -Add a function plugin, `@deepseek-ai/dsh-llm-retry`, that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow. +`@deepseek-ai/dsh-llm-retry` is a function plugin that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow. -Replace the scalar `retryAttempt` argument with the current `LlmFailure` and an immutable list of prior failures that led to another request attempt in this consecutive recovery sequence. `dsh-llm-retry` counts only prior failures whose codes are in its configured transient set, while `dsh-compact-basic` counts only prior context-overflow failures. A successful model request clears the history as it clears the current scalar. Alternating transient and context-overflow failures therefore consume their owning policy budgets independently; the maximum request count is one plus the sum of the finite budgets of the loaded recovery policies. +The `agent/request-error` seam carries the current `LlmFailure` and an immutable list of prior failures that led to another request attempt in this consecutive recovery sequence. `dsh-llm-retry` counts only prior failures whose codes are in its configured transient set, while `dsh-compact-basic` counts only prior context-overflow failures. A successful model request clears the history. Alternating transient and context-overflow failures therefore consume their owning policy budgets independently; the maximum request count is one plus the sum of the finite budgets of the loaded recovery policies. The plugin resolves and validates this deployment configuration at load: @@ -78,15 +78,15 @@ The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot C Adapters perform one provider request per `stream()` call. The pi-ai adapter removes public `maxRetries` and `maxRetryDelayMs` profile fields and disables library retries; the hand-written adapter keeps its current single-attempt behavior. This prevents an SDK budget from multiplying the agent budget and ensures every transient retry is represented by a closed failed step plus `llm/retry`. -`ctx.llm.stream()` remains the raw one-attempt waterfall. Direct callers such as compaction summarization receive the structured failure but do not gain automatic retry, because they have no agent step boundary or general durable place to separate attempts. A future direct-call consumer may justify a buffering helper that retries only before emitting a chunk, but this proposal does not add one speculatively. +`ctx.llm.stream()` remains the raw one-attempt waterfall. Direct callers such as compaction summarization receive the structured failure but do not gain automatic retry, because they have no agent step boundary or general durable place to separate attempts. A future direct-call consumer may justify a buffering helper that retries only before emitting a chunk; this decision adds no such helper. ### Bound stalled streams where they can be stopped -Each adapter exposes a validated `streamIdleTimeoutMs` configuration field with the five-minute prior-art default cited above. The interval covers each outstanding iterator `next()` from demand to the next valid `StreamChunk`; time a consumer spends between `next()` calls is not provider idle time. +Each adapter exposes a validated `streamIdleTimeoutMs` configuration field with the five-minute prior-art default cited above. The interval is capped at Node's maximum timer delay so it cannot be clamped to one millisecond. It covers each outstanding iterator `next()` from demand to the next valid `StreamChunk`; time a consumer spends between `next()` calls is not provider idle time. -Extend `@deepseek-ai/dsh-timeout` with a rearmable idle-watchdog primitive. One stable local `AbortController` is fused with the caller signal and passed to the transport for the whole adapter call; each outstanding `next()` arms the watchdog, resolution disarms it, and the next demand rearms it. Timeout aborts that stable controller with a capability-owned `TimeoutReason`, and `finally` clears the timer. The adapter classifies its watchdog as `TIMEOUT` and an earlier upstream abort as `ABORTED`. The existing one-shot `deadline()` is not presented as a sliding timer. +`@deepseek-ai/dsh-timeout` exposes a rearmable idle-watchdog primitive. One stable local `AbortController` is fused with the caller signal and passed to the transport for the whole adapter call; each outstanding `next()` arms the watchdog, resolution disarms it, and the next demand rearms it. Timeout aborts that stable controller with a capability-owned `TimeoutReason`, and `finally` clears the timer. The adapter classifies its watchdog as `TIMEOUT` and an earlier upstream abort as `ABORTED`. The existing one-shot `deadline()` is not presented as a sliding timer. -The two adapters must prove termination at their actual boundaries. The hand-written adapter aborts its fetch/reader, and the pi-ai adapter maps the stable signal through the SDK only after a test proves the SDK stops the request. A timer that merely rejects a consumer promise while leaving the request running does not satisfy the contract. +Boundary tests prove termination at both actual transports. The hand-written adapter aborts its fetch/reader, and the pi-ai adapter maps the stable signal through the SDK and proves the SDK closes the response. A timer that merely rejects a consumer promise while leaving the request running does not satisfy the contract. ### Keep attempts separate in the existing log @@ -112,31 +112,30 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason` - **Log retry status only through the process logger** — rejected because process logs do not reconstruct session behavior and cannot drive replayed UI state. - **Keep only flat codes** — rejected because retry delay and provider request id are structured provider facts, and HTTP status is necessary for diagnosis when different wire failures share one stable code. -## Acceptance criteria +## Verification - `LlmFailure` is the single serializable payload for thrown, error-finish, and aborted-finish final-adapter failures; normalization preserves stable code, status, retry delay, branded provider request id, error cause, and caller-abort versus adapter-timeout classification where available. - An adapter-thrown `Error` reaches `agent/request-error` as the exact same object while its sidecar `LlmFailure` reaches the adjacent argument; tests retain the existing identity assertion for extensible and frozen third-party errors. - DeepSeek and pi-ai adapter tests cover representative 400, 401/403, 429, 5xx, connection, malformed/truncated stream, timeout, abort, retry-after seconds/date, request-id, and unknown-SDK-error paths without recovery policy parsing message text. -- Pi-ai performs one wire attempt per adapter call, and a wire-level test rejects any regression that silently restores SDK retries. +- Pi-ai pins the SDK option to zero retries and performs one observed wire attempt for a retryable provider response; separate tests make removing either boundary fail. - `agent/request-error` carries current failure facts plus immutable prior-retried failure facts; a success clears that history, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets. - `dsh-llm-retry` validates every config field at Loader startup, delegates all ineligible paths with `next()`, and makes at most `maxTransientRetries + 1` provider requests when no other policy applies. - HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, emits no retry decision after disposal, and leaves no timer or promise alive. - Pure unit tests cover transient-code selection, exponential backoff and jitter bounds, valid and over-cap `Retry-After`, exhausted budgets, deterministic timer/random seams, and abort during backoff. - Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new step, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery. - The partial-chunk integration test proves failed chunks remain attributed to the failed step, no assistant message or tool side effect is committed for that step, and the successful retry has distinct provenance. -- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and has a production UI consumer with keyless ACP or TUI snapshot coverage for scheduled retry, cancellation during delay, and eventual success or exhaustion. +- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and drives TUI retraction plus durable discarded-attempt markers in append-only ACP and stdio streams. Keyless snapshots cover scheduling, cancellation, success, and exhaustion. - Idle-watchdog tests prove the stable signal is rearmed only while `next()` is outstanding, disarmed during consumer think time and in `finally`, and classified separately from a total-call deadline and an earlier caller abort; adapter tests prove the signal stops the underlying request rather than merely detaching it. - Direct `ctx.llm.stream()` callers remain single-attempt and receive the same structured failure facts. -- The architecture LLM section, the implemented request-recovery and timeout notes, agent-loop and adapter READMEs, package catalogs, example configuration, persistence catalog, and testing documentation are updated in the implementation change; all generated outputs and bilingual counterparts required by those files are refreshed together. -## Risks +## Consequences -- A retry can duplicate provider billing even when no chunk arrived; the finite attempt budget limits but cannot remove that risk. -- Provider SDKs may hide status or retry headers. Those adapters must use `UNKNOWN` or a stable coarse code rather than infer policy from fragile text. +- Every transient recovery attempt is visible as a closed step plus `llm/retry`, and the bounded policy prevents hidden SDK retries from multiplying cost. A retry can still duplicate provider billing even when no chunk arrived; the finite attempt budget limits but cannot remove that risk. +- Provider SDKs may hide status or retry headers. Those adapters retain the stable facts they expose and otherwise use a coarse code rather than letting recovery policy parse fragile text. - Durable retry events expand the session protocol and UI state machine. Shipping the event and its consumer together prevents an unused telemetry vocabulary, but later schema changes still require persistence and replay work. -- Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots must make the transition explicit. -- Adapter-local timeout enforcement can drift across transport libraries. Contract tests at the termination boundary are required for both implementations. -- Multiple recovery plugins add their finite budgets. Their classifiers should remain disjoint; an overlapping classifier is registration-order policy and must be documented and tested by the plugins that introduce it. +- Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots pin the transition. +- Adapter-local idle enforcement stops stalled transports without counting consumer think time. Contract tests at each transport boundary guard against SDK drift. +- Multiple recovery plugins add their finite budgets. Their classifiers remain disjoint here; an overlapping classifier would be registration-order policy and must be documented and tested by the plugins that introduce it. ## Related diff --git a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md index 335581ecff..1c407d777d 100644 --- a/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md +++ b/.agents/notes/implemented/architecture/2026-07-06-timeout-deadline-library.md @@ -18,7 +18,7 @@ Each new external-process or network tool re-derived the same four things — cl ### The library surface -Three functions plus one reason type: +Four functions, one watchdog interface, and one reason type: ```ts ignore-check /** The internal reason attached to a timeout abort, so consumers can classify it after the fact. */ @@ -51,19 +51,34 @@ export function deadline( code: string, ): { signal: AbortSignal; [Symbol.dispose](): void } +/** A stable signal plus one-at-a-time, timer-guarded async-iterator demand. */ +export interface IdleWatchdog { + readonly signal: AbortSignal + next(iterator: AsyncIterator): Promise> + [Symbol.dispose](): void +} + +/** Arm only while one iterator `next()` is outstanding, then rearm on later demand. */ +export function idleWatchdog( + upstream: AbortSignal | undefined, + timeoutMs: number, + code: string, +): IdleWatchdog + /** Recover the TimeoutReason from an aborted signal (or error); `code` scopes the match to this deadline's timer. */ export function timeoutOf(x: AbortSignal | { reason?: unknown }, code?: string): TimeoutReason | undefined ``` -`deadline` fuses an upstream signal with a timer through `AbortSignal.any`, adds a typed `TimeoutReason`, and exposes disposable timer cleanup. Non-positive timeouts are an internal no-timeout sentinel for backend-owned background work; external hints pass through `clampTimeout` and must be positive and finite. Without a timer or upstream signal, the function returns a never-aborting signal with the same disposal shape. Providers translate timeout reasons into seam-specific results. `timeoutOf(signal, code)` scopes classification so an outer nested deadline is treated as upstream cancellation rather than the inner capability's timeout. +`deadline` fuses an upstream signal with a one-shot timer through `AbortSignal.any`, adds a typed `TimeoutReason`, and exposes disposable timer cleanup. Non-positive timeouts are an internal no-timeout sentinel for backend-owned background work; external hints pass through `clampTimeout` and must be positive and finite. Without a timer or upstream signal, the function returns a never-aborting signal with the same disposal shape. `idleWatchdog` instead requires a positive finite interval, keeps one stable fused signal for the entire stream, and arms its timer only while one iterator `next()` is outstanding; resolution disarms it, later demand rearms it, concurrent demand fails, and disposal clears the active arm. Providers translate timeout reasons into seam-specific results. `timeoutOf(signal, code)` scopes classification so an outer nested deadline is treated as upstream cancellation rather than the inner capability's timeout. ### The division of labor | Concern | Owner | |---|---| | Validate request hint and clamp default/max | `dsh-timeout` (`clampTimeout`) — pure arithmetic plus the shared positive-finite request contract | -| Arm timer, abort on deadline, carry reason, fuse with upstream cancel | `dsh-timeout` (`deadline`) | -| Clear the timer | `dsh-timeout` (`[Symbol.dispose]`) | +| Arm one-shot timer, abort on deadline, carry reason, fuse with upstream cancel | `dsh-timeout` (`deadline`) | +| Arm and rearm only around outstanding iterator demand | `dsh-timeout` (`idleWatchdog`) | +| Clear the timer | `dsh-timeout` (`[Symbol.dispose]` on either primitive) | | Classify the first abort reason after abort | `dsh-timeout` (`timeoutOf`) | | **Actually terminate the work** | the capability's implementation | | The default/max *values* | the capability's config | @@ -75,6 +90,7 @@ The signal only *notifies*; termination is always the listener's job, and the li - **web_fetch** — the tool stays validate-and-forward; the provider's hand-rolled controller + `setTimeout` + manual listener + `finally` + `signal.reason` recovery is replaced by provider-owned `deadline`/`timeoutOf`. A pre-aborted upstream signal still throws `WEB_ABORTED` up front; otherwise `fetch` runs against the fused `d.signal`, and `translateAbortOrNetwork` classifies a thrown error by the signal (`timeoutOf` → `WEB_FETCH_TIMEOUT`, else aborted → `WEB_ABORTED`, else network → `WEB_PROVIDER_ERROR`). The public error-code contract is unchanged, and `TimeoutReason` never crosses the web seam as the public error. - **bash** — `resolve()` clamps the request into an explicit spec. Foreground `run()` creates the deadline and passes its signal to process execution, whose existing abort listener performs the process-group kill. The executor classifies the first abort as timeout or cancellation. Background starts remain timeout-free and forward only upstream cancellation. +- **LLM adapters** — `dsh-llm-deepseek` and `dsh-llm-pi-ai` wrap actual transport iteration with `idleWatchdog`. The five-minute configured interval covers only outstanding provider demand, not time the downstream consumer spends between chunks. The stable signal reaches `fetch` or the SDK for the whole call, so timeout closes the underlying request and maps to `TIMEOUT`, while an earlier caller abort maps to `ABORTED`. ## Consequences @@ -82,6 +98,7 @@ The signal only *notifies*; termination is always the listener's job, and the li - `SpawnSpec.timeoutMs` and `SpawnOutcome.timedOut`/`aborted` were removed rather than kept as always-zero/always-false vestiges: with `runBash` owning no timer and the executor owning classification, they were read nowhere. This is the one deviation from the literal proposal shape (which passed `timeoutMs: 0` into `runBash`); an always-0 field read by nothing is dead weight under the per-file coverage gate. - web_fetch shed its bespoke controller/timer/listener/reason-recovery; the classifier now keys off the deadline signal (`timeoutOf` + `aborted`) rather than the thrown error's shape, which is robust across both the request-phase reject-with-reason and the read-phase bare-`AbortError`. - `AbortSignal.any` and `using`/`Symbol.dispose` enter the repo for the first time here (Node ≥ 24 baseline, already met). +- Model streams now share one rearmable timer contract without turning a sliding idle interval into a total-call deadline or charging consumer think time. The primitive still only notifies; adapter tests prove their transports observe its stable signal and terminate. Out of scope, named to mark the boundary: `web_search` can gain an optional model-facing `timeout_ms` once its tool-schema/snapshot coverage is planned; future ripgrep-backed fs discovery tools can consume the same provider-owned deadline shape once they exist; a `tools/execute` waterfall middleware could arm a default deadline for every tool call by driving `exec.signal` — that would be a plugin that *consumes* this library and still only notifies, the hard kill remaining each capability's job. diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml index 3ed38c1282..3a6b18cecf 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-14-provider-routed-llm-adapters.md: b7944bd31fdb5f63894e867d7c1224215d694f11 -2026-07-14-provider-routed-llm-adapters.zh.md: 7dcadf2521bab079e328b5f0d0a45185778b3b8d +2026-07-14-provider-routed-llm-adapters.md: 98205d18d07752e0cdba86d7cba80368d45fd816 +2026-07-14-provider-routed-llm-adapters.zh.md: c35225a86baf4c2d09732b5940abbc8046d365fb diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md index b7944bd31f..98205d18d0 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md @@ -28,7 +28,7 @@ A provider has exactly one adapter owner in a Cordis context. `dsh-llm-deepseek` ### Explicit pi-ai provider profiles -`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, timeouts, and retry settings. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback. +`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, SDK timeouts, and a Harness stream-idle timeout. Provider retry fields are deliberately absent: the adapter forces pi-ai's `maxRetries` to zero so one `stream()` call makes one visible provider attempt, while `dsh-llm-retry` owns bounded agent-level recovery. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback. The plugin registers all configured provider names against one `PiAiAdapter` in one all-or-nothing call. A request uses its provider to select the matching profile and finds its model in `getModels(provider)` to obtain the catalog descriptor. An unknown provider fails at plugin load; an unknown model fails before network I/O with `UNKNOWN_MODEL`. The catalog object is never mutated. When a profile supplies `baseURL`, the adapter clones the selected descriptor and overrides only `baseUrl`, so a private endpoint can retain pi-ai's API, capabilities, compatibility flags, context limits, and reasoning map. The private endpoint must implement the selected provider's protocol, and the model id must still exist in the installed pi-ai catalog. @@ -75,14 +75,14 @@ The on-disk session format remains the pre-release pinned version `0`, with no c - Provider names are deployment-wide route ownership keys: two providers may use the same model string, but mounting two adapters for one provider fails at load instead of creating fallback order. - Model selection no longer changes the Cordis plugin graph. Catalog-backed adapters can accept any installed catalog model selected after startup, while the native DeepSeek adapter forwards arbitrary DeepSeek model ids. - A custom `baseURL` preserves the selected catalog model's protocol and capabilities; it does not make catalog-external model ids valid. Private endpoints must implement that catalog entry's protocol. -- pi-ai credentials and transport knobs are scoped per provider profile. An omitted key delegates to pi-ai ambient authentication, while an explicitly empty key is invalid. +- pi-ai credentials, transport knobs, SDK timeouts, and the five-minute-default `streamIdleTimeoutMs` watchdog are scoped per provider profile. Hidden provider retries are disabled; bounded retries belong to the separately composed agent recovery policy. - `dsh-llm-pi-ai` rejects stop sequences because pi-ai's common stream API cannot express them; the native DeepSeek adapter retains its stop support. - Replay state is portable only within the adapter instance that owns both the historical and target providers. Cross-provider and cross-model restoration is an adapter responsibility, and another adapter receives provider-neutral history without the opaque state. - Current pre-release session JSONL requires provider/model request headers and assistant provenance. Older shapes remain version `0` but are rejected rather than migrated. ## Testing -- Unit coverage exercises registry conflicts, request reconstruction, session validation, profile resolution, option forwarding, native API selection including OpenAI Responses, conversion, replay validation, error mapping, cancellation, content rewrites, and same-instance versus different-instance replay dispatch. +- Unit coverage exercises registry conflicts, request reconstruction, session validation, profile resolution, single-attempt option forwarding, native API selection including OpenAI Responses, conversion, replay validation, error mapping, caller cancellation, idle-timeout transport termination, content rewrites, and same-instance versus different-instance replay dispatch. - Keyless loop/session tests and ACP snapshots exercise durable provider/model metadata, resume and fork propagation, workflow/subagent overrides, and unchanged user-visible transcripts; the key-gated DeepSeek e2e retains real provider streaming and tool follow-up coverage. - Public JSDoc, package READMEs, architecture and core-data-structure docs, generated catalogs, examples, session fixtures, and Python SDK pairs use provider/model targets consistently and are checked by the repository documentation and type-equivalence gates. diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md index 7dcadf2521..c35225a86b 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md @@ -28,7 +28,7 @@ Status: implemented ### 显式 pi-ai 提供方配置 -`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一，并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称，以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、超时和重试设置。凭据不设全局值：显式密钥仅对所属配置生效；未提供密钥时，pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置，不会回退到环境认证。 +`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一，并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称，以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、SDK 超时和 Harness 流空闲超时。配置中有意不提供重试字段：适配器强制将 pi-ai 的 `maxRetries` 设为零，使一次 `stream()` 调用只发起一次可见的提供方请求；有界的 agent 层恢复由 `dsh-llm-retry` 负责。凭据不设全局值：显式密钥仅对所属配置生效；未提供密钥时，pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置，不会回退到环境认证。 插件通过一次全有或全无调用，将所有已配置的提供方名称注册到同一个 `PiAiAdapter`。请求按 provider 选择对应配置，并在 `getModels(provider)` 中查找模型以取得目录描述符。未知提供方会在插件加载时失败；未知模型会在网络 I/O 前以 `UNKNOWN_MODEL` 失败。适配器不会修改目录对象。当配置提供 `baseURL` 时，适配器复制选中的描述符，仅覆盖 `baseUrl`，使私有端点保留 pi-ai 的 API、能力、兼容标志、上下文限制与推理映射。私有端点必须实现所选提供方的协议，模型 ID 也仍须存在于已安装的 pi-ai 目录中。 @@ -75,14 +75,14 @@ JSON-RPC 运行时显式接收 provider 与 model。仅当 `deepseek` 提供方 - 提供方名称是部署范围内的路由所有权键：两个提供方可以使用相同的模型字符串，但为同一个提供方挂载两个适配器会在加载时失败，不会形成回退顺序。 - 模型选择不再改变 Cordis 插件图。目录型适配器可以接受启动后选择的任意已安装目录模型，原生 DeepSeek 适配器则会转发任意 DeepSeek 模型 ID。 - 自定义 `baseURL` 会保留所选目录模型的协议与能力，但不会让目录外模型 ID 变为有效。私有端点必须实现该目录项对应的协议。 -- pi-ai 凭据与传输选项按提供方配置隔离。省略密钥时委托 pi-ai 使用环境认证；显式空密钥无效。 +- pi-ai 凭据、传输选项、SDK 超时，以及默认五分钟的 `streamIdleTimeoutMs` 空闲超时机制均按提供方配置隔离。系统禁用隐藏的提供方重试；有界重试由单独组合的 agent 恢复策略负责。 - pi-ai 的通用流 API 无法表达停止序列，因此 `dsh-llm-pi-ai` 会拒绝停止序列；原生 DeepSeek 适配器仍支持停止序列。 - 仅当历史提供方与目标提供方归同一个适配器实例所有时，回放状态才可移植。适配器负责跨提供方和跨模型恢复；其他适配器只接收不含不透明状态的提供方无关历史。 - 当前预发布会话 JSONL 要求请求头包含 provider/model，助手消息包含来源信息。旧格式仍使用版本 `0`，但会被拒绝，不执行迁移。 ## 测试 -- 单元测试覆盖注册表冲突、请求重建、会话验证、配置解析、选项转发、包括 OpenAI Responses 在内的原生 API 选择、转换、回放验证、错误映射、取消、内容重写，以及同一实例与不同实例间的回放分发。 +- 单元测试覆盖注册表冲突、请求重建、会话验证、配置解析、单次请求的选项转发、包括 OpenAI Responses 在内的原生 API 选择、转换、回放验证、错误映射、调用方取消、空闲超时导致的传输终止、内容重写，以及同一实例与不同实例间的回放分发。 - 无密钥的 agent loop/会话测试和 ACP 快照覆盖持久化 provider/model 元数据、恢复与 fork 传播、工作流/subagent 覆盖，以及不变的用户可见 transcript（文本记录）；密钥门控的 DeepSeek e2e 测试保留真实提供方的流式输出与工具后续调用覆盖率。 - 公共 JSDoc、package README、架构与核心数据结构文档、生成目录、示例、会话 fixture（测试前置数据）和 Python SDK 配对文档统一使用 provider/model 目标，并由仓库文档与类型等价门禁校验。 diff --git a/docs/architecture.md b/docs/architecture.md index 7cd9b34a2c..befa24c4c2 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -85,7 +85,7 @@ forever: agent/request (config only) -> log request/header -> llm/stream (frozen) on final adapter-path or terminal in-band failure: 'step/end' - agent/request-error(original error, consecutive retry attempt, signal) + agent/request-error(original error, failure facts, immutable prior failures, signal) retry in the next numbered step or preserve the original error otherwise: 'assistant/chunk' @@ -110,11 +110,11 @@ Each step assembles ordered prompt sections, tool schemas, and `{{name}}` variab Tool-time context—including async `agent.inject()` notices and post-tool `additionalContexts`—settles, then follows recorded results. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering before signal closure. Leftovers become queued input. Terminal `agent/turn-stop` runs after continuation and steering folding, stays authoritative through turn close and flush, and discards later steering but preserves queued prompts. -`dsh-compact-basic` handles pressure and canonical overflow at checkpoints; retry requires a balanced surface replacement ([decision](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)). +`dsh-compact-basic` handles pressure/overflow; `dsh-llm-retry` handles bounded transient backoff. Independent budgets compose on `agent/request-error` ([recovery decision](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md)). ### Failure Boundaries -The turn is the containment boundary. Final adapter-path and terminal in-band failures close the step before `agent/request-error`; retry opens a numbered step; otherwise, the provider error survives. Attempts reset on success. +The turn is the containment boundary. Adapter failures close the step, entering `agent/request-error` with the exact `Error`, `LlmFailure`, and retry history. Retry opens a numbered step; success clears history; exhaustion stores the failure on `turn/end`. Failed chunks commit no message or tool. Other failures use `agent/error`. Cancellation and disposal beat recovery; undispatched model tool calls receive synthetic `tool/call` and `ABORTED` result pairs before `turn/end`. `cancel()` clears queues and aborts active work; disposal awaits quiescence before unregistering. @@ -142,7 +142,7 @@ Durability is a plugin concern. Persistence backends buffer synchronous `session Messages contain typed blocks (`text`, `reasoning`, `tool-call`, `tool-result`) derived from merge-extensible `ContentBlockMap`; the same pattern types `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`. New block types coordinate adapters, UI bridges, compaction pricing, token metering, and persistence as one repo-wide contract; replay measurement types live in [token-meter.md](core-data-structures/token-meter.md). -Streaming uses raw chunks (`block-start` through `finish`) and `BlockAssembler`. The loop logs and assembles chunks, storing provider/model provenance plus replay state. An `LlmAdapter` implements `stream()`, registers provider routes, and may expose selector metadata; it resolves and validates model ids. Replay state reaches targets only when both routes map to one adapter instance, which owns validation and conversion. The contract lives in [llm-streaming.md](core-data-structures/llm-streaming.md). +Streaming uses raw chunks and `BlockAssembler`. One `LlmAdapter.stream()` is one provider attempt; adapters report facts, while recovery policy lives on `agent/request-error`. The loop logs chunks and successful provenance/replay state. Remote adapters stop stalled transport with per-read idle watchdogs. Replay state reaches targets only when routes share an adapter instance ([contract](core-data-structures/llm-streaming.md)). ## Extension And Composition diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 74d0c6ba7e..ea1b24ddf1 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -27,7 +27,7 @@ export interface AcpConfig { Depends on: `Stream` (`@agentclientprotocol/sdk`) -Source: [`packages/ui/acp/src/index.ts:206`](../packages/ui/acp/src/index.ts) +Source: [`packages/ui/acp/src/index.ts:207`](../packages/ui/acp/src/index.ts) ## `@deepseek-ai/dsh-acp-demo` @@ -66,6 +66,8 @@ export interface Config { toolBash?: NonNullable /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable + /** Bounded transient model-request retry policy forwarded through agent-core. */ + llmRetry?: NonNullable } ``` @@ -142,6 +144,8 @@ export interface Config { toolBash?: toolBash.Config /** Generic background-task controls; set false to keep the task service without model-facing task tools. */ toolTasks?: toolTasks.Config | false + /** Bounded transient model-request retry policy. */ + llmRetry?: llmRetry.Config } /** Skill bundle config forwarded to the registry, local provider, and model-facing consumer. */ @@ -157,9 +161,9 @@ export interface SkillConfig { } ``` -Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts) +Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`llmRetry`](../packages/llm/llm-retry/src/index.ts) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts) -Source: [`packages/examples/agent-spine-demo/src/index.ts:59`](../packages/examples/agent-spine-demo/src/index.ts) +Source: [`packages/examples/agent-spine-demo/src/index.ts:60`](../packages/examples/agent-spine-demo/src/index.ts) ## `@deepseek-ai/dsh-bash-local` @@ -237,6 +241,8 @@ export interface Config { toolBash?: NonNullable /** Generic background-task control-tool config forwarded through agent-spine-demo. */ toolTasks?: NonNullable + /** Bounded transient model-request retry policy forwarded through agent-spine-demo. */ + llmRetry?: NonNullable /** Controls automatic AGENTS.md/CLAUDE.md loading; configure a byte budget or set `false`. */ workspaceContext: agentCore.Config['workspaceContext'] } @@ -427,6 +433,8 @@ export interface Config { reasoningEffort?: 'high' | 'max' /** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */ models?: DeepSeekCatalogModel[] + /** Maximum provider idle time while one stream read is outstanding (default five minutes). */ + streamIdleTimeoutMs?: number } /** One optional model entry advertised by the hand-written adapter. */ @@ -440,7 +448,7 @@ export interface DeepSeekCatalogModel { } ``` -Source: [`packages/llm/llm-deepseek/src/index.ts:33`](../packages/llm/llm-deepseek/src/index.ts) +Source: [`packages/llm/llm-deepseek/src/index.ts:34`](../packages/llm/llm-deepseek/src/index.ts) ## `@deepseek-ai/dsh-llm-pi-ai` @@ -475,16 +483,14 @@ export interface PiAiProviderProfile { timeoutMs?: number /** WebSocket connection timeout in milliseconds. */ websocketConnectTimeoutMs?: number - /** Provider SDK retry count. */ - maxRetries?: number - /** Maximum provider-requested retry delay in milliseconds. */ - maxRetryDelayMs?: number + /** Maximum provider idle time while one stream read is outstanding. */ + streamIdleTimeoutMs?: number } ``` Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `ThinkingLevel` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`) -Source: [`packages/llm/llm-pi-ai/src/config.ts:40`](../packages/llm/llm-pi-ai/src/config.ts) +Source: [`packages/llm/llm-pi-ai/src/config.ts:48`](../packages/llm/llm-pi-ai/src/config.ts) ## `@deepseek-ai/dsh-llm-replay` @@ -530,6 +536,28 @@ export interface ReplayModelConfig { Source: [`packages/support/llm-replay/src/index.ts:375`](../packages/support/llm-replay/src/index.ts) +## `@deepseek-ai/dsh-llm-retry` + +Requires: `agents` + +```ts config-catalog +/** Deployment-owned limits and classification for transient request recovery. */ +export interface Config { + /** Maximum transient retries after the first request (default 2). */ + maxTransientRetries?: number + /** Initial local exponential-backoff delay in milliseconds (default 500). */ + initialDelayMs?: number + /** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */ + maxDelayMs?: number + /** Symmetric random multiplier range around one (default 0.1). */ + jitterRatio?: number + /** Stable failure codes eligible for this policy. */ + retryableCodes?: string[] +} +``` + +Source: [`packages/llm/llm-retry/src/index.ts:39`](../packages/llm/llm-retry/src/index.ts) + ## `@deepseek-ai/dsh-mcp-client` Requires: `tools` @@ -822,7 +850,7 @@ export interface Config { } ``` -Source: [`packages/ui/stdio/src/index.ts:33`](../packages/ui/stdio/src/index.ts) +Source: [`packages/ui/stdio/src/index.ts:34`](../packages/ui/stdio/src/index.ts) ## `@deepseek-ai/dsh-stdio-demo` @@ -864,6 +892,8 @@ export interface Config { toolBash?: NonNullable /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable + /** Bounded transient model-request retry policy forwarded through agent-core. */ + llmRetry?: NonNullable /** * If set, the pre-created agent RESUMES this persisted session id instead of * starting fresh. Sourced from an env var in the leaf `cordis.yml` @@ -1266,7 +1296,7 @@ export interface TuiConfig { } ``` -Source: [`packages/ui/tui/src/index.ts:100`](../packages/ui/tui/src/index.ts) +Source: [`packages/ui/tui/src/index.ts:101`](../packages/ui/tui/src/index.ts) ## `@deepseek-ai/dsh-user-approval` diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index 5d6c627751..2e303739da 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -75,7 +75,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) -Source: [`packages/core/agent/src/types.ts:311`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:312`](../../packages/core/agent/src/types.ts) ### `agent/post-step` — serial @@ -201,17 +201,18 @@ Recover a model-request failure after its failed step has closed. `retry` opens * @param turn - the open turn number. * @param step - the failed step number. * @param error - the original model-request failure. - * @param retryAttempt - zero-based number of prior recovery retries. + * @param failure - serializable facts normalized at the final adapter boundary. + * @param priorFailures - immutable failures that already authorized another request in this consecutive sequence. * @param signal - the turn abort signal. * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. * @mode waterfall */ -'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise +'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], signal: AbortSignal, next: () => Promise): Promise ``` -Types: [Agent](../core-data-structures/core.md) · [RequestError](../core-data-structures/core.md) · [RequestErrorDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) +Types: [Agent](../core-data-structures/core.md) · [LlmFailure](../core-data-structures/llm-streaming.md) · [RequestError](../core-data-structures/core.md) · [RequestErrorDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) -Source: [`packages/core/agent/src/types.ts:278`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:279`](../../packages/core/agent/src/types.ts) ### `agent/session-prefix` — waterfall @@ -322,7 +323,7 @@ Override whether the turn continues. The default continues after tool calls or s Types: [Agent](../core-data-structures/core.md) · [ContinuationDecision](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) -Source: [`packages/core/agent/src/types.ts:288`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:289`](../../packages/core/agent/src/types.ts) ### `agent/turn-stop` — serial @@ -343,7 +344,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a Types: [Agent](../core-data-structures/core.md) · [ContinuationStop](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) -Source: [`packages/core/agent/src/types.ts:298`](../../packages/core/agent/src/types.ts) +Source: [`packages/core/agent/src/types.ts:299`](../../packages/core/agent/src/types.ts) ## `agent-loop/*` @@ -473,7 +474,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t Types: [GenerateOptions](../core-data-structures/core.md) · [LlmService](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:43`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:44`](../../packages/llm/llm/src/index.ts) ## `session/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 10cd0bbf05..8064c5d2f2 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -525,7 +525,7 @@ stream(options: GenerateOptions): AsyncIterable Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:97`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:137`](../../packages/llm/llm/src/index.ts) ## `ctx.permission` — `PermissionService` diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index 1b87513e11..d08297a751 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -226,6 +226,22 @@ interface GenerateOptions { Why a model response stopped is a merge-extensible reason: +```ts type-equiv +/** Serializable provider-boundary facts; policy decides whether they are retryable. */ +interface LlmFailure { + /** Human-readable provider or transport failure. */ + readonly message: string + /** Stable provider-neutral machine-routing code. */ + readonly code: string + /** HTTP status observed at the provider boundary, when available. */ + readonly status?: number + /** Provider-requested delay in milliseconds, when valid and available. */ + readonly retryAfterMs?: number + /** Opaque provider-issued request identifier for diagnostics. */ + readonly requestId?: ProviderRequestId +} +``` + ```ts type-equiv /** * Why a model response stopped. @@ -235,8 +251,8 @@ interface FinishReasonMap { 'stop': { kind: 'stop' } 'tool-calls': { kind: 'tool-calls' } 'max-tokens': { kind: 'max-tokens' } - 'aborted': { kind: 'aborted' } - 'error': { kind: 'error'; message: string; code?: string } + 'aborted': { kind: 'aborted'; failure: LlmFailure } + 'error': { kind: 'error'; failure: LlmFailure } } ``` @@ -445,14 +461,14 @@ type ContinuationDecision = | { action: 'continue'; reason?: { content: ContentBlock[]; source: MessageSource } } ``` -`agent/request-error` receives the original `RequestError`, whose optional provider-neutral `code` supports stable routing without message parsing: +`agent/request-error` receives the exact original `RequestError` beside its immutable `LlmFailure`, an immutable list of failures that already authorized another request in the consecutive sequence, the turn signal, and `next()`. Recovery plugins route on `failure.code`, not the live error's message; each policy counts only its own codes, and a successful request clears the history: ```ts type-equiv /** Model-request failure with an optional machine-routable provider code. */ type RequestError = Error & { code?: string } ``` -It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the recovery listener's durable mutation, while `fail` preserves that error: +It returns a `RequestErrorDecision`; `retry` opens a new numbered step after the recovery listener's durable mutation, while `fail` retains the structured failure on `turn/end`: ```ts type-equiv /** Failed-request recovery decision; `retry` opens another numbered step while listeners delegate by calling `next()`. */ diff --git a/docs/core-data-structures/llm-streaming.md b/docs/core-data-structures/llm-streaming.md index 41ce6427e6..1ecaa00943 100644 --- a/docs/core-data-structures/llm-streaming.md +++ b/docs/core-data-structures/llm-streaming.md @@ -31,18 +31,38 @@ type StreamChunk = } ``` +Every thrown or in-band final-adapter failure normalizes to one serializable provider-neutral payload. `retryAfterMs` is a validated positive delay observed at the provider boundary, not a retry decision; `ProviderRequestId` is an opaque branded string for diagnostics. + +```ts type-equiv +/** Serializable provider-boundary facts; policy decides whether they are retryable. */ +interface LlmFailure { + /** Human-readable provider or transport failure. */ + readonly message: string + /** Stable provider-neutral machine-routing code. */ + readonly code: string + /** HTTP status observed at the provider boundary, when available. */ + readonly status?: number + /** Provider-requested delay in milliseconds, when valid and available. */ + readonly retryAfterMs?: number + /** Opaque provider-issued request identifier for diagnostics. */ + readonly requestId?: ProviderRequestId +} +``` + ## The adapter contract Every adapter MUST obey these, and every consumer may rely on them: - **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering. - **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`. -- **Two sanctioned error paths.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted'}` (provider in-band errors, for adapters that can't throw mid-stream). Consumers must handle *both*. The agent loop closes the failed step and offers either form to `agent/request-error`; absent recovery it becomes a turn error, and no normal completed assistant message is logged for that request. +- **Two sanctioned error paths, one fact shape.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted', failure}` (provider in-band errors, for adapters that can't throw mid-stream). `LlmError.failure` carries the same `LlmFailure`. The final adapter boundary preserves the exact thrown `Error` object and associates immutable facts with that call; the agent loop closes the failed step and offers the error, facts, and immutable prior-retried facts to `agent/request-error`. Absent recovery the structured failure becomes the turn error, and no normal assistant message or tool side effect is committed for that attempt. +- **One adapter call is one provider attempt.** Adapters disable library retries. Agent-level recovery opens another durable numbered step; direct `ctx.llm.stream()` callers remain single-attempt. +- **Provider stalls are bounded at the transport.** Both shipping remote adapters expose positive finite `streamIdleTimeoutMs` with a five-minute default. The watchdog arms only while iterator `next()` is outstanding, uses one stable signal for the whole request, maps its own expiry to `TIMEOUT`, and keeps an earlier caller abort as `ABORTED`. - **Context overflow has one canonical code.** Both DeepSeek adapters classify explicit provider detail through `isContextWindowExceededError()` and surface `CONTEXT_WINDOW_EXCEEDED`, whether the failure arrives as a thrown HTTP `LlmError` or an in-band finish error. Consumers route on the code, never provider text. - **Every provider HTTP request carries the app-attribution header.** Adapters send `attributionHeaders()` (below) - the `User-Agent` baseline - and prove it with a wire-level test (mock server asserting the received header, or the library's header hook for a library-backed adapter). - **Replay state is adapter-owned.** A successful `finish` may carry lossless-JSON state needed to reconstruct a native provider response. The loop stores it with the assembled assistant message unless an `agent/step-result` listener rewrote the content. On a later request, `LlmService` passes the state only when the historical provider and target provider are currently registered to the exact same adapter instance. That adapter validates the state and owns any cross-model or cross-provider conversion; other adapters receive the provider-neutral content and provenance without the private state. -This contract was pinned down by two deliberately independent implementations: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (a generic multi-provider adapter through `@earendil-works/pi-ai`). The library-backed adapter cannot throw mid-stream, so it exercises the finish-chunk error path the hand-rolled one might not. +This contract is pinned down by two deliberately independent implementations: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (a generic multi-provider adapter through `@earendil-works/pi-ai`). The library-backed adapter exercises the finish-chunk error path, while transport-boundary tests prove each idle watchdog stops its actual request. ## `AppIdentity` — app attribution diff --git a/docs/core-data-structures/session.md b/docs/core-data-structures/session.md index 263d991f86..1d8c040991 100644 --- a/docs/core-data-structures/session.md +++ b/docs/core-data-structures/session.md @@ -435,7 +435,7 @@ declare class Session { - `context/message` → a user-role message at its chronological position. The default `envelope` is `context`, which wraps content as `…`; `envelope: 'raw'` uses caller-owned framing verbatim. Optional JSON `meta` remains in the event log and is never rendered. - `steering/message` → a user-role message wrapped in `…` at its chronological position. -Everything else (`turn/*`, `step/*`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data. +Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`, with normalized `LlmFailure` facts for a final model-request failure and message/code for other live errors. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data. ## Live-session fork API @@ -479,9 +479,13 @@ interface TurnEndReasonMap { * The turn failed: a step threw or the model reported a failure. `step` is the * step number the failure occurred on (the operational error's location — the * single durable record of an in-turn failure; live diagnostics also fire via - * `agent/error`). `code` is the error's code when one was attached. + * `agent/error`). Final model-request failures retain their normalized facts + * as one `failure`; other turn failures retain their live Error message/code. */ - error: { kind: 'error'; step: number; message: string; code?: string } + error: { kind: 'error'; step: number } & ( + | { failure: LlmFailure; message?: never; code?: never } + | { message: string; code?: string; failure?: never } + ) disposed: { kind: 'disposed' } /** At least one step reached its output-token ceiling, even if a plugin continued the turn. */ 'max-tokens': { kind: 'max-tokens' } diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 4f6e6daff0..8471a2b8f5 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -10,24 +10,24 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:362`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) | | `agent/created` | `emit` | [`packages/core/agent/src/types.ts:147`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) | | `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:156`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) | -| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:311`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`tui`](../packages/ui/tui) | +| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:312`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`tui`](../packages/ui/tui) | | `agent/post-step` | `serial` | [`packages/core/agent/src/types.ts:264`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) | | `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:204`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`time-context`](../packages/context/time-context), [`user-approval`](../packages/ui/user-approval) | | `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:214`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | | `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:175`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - | | `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:226`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`acp`](../packages/ui/acp) | -| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:278`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic) | +| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:279`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic), [`llm-retry`](../packages/llm/llm-retry) | | `agent/session-prefix` | `waterfall` | [`packages/core/agent/src/types.ts:241`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`tool-skill`](../packages/skill/tool-skill), [`workspace-context`](../packages/context/workspace-context) | | `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:188`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`stdio`](../packages/ui/stdio) | | `agent/status` | `emit` | [`packages/core/agent/src/types.ts:165`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`invariants`](../packages/support/invariants), [`stdio`](../packages/ui/stdio), [`tui`](../packages/ui/tui) | | `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:252`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - | -| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:288`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | -| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:298`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | +| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:289`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) | +| `agent/turn-stop` | `serial` | [`packages/core/agent/src/types.ts:299`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | | `approval/request` | `waterfall` | [`packages/ui/user-approval/src/index.ts:31`](../packages/ui/user-approval/src/index.ts) | [`user-approval`](../packages/ui/user-approval) (`waterfall`) | [`acp`](../packages/ui/acp) | | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:61`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:70`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:53`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:43`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:44`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | | `session/created` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/disposed` | `emit` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/event` | `emit` | [`packages/core/session/src/index.ts:69`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/ui/acp), [`cli-demo`](../packages/examples/cli-demo), [`invariants`](../packages/support/invariants), [`jsonrpc`](../packages/ui/jsonrpc), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio`](../packages/ui/stdio), [`token-meter`](../packages/llm/token-meter), [`tui`](../packages/ui/tui), [`workspace-context`](../packages/context/workspace-context) | diff --git a/docs/module-graph.md b/docs/module-graph.md index e99a53c030..02424b6765 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -18,6 +18,7 @@ flowchart TD pkg_llm["llm"] pkg_llm_deepseek["llm-deepseek"] pkg_llm_pi_ai["llm-pi-ai"] + pkg_llm_retry["llm-retry"] pkg_token_meter["token-meter"] end subgraph group_core["packages/core"] @@ -157,7 +158,9 @@ flowchart TD pkg_scripts --> pkg_app_boot pkg_telemetry --> pkg_brand pkg_llm_deepseek --> pkg_llm + pkg_llm_deepseek --> pkg_timeout pkg_llm_pi_ai --> pkg_llm + pkg_llm_pi_ai --> pkg_timeout pkg_session --> pkg_brand pkg_session --> pkg_llm pkg_session --> pkg_scope @@ -196,6 +199,10 @@ flowchart TD pkg_llm_replay --> pkg_session pkg_sandbox_local --> pkg_llm pkg_sandbox_local --> pkg_sandbox + pkg_llm_retry --> pkg_agent + pkg_llm_retry --> pkg_llm + pkg_llm_retry --> pkg_session + pkg_llm_retry --> pkg_timeout pkg_bash_local --> pkg_bash pkg_bash_local --> pkg_timeout pkg_compact_basic --> pkg_agent @@ -318,6 +325,7 @@ flowchart TD pkg_acp --> pkg_agent pkg_acp --> pkg_bash pkg_acp --> pkg_llm + pkg_acp --> pkg_llm_retry pkg_acp --> pkg_permission pkg_acp --> pkg_sandbox pkg_acp --> pkg_session @@ -380,11 +388,13 @@ flowchart TD pkg_stdio --> pkg_agent pkg_stdio --> pkg_agent_loop pkg_stdio --> pkg_llm + pkg_stdio --> pkg_llm_retry pkg_stdio --> pkg_session pkg_stdio --> pkg_user_interaction pkg_tui --> pkg_agent pkg_tui --> pkg_agent_loop pkg_tui --> pkg_llm + pkg_tui --> pkg_llm_retry pkg_tui --> pkg_session pkg_tui --> pkg_tools pkg_tui --> pkg_user_interaction @@ -393,6 +403,7 @@ flowchart TD pkg_agent_spine_demo --> pkg_home pkg_agent_spine_demo --> pkg_invariants pkg_agent_spine_demo --> pkg_llm + pkg_agent_spine_demo --> pkg_llm_retry pkg_agent_spine_demo --> pkg_session pkg_agent_spine_demo --> pkg_skill pkg_agent_spine_demo --> pkg_skill_local @@ -466,8 +477,8 @@ flowchart TD | [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand) | | [`scripts`](../packages/sdk/scripts) | `sdk` | [`app-boot`](../packages/ui/app-boot) | | [`telemetry`](../packages/sdk/telemetry) | `sdk` | [`brand`](../packages/util/brand) | -| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm) | -| [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm) | +| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`llm`](../packages/llm/llm), [`timeout`](../packages/util/timeout) | +| [`llm-pi-ai`](../packages/llm/llm-pi-ai) | `llm` | [`llm`](../packages/llm/llm), [`timeout`](../packages/util/timeout) | | [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) | | [`system-prompt`](../packages/core/system-prompt) | `core` | [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) | | [`fs`](../packages/fs/fs) | `fs` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm) | @@ -488,6 +499,7 @@ flowchart TD | [`session-persistence`](../packages/session-persistence/session-persistence) | `session-persistence` | [`session`](../packages/core/session) | | [`llm-replay`](../packages/support/llm-replay) | `support` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | | [`sandbox-local`](../packages/sandbox/sandbox-local) | `sandbox` | [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox) | +| [`llm-retry`](../packages/llm/llm-retry) | `llm` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | | [`bash-local`](../packages/bash/bash-local) | `bash` | [`bash`](../packages/bash/bash), [`timeout`](../packages/util/timeout) | | [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`token-meter`](../packages/llm/token-meter) | | [`spill-local`](../packages/spill/spill-local) | `spill` | [`spill`](../packages/spill/spill) | @@ -517,7 +529,7 @@ flowchart TD | [`tool-cordis`](../packages/cordis/tool-cordis) | `cordis` | [`scope`](../packages/core/scope), [`tools`](../packages/core/tools) | | [`hooks-codex`](../packages/hooks/hooks-codex) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`tools`](../packages/core/tools) | | [`agent-loop-testkit`](../packages/support/agent-loop-testkit) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | -| [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`permission`](../packages/ui/permission), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval), [`user-interaction`](../packages/ui/user-interaction) | +| [`acp`](../packages/ui/acp) | `ui` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`permission`](../packages/ui/permission), [`sandbox`](../packages/sandbox/sandbox), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval), [`user-interaction`](../packages/ui/user-interaction) | | [`tool-ask-user`](../packages/ui/tool-ask-user) | `ui` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools), [`user-interaction`](../packages/ui/user-interaction) | | [`workspace-context`](../packages/context/workspace-context) | `context` | [`agent`](../packages/core/agent), [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm), [`paths`](../packages/util/paths), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | | [`repeat-tool-guard`](../packages/guard/repeat-tool-guard) | `guard` | [`agent`](../packages/core/agent), [`tools`](../packages/core/tools) | @@ -529,9 +541,9 @@ flowchart TD | [`tool-subagent`](../packages/subagent/tool-subagent) | `subagent` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`subagent`](../packages/subagent/subagent), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools) | | [`hooks-claude`](../packages/hooks/hooks-claude) | `hooks` | [`agent`](../packages/core/agent), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools) | | [`jsonrpc`](../packages/ui/jsonrpc) | `ui` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`llm-deepseek`](../packages/llm/llm-deepseek), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent) | -| [`stdio`](../packages/ui/stdio) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`user-interaction`](../packages/ui/user-interaction) | -| [`tui`](../packages/ui/tui) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`user-interaction`](../packages/ui/user-interaction) | -| [`agent-spine-demo`](../packages/examples/agent-spine-demo) | `examples` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`home`](../packages/util/home), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`skill`](../packages/skill/skill), [`skill-local`](../packages/skill/skill-local), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tool-bash`](../packages/bash/tool-bash), [`tool-skill`](../packages/skill/tool-skill), [`tool-tasks`](../packages/tasks/tool-tasks), [`tools`](../packages/core/tools), [`workspace-context`](../packages/context/workspace-context) | +| [`stdio`](../packages/ui/stdio) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`user-interaction`](../packages/ui/user-interaction) | +| [`tui`](../packages/ui/tui) | `ui` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`tools`](../packages/core/tools), [`user-interaction`](../packages/ui/user-interaction) | +| [`agent-spine-demo`](../packages/examples/agent-spine-demo) | `examples` | [`agent`](../packages/core/agent), [`agent-loop`](../packages/core/agent-loop), [`home`](../packages/util/home), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`skill`](../packages/skill/skill), [`skill-local`](../packages/skill/skill-local), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tool-bash`](../packages/bash/tool-bash), [`tool-skill`](../packages/skill/tool-skill), [`tool-tasks`](../packages/tasks/tool-tasks), [`tools`](../packages/core/tools), [`workspace-context`](../packages/context/workspace-context) | | [`workflow-workerthread`](../packages/workflow/workflow-workerthread) | `workflow` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`tools`](../packages/core/tools), [`workflow`](../packages/workflow/workflow) | | [`subagent-fork`](../packages/subagent/subagent-fork) | `subagent` | [`agent`](../packages/core/agent), [`session`](../packages/core/session), [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | | [`subagent-spawn`](../packages/subagent/subagent-spawn) | `subagent` | [`subagent`](../packages/subagent/subagent), [`subagent-inprocess`](../packages/subagent/subagent-inprocess) | diff --git a/docs/persistence-catalog.md b/docs/persistence-catalog.md index 688d3b545c..74f677d688 100644 --- a/docs/persistence-catalog.md +++ b/docs/persistence-catalog.md @@ -79,7 +79,7 @@ export type SessionEvent = { }[T] ``` -Sources: [`packages/core/session/src/types.ts:255`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:262`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:292`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:324`](../packages/core/session/src/types.ts) +Sources: [`packages/core/session/src/types.ts:259`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:266`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:296`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:328`](../packages/core/session/src/types.ts) ## Events @@ -151,7 +151,7 @@ Source: [`packages/ui/user-approval/src/index.ts:68`](../packages/ui/user-approv Types: [StreamChunk](core-data-structures/llm-streaming.md) -Source: [`packages/core/session/src/types.ts:219`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:223`](../packages/core/session/src/types.ts) #### `assistant/message` — surface @@ -167,7 +167,7 @@ Source: [`packages/core/session/src/types.ts:219`](../packages/core/session/src/ Types: [ContentBlock](core-data-structures/core.md) · [TokenUsage](core-data-structures/llm-streaming.md) -Source: [`packages/core/session/src/types.ts:226`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:230`](../packages/core/session/src/types.ts) ### `bash/*` @@ -258,7 +258,7 @@ Source: [`packages/compact/compact/src/types.ts:22`](../packages/compact/compact Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:212`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:216`](../packages/core/session/src/types.ts) ### `hook/*` @@ -306,6 +306,24 @@ Source: [`packages/hooks/hook-protocol/src/types.ts:19`](../packages/hooks/hook- Source: [`packages/hooks/hook-protocol/src/types.ts:31`](../packages/hooks/hook-protocol/src/types.ts) +### `llm/*` + +#### `llm/retry` — log-only + +```ts persistence-catalog +/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */ +'llm/retry': { + turn: number + step: number + retry: number + maxRetries: number + delayMs: number + failure: LlmFailure +} +``` + +Source: [`packages/llm/llm-retry/src/index.ts:18`](../packages/llm/llm-retry/src/index.ts) + ### `permission/*` #### `permission/preset` — log-only @@ -336,7 +354,7 @@ Source: [`packages/ui/permission/src/index.ts:33`](../packages/ui/permission/src Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:204`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:208`](../packages/core/session/src/types.ts) ### `request/*` @@ -350,7 +368,7 @@ Source: [`packages/core/session/src/types.ts:204`](../packages/core/session/src/ 'request/header': { header: EpochHeader; reason: RequestHeaderReason } ``` -Source: [`packages/core/session/src/types.ts:251`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:255`](../packages/core/session/src/types.ts) ### `steering/*` @@ -363,7 +381,7 @@ Source: [`packages/core/session/src/types.ts:251`](../packages/core/session/src/ Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:244`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:248`](../packages/core/session/src/types.ts) ### `step/*` @@ -374,7 +392,7 @@ Source: [`packages/core/session/src/types.ts:244`](../packages/core/session/src/ 'step/end': { turn: number; step: number } ``` -Source: [`packages/core/session/src/types.ts:197`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:201`](../packages/core/session/src/types.ts) #### `step/start` — log-only @@ -383,7 +401,7 @@ Source: [`packages/core/session/src/types.ts:197`](../packages/core/session/src/ 'step/start': { turn: number; step: number } ``` -Source: [`packages/core/session/src/types.ts:195`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:199`](../packages/core/session/src/types.ts) ### `todo/*` @@ -396,7 +414,7 @@ Source: [`packages/core/session/src/types.ts:195`](../packages/core/session/src/ Types: [TodoItem](core-data-structures/session.md) -Source: [`packages/core/session/src/types.ts:246`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:250`](../packages/core/session/src/types.ts) ### `tool/*` @@ -413,7 +431,7 @@ Source: [`packages/core/session/src/types.ts:246`](../packages/core/session/src/ Types: [CallId](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:232`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:236`](../packages/core/session/src/types.ts) #### `tool/code-dispatch` — log-only @@ -457,7 +475,7 @@ Source: [`packages/core/tools/src/code-mode.ts:34`](../packages/core/tools/src/c Types: [CallId](core-data-structures/core.md) · [ContentBlock](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:242`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:246`](../packages/core/session/src/types.ts) ### `turn/*` @@ -474,7 +492,7 @@ Source: [`packages/core/session/src/types.ts:242`](../packages/core/session/src/ Types: [TurnEndReason](core-data-structures/session.md) -Source: [`packages/core/session/src/types.ts:193`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:197`](../packages/core/session/src/types.ts) #### `turn/start` — log-only @@ -490,7 +508,7 @@ Source: [`packages/core/session/src/types.ts:193`](../packages/core/session/src/ Types: [TurnTrigger](core-data-structures/session.md) -Source: [`packages/core/session/src/types.ts:187`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:191`](../packages/core/session/src/types.ts) ### `user/*` @@ -503,4 +521,4 @@ Source: [`packages/core/session/src/types.ts:187`](../packages/core/session/src/ Types: [ContentBlock](core-data-structures/core.md) · [MessageSource](core-data-structures/core.md) -Source: [`packages/core/session/src/types.ts:199`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:203`](../packages/core/session/src/types.ts) diff --git a/docs/testing.md b/docs/testing.md index 84629aae45..d2260f9221 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -17,6 +17,8 @@ We are DeepSeek — do not ration real-API tests. A no-key test proves plumbing; Mock only the genuinely expensive or non-deterministic boundary (the LLM adapter, the network, the clock); keep everything downstream real. A hand-rolled stand-in proves the bridge moves bytes, not that the shipping tool behaves as asserted — the two drift while the test stays green. Example: bridge tool-call tests run the scripted mock MODEL but the real tool + real executor (`makeBridgeHarness({ withBash: true })` plugs `dsh-bash-local` + `dsh-tool-bash` and runs an actual `echo`). +Recovery tests separate pre/post-chunk failures by step and prove failed chunks derive no message or tool side effect. Cover exhaustion, cancellation, policy composition, persistence, status, wire counts, transport-closing idle timeouts, and shipping Loader composition. + ## Verify the world, not the self-report An e2e assertion re-runs the command or re-reads the file externally; a keyword probe on the agent's own output lets a cheating agent pass. Assert untouched files are byte-identical. e2e tests own their resources: create the harness in the test, dispose in `afterEach` (even on failure/retry/timeout); shared fixtures live in a plain `tests/harness.ts`, never another `*.e2e.ts` (importing a spec re-registers its `describe` and duplicates real API calls). diff --git a/examples/acp-agent/tests/snapshots/error-finish/session.jsonl b/examples/acp-agent/tests/snapshots/error-finish/session.jsonl index f0ef4267ac..a069fb1ebe 100644 --- a/examples/acp-agent/tests/snapshots/error-finish/session.jsonl +++ b/examples/acp-agent/tests/snapshots/error-finish/session.jsonl @@ -4,4 +4,4 @@ {"type":"step/start","seq":2,"time":0,"data":{"turn":1,"step":1}} {"type":"request/header","seq":3,"time":0,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} {"type":"step/end","seq":4,"time":0,"data":{"turn":1,"step":1}} -{"type":"turn/end","seq":5,"time":0,"data":{"turn":1,"reason":{"kind":"error","step":1,"message":"simulated provider error (HTTP 401)","code":"AUTH"}}} +{"type":"turn/end","seq":5,"time":0,"data":{"turn":1,"reason":{"kind":"error","step":1,"failure":{"message":"simulated provider error (HTTP 401)","code":"AUTH"}}}} diff --git a/examples/acp-agent/tests/snapshots/error-finish/stdout.expected.jsonl b/examples/acp-agent/tests/snapshots/error-finish/stdout.expected.jsonl index 540eb2338a..f941121f12 100644 --- a/examples/acp-agent/tests/snapshots/error-finish/stdout.expected.jsonl +++ b/examples/acp-agent/tests/snapshots/error-finish/stdout.expected.jsonl @@ -1,3 +1,4 @@ {"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"loadSession":true,"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} {"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}","configOptions":[{"id":"model","name":"Model","description":"Sets this session's provider and model.","category":"model","type":"select","currentValue":"[\"deepseek\",\"deepseek-v4-flash\"]","options":[{"value":"[\"deepseek\",\"deepseek-v4-flash\"]","name":"deepseek-v4-flash"},{"value":"[\"deepseek\",\"deepseek-v4-pro\"]","name":"deepseek-v4-pro"}]},{"id":"permission","name":"Permissions","description":"The session permission preset: each choice bundles a sandbox mode and an approval policy.","category":"mode","type":"select","currentValue":"danger-full-access","options":[{"value":"workspace-write","name":"workspace-write","description":"Write inside the workspace and permitted temporary directories; wider retries require approval."},{"value":"danger-full-access","name":"danger-full-access","description":"Full file access without approval prompts."}]}]}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"\n\n[Model attempt failed; any partial output above is discarded: simulated provider error (HTTP 401)]\n\n"}}}} {"jsonrpc":"2.0","id":3,"error":{"code":-32603,"message":"Internal error: turn failed: simulated provider error (HTTP 401)"}} diff --git a/packages/compact/compact-basic/package.json b/packages/compact/compact-basic/package.json index a0ee2b4036..56f512ba8d 100644 --- a/packages/compact/compact-basic/package.json +++ b/packages/compact/compact-basic/package.json @@ -41,6 +41,7 @@ "@deepseek-ai/dsh-compact": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-token-meter": "workspace:^", "@deepseek-ai/dsh-tools": "workspace:^", diff --git a/packages/compact/compact-basic/src/index.ts b/packages/compact/compact-basic/src/index.ts index 5d325d57ba..2a958bff9d 100644 --- a/packages/compact/compact-basic/src/index.ts +++ b/packages/compact/compact-basic/src/index.ts @@ -93,9 +93,10 @@ export class BasicCompactService extends CompactService { } }) - ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => { - if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE - || retryAttempt >= this.config.maxOverflowRetries + ctx.on('agent/request-error', async (agent, _turn, _step, _error, failure, priorFailures, signal, next) => { + const priorOverflowFailures = priorFailures.filter(item => item.code === CONTEXT_WINDOW_EXCEEDED_CODE).length + if (failure.code !== CONTEXT_WINDOW_EXCEEDED_CODE + || priorOverflowFailures >= this.config.maxOverflowRetries || signal.aborted) return next() let generation: number diff --git a/packages/compact/compact-basic/src/summarizer.ts b/packages/compact/compact-basic/src/summarizer.ts index 62b5f5f5e2..32e308bce4 100644 --- a/packages/compact/compact-basic/src/summarizer.ts +++ b/packages/compact/compact-basic/src/summarizer.ts @@ -141,14 +141,10 @@ export function frameSummary(summary: readonly ContentBlock[]): ContentBlock[] { /** Map a terminal summarization finish to its fail-closed error. */ function finishError(finish: FinishReason): Error | undefined { switch (finish.kind) { - case 'error': { - const error = new Error(finish.message) as Error & { code?: string } - if (finish.code !== undefined) error.code = finish.code - return error - } + case 'error': case 'aborted': { - const error = new Error('summarization stream aborted') as Error & { code?: string } - error.code = 'ABORTED' + const error = new Error(finish.failure.message) as Error & { code?: string } + error.code = finish.failure.code return error } case 'max-tokens': { diff --git a/packages/compact/compact-basic/tests/compact-basic.spec.ts b/packages/compact/compact-basic/tests/compact-basic.spec.ts index 0a411440b3..14038a03e8 100644 --- a/packages/compact/compact-basic/tests/compact-basic.spec.ts +++ b/packages/compact/compact-basic/tests/compact-basic.spec.ts @@ -7,7 +7,7 @@ import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-a import { resolveConfig } from '@deepseek-ai/dsh-compact-basic/src/config.ts' import type { CompactionResult } from '@deepseek-ai/dsh-compact' import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@deepseek-ai/dsh-llm' -import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, GenerateOptions, LlmFailure, StreamChunk } from '@deepseek-ai/dsh-llm' import { Session, SessionId } from '@deepseek-ai/dsh-session' import TokenMeterService from '@deepseek-ai/dsh-token-meter' import type { Agent } from '@deepseek-ai/dsh-agent' @@ -762,9 +762,9 @@ describe('default one-shot summarizer', () => { }) it.each([ - [{ kind: 'error', message: 'provider failed', code: 'PROVIDER' }, 'PROVIDER', /provider failed/], - [{ kind: 'error', message: 'opaque' }, undefined, /opaque/], - [{ kind: 'aborted' }, 'ABORTED', /aborted/], + [{ kind: 'error', failure: { message: 'provider failed', code: 'PROVIDER' } }, 'PROVIDER', /provider failed/], + [{ kind: 'error', failure: { message: 'opaque', code: 'UNKNOWN' } }, 'UNKNOWN', /opaque/], + [{ kind: 'aborted', failure: { message: 'summarization aborted', code: 'ABORTED' } }, 'ABORTED', /aborted/], [{ kind: 'max-tokens' }, 'MAX_TOKENS', /token cap/], ] as Array<[(StreamChunk & { type: 'finish' })['reason'], string | undefined, RegExp]>) ( 'rejects terminal finish %#', @@ -802,7 +802,9 @@ describe('automatic listener and loader composition', () => { signal = SIGNAL, next: () => Promise<{ action: 'fail' | 'retry' }> = () => Promise.resolve({ action: 'fail' }), ): Promise<{ action: 'fail' | 'retry' }> { - return ctx.waterfall('agent/request-error', owner, 1, 1, error, retryAttempt, signal, next) + const failure: LlmFailure = { message: error.message, code: error.code ?? 'UNKNOWN' } + const priorFailures = Object.freeze(Array.from({ length: retryAttempt }, () => failure)) + return ctx.waterfall('agent/request-error', owner, 1, 1, error, failure, priorFailures, signal, next) } function overflow(message = 'provider overflow'): Error & { code: string } { diff --git a/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts b/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts index 1327f073c5..e899979f46 100644 --- a/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts +++ b/packages/compact/compact-basic/tests/compact-loop-repro.spec.ts @@ -11,6 +11,7 @@ import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-test import * as Invariants from '@deepseek-ai/dsh-invariants' import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic' import TokenMeterService from '@deepseek-ai/dsh-token-meter' +import * as LlmRetry from '@deepseek-ai/dsh-llm-retry' import { SessionId, type SurfaceEvent } from '@deepseek-ai/dsh-session' /** @@ -61,7 +62,10 @@ class OverflowRecoveryAdapter extends LlmAdapter { readonly conversationRequests: GenerateOptions[] = [] readonly summaryRequests: GenerateOptions[] = [] - constructor(private readonly delivery: 'thrown' | 'in-band') { + constructor( + private readonly delivery: 'thrown' | 'in-band', + private readonly transientAfterOverflow = false, + ) { super() } @@ -83,12 +87,17 @@ class OverflowRecoveryAdapter extends LlmAdapter { type: 'finish', reason: { kind: 'error', - message: 'request too large for model context', - code: CONTEXT_WINDOW_EXCEEDED_CODE, + failure: { + message: 'request too large for model context', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }, }, } return } + if (this.transientAfterOverflow && this.conversationRequests.length === 2) { + throw new LlmError('temporary provider outage', 'SERVER') + } yield { type: 'block-start', index: 0, blockType: 'text' } yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } } yield { type: 'finish', reason: { kind: 'stop' } } @@ -134,6 +143,29 @@ function waitForIdle(ctx: Context, agent: Agent): Promise { }) } +function seedOverflowHistory(agent: Agent): void { + for (let turn = 1; turn <= 2; turn += 1) { + const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY' + agent.session.append('turn/start', { + turn, + trigger: { kind: 'message', source: { kind: 'user' } }, + }) + agent.session.append('user/message', { + content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }], + source: { kind: 'user' }, + }, { surfaceOp: 'append' }) + agent.session.append('step/start', { turn, step: 1 }) + agent.session.append('assistant/message', { + provenance: { provider: 'mock', model: 'mock' }, + turn, + step: 1, + content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }], + }, { surfaceOp: 'append' }) + agent.session.append('step/end', { turn, step: 1 }) + agent.session.append('turn/end', { turn, reason: { kind: 'completed' } }) + } +} + describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () => { it('uses the model actually routed by agent/request for post-step pressure', async () => { const { ctx } = await harness(8) @@ -241,26 +273,7 @@ describe('context-overflow recovery across the real loop and compact-basic', () provider: 'unconfigured-agent-fallback', model: 'unconfigured-agent-fallback', }) - for (let turn = 1; turn <= 2; turn += 1) { - const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY' - agent.session.append('turn/start', { - turn, - trigger: { kind: 'message', source: { kind: 'user' } }, - }) - agent.session.append('user/message', { - content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }], - source: { kind: 'user' }, - }, { surfaceOp: 'append' }) - agent.session.append('step/start', { turn, step: 1 }) - agent.session.append('assistant/message', { - provenance: { provider: 'mock', model: 'mock' }, - turn, - step: 1, - content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }], - }, { surfaceOp: 'append' }) - agent.session.append('step/end', { turn, step: 1 }) - agent.session.append('turn/end', { turn, reason: { kind: 'completed' } }) - } + seedOverflowHistory(agent) agent.send([{ type: 'text', text: 'continue from history' }]) await agent.whenIdle() @@ -299,4 +312,47 @@ describe('context-overflow recovery across the real loop and compact-basic', () } }, ) + + it('keeps context-overflow and transient retry budgets independent in one sequence', async () => { + const ctx = new Context() + const adapter = new OverflowRecoveryAdapter('thrown', true) + await mountAgentLoopTestDependencies(ctx) + await ctx.plugin(Invariants) + await ctx.plugin(LlmRetry, { + maxTransientRetries: 1, + initialDelayMs: 1, + maxDelayMs: 1, + jitterRatio: 0, + }) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(TokenMeterService, { contextWindow: 128 }) + ctx.llm.registerAdapter(['mock'], adapter) + await ctx.plugin(BasicCompactService, { + thresholdRatio: 1, + retainTokens: 100, + maxTokens: 64, + compactionRetries: 0, + maxOverflowRetries: 1, + }) + + try { + const agent = ctx.agentLoop.create(SessionId('alternating-recovery'), { provider: 'mock', model: 'mock' }) + seedOverflowHistory(agent) + agent.send([{ type: 'text', text: 'continue from history' }]) + await agent.whenIdle() + + expect(adapter.conversationRequests).toHaveLength(3) + expect(adapter.summaryRequests).toHaveLength(1) + expect(agent.session.events.filter(event => event.type === 'llm/retry').map(event => event.data)) + .toEqual([expect.objectContaining({ step: 2, retry: 1, failure: { message: 'temporary provider outage', code: 'SERVER' } })]) + expect(agent.session.events.filter(event => event.type === 'step/start').slice(-3).map(event => event.data.step)) + .toEqual([1, 2, 3]) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'completed' } }, + }) + } finally { + await ctx.fiber.dispose() + } + }) }) diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index c0c29331ae..e6c4cf05cb 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -672,8 +672,8 @@ export const EVENT_API: readonly EventApiEntry[] = [ { name: 'agent/request-error', mode: 'waterfall', - signature: '\'agent/request-error\'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise', - jsDoc: '/**\n * Recover a model-request failure after its failed step has closed. `retry`\n * opens a new numbered step; `fail` preserves the original request error.\n * Call `next()` to delegate to the next recovery listener or the default.\n * @param agent - the agent whose request failed.\n * @param turn - the open turn number.\n * @param step - the failed step number.\n * @param error - the original model-request failure.\n * @param retryAttempt - zero-based number of prior recovery retries.\n * @param signal - the turn abort signal.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */', + signature: '\'agent/request-error\'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], signal: AbortSignal, next: () => Promise): Promise', + jsDoc: '/**\n * Recover a model-request failure after its failed step has closed. `retry`\n * opens a new numbered step; `fail` preserves the original request error.\n * Call `next()` to delegate to the next recovery listener or the default.\n * @param agent - the agent whose request failed.\n * @param turn - the open turn number.\n * @param step - the failed step number.\n * @param error - the original model-request failure.\n * @param failure - serializable facts normalized at the final adapter boundary.\n * @param priorFailures - immutable failures that already authorized another request in this consecutive sequence.\n * @param signal - the turn abort signal.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */', summary: 'Recover a model-request failure after its failed step has closed.', }, { @@ -1114,7 +1114,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'FinishReasonMap', - declaration: 'export interface FinishReasonMap {\n \'stop\': {\n kind: \'stop\';\n };\n \'tool-calls\': {\n kind: \'tool-calls\';\n };\n \'max-tokens\': {\n kind: \'max-tokens\';\n };\n \'aborted\': {\n kind: \'aborted\';\n };\n \'error\': {\n kind: \'error\';\n message: string;\n code?: string;\n };\n}', + declaration: 'export interface FinishReasonMap {\n \'stop\': {\n kind: \'stop\';\n };\n \'tool-calls\': {\n kind: \'tool-calls\';\n };\n \'max-tokens\': {\n kind: \'max-tokens\';\n };\n \'aborted\': {\n kind: \'aborted\';\n failure: LlmFailure;\n };\n \'error\': {\n kind: \'error\';\n failure: LlmFailure;\n };\n}', }, { name: 'FsDirEntry', @@ -1184,6 +1184,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'LlmCallConfig', declaration: 'export interface LlmCallConfig {\n provider: string;\n model: string;\n temperature?: number;\n maxTokens?: number;\n stop?: string[];\n}', }, + { + name: 'LlmFailure', + declaration: 'export interface LlmFailure {\n readonly message: string;\n readonly code: string;\n readonly status?: number;\n readonly retryAfterMs?: number;\n readonly requestId?: ProviderRequestId;\n}', + }, { name: 'LlmModelInfo', declaration: 'export interface LlmModelInfo {\n provider: string;\n id: string;\n name: string;\n description?: string;\n}', @@ -1220,6 +1224,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'PromptSection', declaration: 'export interface PromptSection {\n readonly name: string;\n readonly order: number;\n readonly text: string | ((context: AssembleContext) => string);\n}', }, + { + name: 'ProviderRequestId', + declaration: 'export type ProviderRequestId = Branded<\'ProviderRequestId\'>;', + }, { name: 'ReasoningBlock', declaration: 'export interface ReasoningBlock {\n type: \'reasoning\';\n text: string;\n}', @@ -1566,7 +1574,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'TurnEndReasonMap', - declaration: 'export interface TurnEndReasonMap {\n completed: {\n kind: \'completed\';\n };\n aborted: {\n kind: \'aborted\';\n reason?: string;\n };\n error: {\n kind: \'error\';\n step: number;\n message: string;\n code?: string;\n };\n disposed: {\n kind: \'disposed\';\n };\n \'max-tokens\': {\n kind: \'max-tokens\';\n };\n rejected: {\n kind: \'rejected\';\n reason: string;\n };\n interrupted: {\n kind: \'interrupted\';\n };\n}', + declaration: 'export interface TurnEndReasonMap {\n completed: {\n kind: \'completed\';\n };\n aborted: {\n kind: \'aborted\';\n reason?: string;\n };\n error: {\n kind: \'error\';\n step: number;\n } & ({\n failure: LlmFailure;\n message?: never;\n code?: never;\n } | {\n message: string;\n code?: string;\n failure?: never;\n });\n disposed: {\n kind: \'disposed\';\n };\n \'max-tokens\': {\n kind: \'max-tokens\';\n };\n rejected: {\n kind: \'rejected\';\n reason: string;\n };\n interrupted: {\n kind: \'interrupted\';\n };\n}', }, { name: 'TurnTrigger', diff --git a/packages/core/agent-loop/README.md b/packages/core/agent-loop/README.md index 9c5c869168..6d70d00746 100644 --- a/packages/core/agent-loop/README.md +++ b/packages/core/agent-loop/README.md @@ -54,7 +54,7 @@ The driver owns one agent for its lifetime and runs inside `ctx.agents.withIniti Every provider call that reaches a successful finish appends exactly one `assistant/message` completion anchor, including content-less calls and `max-tokens` finishes. A successful `agent/step-result` stores its transformed content; a rejected result records empty content before the original failure continues. The anchor retains exact chunk provenance (`[]` for a stream with no chunks) and usage when available, while empty content stays out of derived message history. -Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and `agent/post-step` remain ordinary turn failures. Recovery observes a closed failed step, and a retry rebuilds the request from the durable log in a new numbered step. Cancellation clears pending work and aborts the current step without leaking to the next prompt; undispatched model tool calls receive synthetic `tool/call` and aborted result pairs. Terminal continuation stops remain authoritative through turn close and durability flush. +Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and `agent/post-step` remain ordinary turn failures. Recovery receives the exact live error, immutable provider facts, and immutable prior failures after the failed step closes. A retry rebuilds from the durable log in a new numbered step, success clears the consecutive history, and exhaustion records the structured failure once on `turn/end`. Cancellation clears pending work and aborts the current step without leaking to the next prompt; undispatched model tool calls receive synthetic `tool/call` and aborted result pairs. Terminal continuation stops remain authoritative through turn close and durability flush. Within a step, exclusive calls form barriers; parallel-safe calls use a bounded rolling pool and are reclassified before start. Only dispatch/body overlaps. Policy, durable results, and result context remain model-ordered. Abort stops new calls, drains started results, then drains accepted batch context before the turn closes through the normal abort path. @@ -63,6 +63,7 @@ Within a step, exclusive calls form barriers; parallel-safe calls use a bounded Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy: - Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute` → `tools/execute` → `tools/post-execute` → `tools/result` pipeline; exact signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md) - Compaction: pressure on `agent/post-step`; canonical context overflow on `agent/request-error` +- Transient model recovery: `dsh-llm-retry` on `agent/request-error`, with finite code-specific budgets and non-surface `llm/retry` status events - Sandbox, permission, plan mode: `tools/pre-execute` for extensible deny/ask, `tools.guard()` for monotonic owner policy, `tools/post-execute` for result decisions, and `tools/result` for final observation - Sub-agents: implemented outside the loop as `ctx.subagents` providers; in-process providers use `ctx.agents.create()` and owned `AgentHandle` teardown, while generic [`ctx.tasks`](../../tasks/tasks/) plus [`dsh-tool-subagent`](../../subagent/tool-subagent/) own background collection. - Persistence: `session/event` + `session/flush` diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 9016c16d9b..926674f866 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -6,9 +6,9 @@ */ import type { Context } from 'cordis' -import type { ContentBlock, FinishReason, GenerateOptions, LlmCallConfig, Message } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, FinishReason, GenerateOptions, LlmCallConfig, LlmFailure, Message } from '@deepseek-ai/dsh-llm' import { isDeepStrictEqual } from 'node:util' -import { BlockAssembler, HarnessError, assertNever, deepFreeze, isLlmAdapterFailure } from '@deepseek-ai/dsh-llm' +import { BlockAssembler, HarnessError, LlmError, assertNever, deepFreeze, llmFailureOf } from '@deepseek-ai/dsh-llm' import { agentEvents, assembleContextFor } from '@deepseek-ai/dsh-agent' import type { AgentEventDispatch, ContinuationDecision, HookContext, PromptDecision, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent' import { canonicalHeader } from '@deepseek-ai/dsh-session' @@ -28,24 +28,27 @@ function toError(error: unknown): RequestError { /** Distinguishes final model-request failures from failures in later step processing. */ class TerminalModelRequestFailure extends Error { - constructor(readonly requestError: RequestError) { + constructor( + readonly requestError: RequestError, + readonly failure: LlmFailure, + ) { super(requestError.message, { cause: requestError }) this.name = 'TerminalModelRequestFailure' } } /** Convert terminal failure finishes into step errors; unknown extensible finishes remain successful. */ -function finishError(finish: FinishReason): RequestError | undefined { +function finishError(finish: FinishReason): { error: RequestError; failure: LlmFailure } | undefined { switch (finish.kind) { - case 'error': { - const error: RequestError = new Error(finish.message) - if (finish.code !== undefined) error.code = finish.code - return error - } + case 'error': case 'aborted': { - const error: RequestError = new Error('model stream aborted') - error.code = 'ABORTED' - return error + const facts = finish.failure + const error = new LlmError(facts.message, facts.code, { + ...facts.status === undefined ? {} : { status: facts.status }, + ...facts.retryAfterMs === undefined ? {} : { retryAfterMs: facts.retryAfterMs }, + ...facts.requestId === undefined ? {} : { requestId: facts.requestId }, + }) + return { error, failure: error.failure } } // stop / tool-calls / max-tokens / plugin-added kinds → not a failure. default: @@ -191,7 +194,7 @@ async function runTurn( let reason: TurnEndReason = { kind: 'completed' } let step = 0 - let requestRetryAttempt = 0 + let requestFailureHistory: readonly LlmFailure[] = Object.freeze([]) let stepOpen = false let errorReported = false let terminalStopped = false @@ -204,10 +207,12 @@ async function runTurn( } // Record the durable turn failure once and contain the live error notification. - const failTurn = (err: RequestError): void => { + const failTurn = (err: RequestError, failure?: LlmFailure): void => { if (errorReported) return errorReported = true - reason = { kind: 'error', step, ...errorData(err) } + reason = failure === undefined + ? { kind: 'error', step, ...errorData(err) } + : { kind: 'error', step, failure } try { events.emit('agent/error', turn, step, err) } catch { @@ -353,14 +358,14 @@ async function runTurn( let stepOutcome: | { hadToolCalls: boolean; finish: FinishReason } - | { requestError: RequestError } + | { requestError: RequestError; failure: LlmFailure } | { error: RequestError } try { stepOutcome = await runStep( ctx, events, handle, turn, step, assembly, fullSystemPrompt, boundaryMessages, transmission, abort.signal) } catch (error: unknown) { if (error instanceof TerminalModelRequestFailure) { - stepOutcome = { requestError: error.requestError } + stepOutcome = { requestError: error.requestError, failure: error.failure } } else { stepOutcome = { error: toError(error) } } @@ -383,7 +388,7 @@ async function runTurn( try { recoveryDecision = await events.waterfall( 'agent/request-error', turn, step, stepOutcome.requestError, - requestRetryAttempt, abort.signal, + stepOutcome.failure, requestFailureHistory, abort.signal, () => Promise.resolve(defaultDecision), ) } catch (recoveryError: unknown) { @@ -404,10 +409,10 @@ async function runTurn( } switch (recoveryDecision.action) { case 'retry': - requestRetryAttempt += 1 + requestFailureHistory = Object.freeze([...requestFailureHistory, stepOutcome.failure]) continue case 'fail': - failTurn(stepOutcome.requestError) + failTurn(stepOutcome.requestError, stepOutcome.failure) break /* v8 ignore next -- closed-union exhaustiveness guard */ default: @@ -435,7 +440,7 @@ async function runTurn( break } - requestRetryAttempt = 0 + requestFailureHistory = Object.freeze([]) // Preserve max-token completion unless a later disposal, abort, or error wins. const stepReason = stepFinishReason(stepOutcome.finish) @@ -635,13 +640,14 @@ async function runStep( assembler.push(chunk) } } catch (error: unknown) { - if (isLlmAdapterFailure(stream, error)) throw new TerminalModelRequestFailure(error) + const failure = llmFailureOf(stream, error) + if (failure !== undefined && error instanceof Error) throw new TerminalModelRequestFailure(error, failure) throw error } // Normalize failure finish chunks into the same path as thrown stream errors. const stepError = finishError(assembler.finish) - if (stepError) throw new TerminalModelRequestFailure(stepError) + if (stepError) throw new TerminalModelRequestFailure(stepError.error, stepError.failure) const recordAssistantMessage = ( assembledContent: ContentBlock[], diff --git a/packages/core/agent-loop/tests/contract-regressions.spec.ts b/packages/core/agent-loop/tests/contract-regressions.spec.ts index 80d085c32b..f79f917281 100644 --- a/packages/core/agent-loop/tests/contract-regressions.spec.ts +++ b/packages/core/agent-loop/tests/contract-regressions.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' -import LlmService, { CallId, ContentBlock, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm' +import LlmService, { CallId, ContentBlock, MessageSource, ProviderRequestId, StreamChunk } from '@deepseek-ai/dsh-llm' import SessionStore, { Session, SessionEvent, SessionId, TurnEndReason } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { defineTool, type PostToolDecision } from '@deepseek-ai/dsh-tools' @@ -925,8 +925,15 @@ describe('discriminated SessionEvent narrows without casts', () => { describe('a finish-error stream chunk ends the turn as error, not completed', () => { it('translates finish {kind:error} into a turn error with a logged error event', async () => { // A finish-error chunk must not produce a completed assistant turn. + const failure = { + message: 'provider 401', + code: 'AUTH', + status: 401, + retryAfterMs: 2_000, + requestId: ProviderRequestId('finish-request-1'), + } const errorStream: StreamChunk[] = [ - { type: 'finish', reason: { kind: 'error', message: 'provider 401', code: 'AUTH' } }, + { type: 'finish', reason: { kind: 'error', failure } }, ] const adapter = new MockAdapter([errorStream]) const ctx = await harness(adapter) @@ -938,20 +945,20 @@ describe('a finish-error stream chunk ends the turn as error, not completed', () send(agent, 'go') await waitForIdle(ctx, agent) - expect(reasons).toEqual([{ kind: 'error', step: 1, message: 'provider 401', code: 'AUTH' }]) + expect(reasons).toEqual([{ kind: 'error', step: 1, failure }]) const events = [...agent.session.events] // The durable failure lives on turn/end.reason (with the failing step), not // a standalone error event. const turnEnd = events.find(event => event.type === 'turn/end') - expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason).toEqual({ kind: 'error', step: 1, message: 'provider 401', code: 'AUTH' }) + expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason).toEqual({ kind: 'error', step: 1, failure }) // A failed step must not synthesize an assistant message. expect(events.some(event => event.type === 'assistant/message')).toBe(false) }) it('translates finish {kind:aborted} into a turn error coded ABORTED', async () => { const abortedStream: StreamChunk[] = [ - { type: 'finish', reason: { kind: 'aborted' } }, + { type: 'finish', reason: { kind: 'aborted', failure: { message: 'model stream aborted', code: 'ABORTED' } } }, ] const adapter = new MockAdapter([abortedStream]) const ctx = await harness(adapter) @@ -963,13 +970,13 @@ describe('a finish-error stream chunk ends the turn as error, not completed', () send(agent, 'go') await waitForIdle(ctx, agent) - expect(reasons).toEqual([{ kind: 'error', step: 1, message: 'model stream aborted', code: 'ABORTED' }]) + expect(reasons).toEqual([{ kind: 'error', step: 1, failure: { message: 'model stream aborted', code: 'ABORTED' } }]) expect([...agent.session.events].some(event => event.type === 'assistant/message')).toBe(false) }) it('handles a finish error without a code (code key omitted)', async () => { const errorStream: StreamChunk[] = [ - { type: 'finish', reason: { kind: 'error', message: 'codeless failure' } }, + { type: 'finish', reason: { kind: 'error', failure: { message: 'codeless failure', code: 'UNKNOWN' } } }, ] const adapter = new MockAdapter([errorStream]) const ctx = await harness(adapter) @@ -981,7 +988,7 @@ describe('a finish-error stream chunk ends the turn as error, not completed', () send(agent, 'go') await waitForIdle(ctx, agent) - expect(reasons).toEqual([{ kind: 'error', step: 1, message: 'codeless failure' }]) + expect(reasons).toEqual([{ kind: 'error', step: 1, failure: { message: 'codeless failure', code: 'UNKNOWN' } }]) }) }) @@ -1101,7 +1108,7 @@ describe('turn and step boundary recovery', () => { }) it('a one-shot turn/end validation failure preserves the earlier turn error on retry', async () => { - const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', message: 'provider failed' } }] + const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', failure: { message: 'provider failed', code: 'UNKNOWN' } } }] const adapter = new MockAdapter([errorStream]) const ctx = await balancedHarness(adapter) const agent = ctx.agentLoop.create(SessionId('a-turnend-veto'), { provider: 'mock', model: 'mock' }) @@ -1131,7 +1138,7 @@ describe('turn and step boundary recovery', () => { const turnEnd = agent.session.events.findLast(event => event.type === 'turn/end') expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason).toMatchObject({ kind: 'error', - message: 'provider failed', + failure: { message: 'provider failed', code: 'UNKNOWN' }, }) }) @@ -1167,7 +1174,7 @@ describe('turn and step boundary recovery', () => { it('a throwing agent/error listener during a step-error path still balances the turn, loop survives', async () => { // Listener failure cannot interrupt error finalization or the next turn. - const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', message: 'provider 500' } }] + const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', failure: { message: 'provider 500', code: 'SERVER' } } }] const adapter = new MockAdapter([errorStream, textResponse('turn 2 ok')]) const ctx = await balancedHarness(adapter) const agent = ctx.agentLoop.create(SessionId('a-errorlistener'), { provider: 'mock', model: 'mock' }) @@ -1183,7 +1190,11 @@ describe('turn and step boundary recovery', () => { expect(c.turnStart).toBe(1) expect(c.turnEnd).toBe(1) expect(c.stepStart).toBe(c.stepEnd) - expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toMatchObject({ kind: 'error', step: 1, message: 'provider 500' }) + expect(c.lastTurnEnd?.type === 'turn/end' && c.lastTurnEnd.data.reason).toMatchObject({ + kind: 'error', + step: 1, + failure: { message: 'provider 500', code: 'SERVER' }, + }) // loop survives: a second turn runs to completion (invariants oracle would // throw on its turn/start if turn 1 had been left open). @@ -1328,7 +1339,7 @@ describe('turn and step boundary recovery', () => { it('a throwing step/end observer cannot interrupt error finalization', async () => { // Observer failure after step/end commit cannot interrupt turn finalization. - const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', message: 'provider 500' } }] + const errorStream: StreamChunk[] = [{ type: 'finish', reason: { kind: 'error', failure: { message: 'provider 500', code: 'SERVER' } } }] const adapter = new MockAdapter([errorStream, textResponse('turn 2 ok')]) const ctx = await harness(adapter) const agent = ctx.agentLoop.create(SessionId('a-stependthrow'), { provider: 'mock', model: 'mock' }) diff --git a/packages/core/agent-loop/tests/coverage-edges.spec.ts b/packages/core/agent-loop/tests/coverage-edges.spec.ts index a1d449d273..2c5cf06691 100644 --- a/packages/core/agent-loop/tests/coverage-edges.spec.ts +++ b/packages/core/agent-loop/tests/coverage-edges.spec.ts @@ -177,7 +177,9 @@ describe('toError normalization', () => { // String() of { code: 500 } is '[object Object]' expect(errors[0]!.message).toBe('[object Object]') const turnEnd = agent.session.events.find(e => e.type === 'turn/end') - expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason.kind === 'error' && turnEnd.data.reason.code).toBe('UNKNOWN') + expect(turnEnd?.type === 'turn/end' && turnEnd.data.reason.kind === 'error' + && ('failure' in turnEnd.data.reason ? turnEnd.data.reason.failure.code : turnEnd.data.reason.code)) + .toBe('UNKNOWN') }) }) @@ -208,7 +210,8 @@ describe('coded error data emission', () => { const turnEnd = agent.session.events.find(e => e.type === 'turn/end') expect(turnEnd).toBeDefined() if (turnEnd?.type === 'turn/end' && turnEnd.data.reason.kind === 'error') { - expect(turnEnd.data.reason.code).toBe('RATE_LIMIT') + expect('failure' in turnEnd.data.reason ? turnEnd.data.reason.failure.code : turnEnd.data.reason.code) + .toBe('RATE_LIMIT') } }) }) diff --git a/packages/core/agent-loop/tests/request-recovery.spec.ts b/packages/core/agent-loop/tests/request-recovery.spec.ts index bfbcad23ba..72688b96bc 100644 --- a/packages/core/agent-loop/tests/request-recovery.spec.ts +++ b/packages/core/agent-loop/tests/request-recovery.spec.ts @@ -5,8 +5,9 @@ import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter, LlmError, + ProviderRequestId, } from '@deepseek-ai/dsh-llm' -import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, LlmFailure, StreamChunk } from '@deepseek-ai/dsh-llm' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' @@ -258,16 +259,17 @@ describe('agent post-step and request-error lifecycle', () => { it.each([ ['thrown', contextError()], - ['in-band', [{ type: 'finish', reason: { kind: 'error', message: 'too large', code: CONTEXT_WINDOW_EXCEEDED_CODE } }] satisfies StreamChunk[]], + ['in-band', [{ type: 'finish', reason: { kind: 'error', failure: { message: 'too large', code: CONTEXT_WINDOW_EXCEEDED_CODE, status: 400 } } }] satisfies StreamChunk[]], ] as const)('recovers a %s request failure in a new reconstructable step', async (_style, failure) => { const adapter = new FailureScriptAdapter([failure, textResponse('recovered')]) const ctx = await harness(adapter) const agent = ctx.agentLoop.create(SessionId(`recover-${_style}`), { provider: 'mock', model: 'mock' }) const attempts: number[] = [] - ctx.on('agent/request-error', async (subject, turn, step, error, attempt) => { + ctx.on('agent/request-error', async (subject, turn, step, error, facts, history) => { expect(subject).toBe(agent) expect({ turn, step, code: error.code }).toEqual({ turn: 1, step: 1, code: CONTEXT_WINDOW_EXCEEDED_CODE }) - attempts.push(attempt) + expect(facts.code).toBe(CONTEXT_WINDOW_EXCEEDED_CODE) + attempts.push(history.length) subject.session.append('context/message', { content: [{ type: 'text', text: 'RECOVERY SURFACE MUTATION' }], source: { kind: 'plugin', plugin: 'test-recovery' }, @@ -295,7 +297,7 @@ describe('agent post-step and request-error lifecycle', () => { const agent = ctx.agentLoop.create(SessionId(`stream-plugin-${_name.replaceAll(' ', '-')}`), { provider: 'mock', model: 'mock' }) let recoveries = 0 install(ctx) - ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => { recoveries += 1 return next() }) @@ -326,7 +328,7 @@ describe('agent post-step and request-error lifecycle', () => { }) const agent = ctx.agentLoop.create(SessionId('nested-stream-not-recoverable'), { provider: 'mock', model: 'mock' }) let recoveries = 0 - ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => { recoveries += 1 return next() }) @@ -359,7 +361,7 @@ describe('agent post-step and request-error lifecycle', () => { } const agent = ctx.agentLoop.create(SessionId(`${boundary}-not-recoverable`), { provider: 'mock', model: 'mock' }) let recoveries = 0 - ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => { recoveries += 1 return next() }) @@ -387,7 +389,7 @@ describe('agent post-step and request-error lifecycle', () => { } const agent = ctx.agentLoop.create(SessionId(`${failure}-not-recoverable`), { provider: 'mock', model: 'mock' }) let recoveries = 0 - ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, _signal, next) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => { recoveries += 1 return next() }) @@ -406,7 +408,7 @@ describe('agent post-step and request-error lifecycle', () => { const ctx = await harness(makeAdapter(original)) const agent = ctx.agentLoop.create(SessionId(`identity-${_name.replaceAll(' ', '-')}`), { provider: 'mock', model: 'mock' }) let seen: Error | undefined - ctx.on('agent/request-error', async (_agent, _turn, _step, error, _attempt, _signal, next) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, error, _failure, _history, _signal, next) => { seen = error return next() }) @@ -417,12 +419,52 @@ describe('agent post-step and request-error lifecycle', () => { expect(seen).toBe(original) }) + it('passes structured facts beside the original Error and records them on exhaustion', async () => { + const original = new LlmError('provider busy', 'RATE_LIMIT', { + status: 429, + retryAfterMs: 2_000, + requestId: ProviderRequestId('req-9'), + }) + Object.freeze(original) + const ctx = await harness(new SynchronousDispatchFailureAdapter(original)) + const agent = ctx.agentLoop.create(SessionId('structured-request-failure'), { provider: 'mock', model: 'mock' }) + let seenError: Error | undefined + let seenFailure: LlmFailure | undefined + let seenHistory: readonly LlmFailure[] | undefined + ctx.on('agent/request-error', async ( + _agent, _turn, _step, error, failure, history, _signal, next, + ) => { + seenError = error + seenFailure = failure + seenHistory = history + return next() + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(seenError).toBe(original) + expect(seenFailure).toEqual({ + message: 'provider busy', + code: 'RATE_LIMIT', + status: 429, + retryAfterMs: 2_000, + requestId: ProviderRequestId('req-9'), + }) + expect(seenHistory).toEqual([]) + expect(Object.isFrozen(seenHistory)).toBe(true) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'error', step: 1, failure: seenFailure } }, + }) + }) + it('classifies iterator construction and explicit NO_ADAPTER as model-request failures', async () => { for (const scenario of ['iterator', 'no-adapter'] as const) { const ctx = scenario === 'iterator' ? await harness(new IteratorConstructionFailureAdapter()) : await harness() const agent = ctx.agentLoop.create(SessionId(`request-boundary-${scenario}`), { provider: 'mock', model: 'mock' }) let seen = '' - ctx.on('agent/request-error', async (_agent, _turn, _step, error, _attempt, _signal, next) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, error, _failure, _history, _signal, next) => { seen = error.code ?? '' return next() }) @@ -436,14 +478,17 @@ describe('agent post-step and request-error lifecycle', () => { const capped = new FailureScriptAdapter([contextError('first overflow'), contextError('second overflow')]) const cappedCtx = await harness(capped) const cappedAgent = cappedCtx.agentLoop.create(SessionId('retry-cap'), { provider: 'mock', model: 'mock' }) - const cappedAttempts: number[] = [] - cappedCtx.on('agent/request-error', async (_agent, _turn, _step, _error, attempt, _signal, next) => { - cappedAttempts.push(attempt) - return attempt < 1 ? { action: 'retry' } : next() + const cappedHistories: string[][] = [] + cappedCtx.on('agent/request-error', async ( + _agent, _turn, _step, _error, _failure, history, _signal, next, + ) => { + const codes = history.map(entry => entry.code) + cappedHistories.push(codes) + return codes.length < 1 ? { action: 'retry' } : next() }) send(cappedAgent) await waitForIdle(cappedCtx, cappedAgent) - expect(cappedAttempts).toEqual([0, 1]) + expect(cappedHistories).toEqual([[], [CONTEXT_WINDOW_EXCEEDED_CODE]]) const reset = new FailureScriptAdapter([ contextError('first overflow'), @@ -458,14 +503,16 @@ describe('agent post-step and request-error lifecycle', () => { async execute() { return [{ type: 'text', text: 'worked' }] }, })) const resetAgent = resetCtx.agentLoop.create(SessionId('retry-reset'), { provider: 'mock', model: 'mock' }) - const resetAttempts: { step: number; attempt: number }[] = [] - resetCtx.on('agent/request-error', async (_agent, _turn, step, _error, attempt, _signal, next) => { - resetAttempts.push({ step, attempt }) - return resetAttempts.length === 1 ? { action: 'retry' } : next() + const resetHistories: { step: number; codes: string[] }[] = [] + resetCtx.on('agent/request-error', async ( + _agent, _turn, step, _error, _failure, history, _signal, next, + ) => { + resetHistories.push({ step, codes: history.map(entry => entry.code) }) + return resetHistories.length === 1 ? { action: 'retry' } : next() }) send(resetAgent) await waitForIdle(resetCtx, resetAgent) - expect(resetAttempts).toEqual([{ step: 1, attempt: 0 }, { step: 3, attempt: 0 }]) + expect(resetHistories).toEqual([{ step: 1, codes: [] }, { step: 3, codes: [] }]) }) it('preserves the original provider error when recovery throws', async () => { @@ -479,7 +526,7 @@ describe('agent post-step and request-error lifecycle', () => { expect(agent.session.events.at(-1)).toMatchObject({ type: 'turn/end', - data: { reason: { kind: 'error', message: 'original overflow', code: CONTEXT_WINDOW_EXCEEDED_CODE } }, + data: { reason: { kind: 'error', failure: { message: 'original overflow', code: CONTEXT_WINDOW_EXCEEDED_CODE } } }, }) }) @@ -489,7 +536,7 @@ describe('agent post-step and request-error lifecycle', () => { const agent = ctx.agentLoop.create(SessionId(`${action}-recovery`), { provider: 'mock', model: 'mock' }) let entered!: () => void const recoveryEntered = new Promise((resolve) => { entered = resolve }) - ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _attempt, signal) => { + ctx.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, signal) => { entered() await new Promise((resolve) => { signal.addEventListener('abort', () => { resolve() }, { once: true }) diff --git a/packages/core/agent/README.md b/packages/core/agent/README.md index c2b46f210d..4737180007 100644 --- a/packages/core/agent/README.md +++ b/packages/core/agent/README.md @@ -44,7 +44,7 @@ Agent *creation* is provided by the plugin implementing `AgentFactory` (`dsh-age The lifecycle edges have two important local caveats. `agent/created` runs after scoped setup and after both session and agent registry entries exist. Setup is trusted composition-only code; the immediately following non-vetoing `agent/session-start` notification is the first supported startup injection point. `agent/disposed` always means the exact agent has left the registry. AgentLoop emits it after its driver is quiescent, while ordered teardown may still be detaching the session and unwinding the scope; custom agents registered directly own any stronger driver-ordering contract themselves. -Most interception points are cooperative waterfalls returning seam-specific decisions. `agent/pre-step` and `agent/post-step` are serial checkpoints around a step's durable work, while `agent/request-error` is the failed-model-request recovery waterfall: a retry opens a new numbered step after the failed step closes. `agent/turn-stop` is the terminal serial fold: it runs after ordinary continuation and steering folding, and a returned stop remains in force through turn close and flush so later steering cannot create an extra step or turn. Ordinary queued prompts remain intact. The full rationale for scoped dispatch and terminal settlement is in the [agent-scope runtime-design Agent Note](../../../.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way). +Most interception points are cooperative waterfalls returning seam-specific decisions. `agent/pre-step` and `agent/post-step` are serial checkpoints around a step's durable work, while `agent/request-error` is the failed-model-request recovery waterfall: it receives the exact error, normalized failure facts, immutable prior-retried facts, and signal after the failed step closes; a retry opens a new numbered step. `agent/turn-stop` is the terminal serial fold: it runs after ordinary continuation and steering folding, and a returned stop remains in force through turn close and flush so later steering cannot create an extra step or turn. Ordinary queued prompts remain intact. The full rationale for scoped dispatch and terminal settlement is in the [agent-scope runtime-design Agent Note](../../../.agents/notes/implemented/architecture/2026-07-12-agent-scope-runtime-design.md#three-execution-boundaries-are-deliberately-one-way). `PromptDecision.additionalContexts` is an array so every injected context keeps its own source, envelope, and metadata. A `ContinuationDecision` reason is narrower: it becomes a `steering/message`, not a `context/message`, and therefore carries only content and source. diff --git a/packages/core/agent/src/types.ts b/packages/core/agent/src/types.ts index 702861f407..9f19113f0a 100644 --- a/packages/core/agent/src/types.ts +++ b/packages/core/agent/src/types.ts @@ -7,7 +7,7 @@ import type { Context } from 'cordis' import type { Scoped } from '@deepseek-ai/dsh-scope' -import type { ContentBlock, LlmCallConfig, Message, MessageSource } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, LlmCallConfig, LlmFailure, Message, MessageSource } from '@deepseek-ai/dsh-llm' import type { ContextEnvelope, JsonValue, Session, SessionId } from '@deepseek-ai/dsh-session' import type {} from '@deepseek-ai/dsh-system-prompt' declare module '@deepseek-ai/dsh-system-prompt' { @@ -270,12 +270,13 @@ declare module 'cordis' { * @param turn - the open turn number. * @param step - the failed step number. * @param error - the original model-request failure. - * @param retryAttempt - zero-based number of prior recovery retries. + * @param failure - serializable facts normalized at the final adapter boundary. + * @param priorFailures - immutable failures that already authorized another request in this consecutive sequence. * @param signal - the turn abort signal. * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. * @mode waterfall */ - 'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise + 'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], signal: AbortSignal, next: () => Promise): Promise /** * Override whether the turn continues. The default continues after tool * calls or steering and stops otherwise; a continue reason becomes steering. diff --git a/packages/core/session/README.md b/packages/core/session/README.md index a60be8da72..b67caed3ce 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -60,11 +60,11 @@ Durable values need one accepted representation, not a check followed by a secon ### Session event vocabulary (`types.ts`) -The append-only log's event types, enumerated member by member — payloads, surface badges, provenance — in the generated [persistence log event catalog](../../../docs/persistence-catalog.md). Token usage and provider/model/replay provenance ride on `assistant/message`; an operational error's step is on `turn/end.reason` for `kind: 'error'`. +The append-only log's event types, enumerated member by member — payloads, surface badges, provenance — in the generated [persistence log event catalog](../../../docs/persistence-catalog.md). Token usage and provider/model/replay provenance ride on `assistant/message`; an operational error's step is on `turn/end.reason` for `kind: 'error'`, with structured provider facts for a final model-request failure. -Merge-extensible via `SessionEventMap` — a plugin declaration-merges its own types (the compaction seam's `compact/*`, the hook bridges' `hook/*`); merged members appear in the same catalog. +Merge-extensible via `SessionEventMap` — a plugin declaration-merges its own types (the compaction seam's `compact/*`, bounded recovery's non-surface `llm/retry`, the hook bridges' `hook/*`); merged members appear in the same catalog. -Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types for typed turn boundaries — `kind`-tagged instead of strings). +Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types for typed turn boundaries — `kind`-tagged instead of strings). A final model-request error retains one structured `LlmFailure`; other turn errors retain message/code, and both identify the failed step. Every `SessionEvent` carries two optional top-level fields (structural metadata): diff --git a/packages/core/session/src/types.ts b/packages/core/session/src/types.ts index dc34760e9c..9565f97c88 100644 --- a/packages/core/session/src/types.ts +++ b/packages/core/session/src/types.ts @@ -1,5 +1,5 @@ import type { Branded } from '@deepseek-ai/dsh-brand' -import type { AssistantProvenance, CallId, ContentBlock, LlmCallConfig, Message, MessageSource, StreamChunk, TokenUsage, ToolSchema } from '@deepseek-ai/dsh-llm' +import type { AssistantProvenance, CallId, ContentBlock, LlmCallConfig, LlmFailure, Message, MessageSource, StreamChunk, TokenUsage, ToolSchema } from '@deepseek-ai/dsh-llm' import type { JsonValue } from './json.ts' /** Canonical context-tag framing, or caller-owned framing rendered verbatim. */ @@ -102,9 +102,13 @@ export interface TurnEndReasonMap { * The turn failed: a step threw or the model reported a failure. `step` is the * step number the failure occurred on (the operational error's location — the * single durable record of an in-turn failure; live diagnostics also fire via - * `agent/error`). `code` is the error's code when one was attached. + * `agent/error`). Final model-request failures retain their normalized facts + * as one `failure`; other turn failures retain their live Error message/code. */ - error: { kind: 'error'; step: number; message: string; code?: string } + error: { kind: 'error'; step: number } & ( + | { failure: LlmFailure; message?: never; code?: never } + | { message: string; code?: string; failure?: never } + ) disposed: { kind: 'disposed' } /** At least one step reached its output-token ceiling, even if a plugin continued the turn. */ 'max-tokens': { kind: 'max-tokens' } diff --git a/packages/examples/acp-demo/README.md b/packages/examples/acp-demo/README.md index a68fe145c3..d305124547 100644 --- a/packages/examples/acp-demo/README.md +++ b/packages/examples/acp-demo/README.md @@ -35,6 +35,7 @@ Because the package wires no logger entry, an ACP leaf has **nothing to get wron | `skills` | owner defaults | registry-cache, local-provider, and model-facing skill-tool config, routed through `dsh-agent-spine-demo` | | `toolBash` | owner defaults | model-facing bash config routed through `dsh-agent-spine-demo`, including bash's producer-local `enableRunInBackground` | | `toolTasks` | owner defaults | generic `task_output` wait bounds routed through `dsh-agent-spine-demo` | +| `llmRetry` | owner defaults | bounded transient model-request retry policy routed through `dsh-agent-spine-demo` | | `persistenceRoot` | `./.sessions` | the JSONL backend's root directory | The leaf supplies the swappable backends: an LLM adapter (`llm-deepseek` for the real model, `llm-replay` for keyless snapshot replay) and a bash executor. diff --git a/packages/examples/acp-demo/src/index.ts b/packages/examples/acp-demo/src/index.ts index d9baa96394..5453321d09 100644 --- a/packages/examples/acp-demo/src/index.ts +++ b/packages/examples/acp-demo/src/index.ts @@ -55,6 +55,8 @@ export interface Config { toolBash?: NonNullable /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable + /** Bounded transient model-request retry policy forwarded through agent-core. */ + llmRetry?: NonNullable } // Each front door owns a complete, directly readable config schema; extracting @@ -76,6 +78,7 @@ export const Config: z = z.object({ skills: agentCore.SkillConfigSchema, toolBash: agentCore.ToolBashConfigSchema, toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]), + llmRetry: agentCore.LlmRetryConfigSchema, }) /* jscpd:ignore-end */ diff --git a/packages/examples/agent-spine-demo/README.md b/packages/examples/agent-spine-demo/README.md index 9c431d5698..baa01c6819 100644 --- a/packages/examples/agent-spine-demo/README.md +++ b/packages/examples/agent-spine-demo/README.md @@ -17,6 +17,7 @@ Read this package for the whole plugin tree and its composition order. @deepseek-ai/dsh-skill skill provider registry @deepseek-ai/dsh-skill-local local filesystem skill provider @deepseek-ai/dsh-agent agent registry + initiator scope + agent/* events +@deepseek-ai/dsh-llm-retry bounded transient request retry policy @deepseek-ai/dsh-tasks generic background-task registry @deepseek-ai/dsh-invariants dev-mode event-contract assertions @deepseek-ai/dsh-tool-bash the model-facing bash schema @@ -42,19 +43,21 @@ This is the [interface/implementation/consumer seam](../../../.agents/notes/impl ```ts import type { Config } from '@deepseek-ai/dsh-agent-spine-demo' -// { agents?, maxParallelToolCalls?, persona?, toolOrder?, tools?, dshHome?, skills?, workspaceContext, toolBash?, toolTasks? } +// { agents?, maxParallelToolCalls?, persona?, toolOrder?, tools?, dshHome?, skills?, workspaceContext, toolBash?, toolTasks?, llmRetry? } // workspaceContext requires { maxBytes } or false; the other owner schemas supply defaults. ``` -The bundle FORWARDS each field to the child that owns it: `agents` and `maxParallelToolCalls` to `agent-loop` (`agents` defaults to `[]`; the cap defaults there), so each app supplies its own pre-created agents — a stdio app pre-creates `main`, while the ACP app creates agents on demand at `session/new`; `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer; the required `workspaceContext` choice to `dsh-workspace-context` (`{ maxBytes }` enables loading and `false` disables it); and `toolBash`/`toolTasks` to the two model-facing tool plugins the bundle owns. Set `skills.enabled: false` to omit both the local provider and model-facing skill tool, and set `toolTasks: false` to retain the task service for foreground producers without exposing `task_output` / `task_list` / `task_kill`. It resolves `dshHome` once through [`@deepseek-ai/dsh-home`](../../util/home/README.md) and forwards that absolute value to tool-bash's managed environment and enabled local skill discovery. An absent top-level `dshHome` adopts `skills.local.dshHome`; supplying both with different resolved paths fails loudly. `toolBash.enableRunInBackground` controls only the bash producer; independently loaded producers keep their own config. Workspace instructions register before the skill catalog so their session-prefix message renders first. App packages use `pickSpineConfig()` to copy only these bundle-owned fields. +The bundle FORWARDS each field to the child that owns it: `agents` and `maxParallelToolCalls` to `agent-loop` (`agents` defaults to `[]`; the cap defaults there), so each app supplies its own pre-created agents — a stdio app pre-creates `main`, while the ACP app creates agents on demand at `session/new`; `llmRetry` to the bounded retry policy; `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer; the required `workspaceContext` choice to `dsh-workspace-context` (`{ maxBytes }` enables loading and `false` disables it); and `toolBash`/`toolTasks` to the two model-facing tool plugins the bundle owns. Set `skills.enabled: false` to omit both the local provider and model-facing skill tool, and set `toolTasks: false` to retain the task service for foreground producers without exposing `task_output` / `task_list` / `task_kill`. It resolves `dshHome` once through [`@deepseek-ai/dsh-home`](../../util/home/README.md) and forwards that absolute value to tool-bash's managed environment and enabled local skill discovery. An absent top-level `dshHome` adopts `skills.local.dshHome`; supplying both with different resolved paths fails loudly. `toolBash.enableRunInBackground` controls only the bash producer; independently loaded producers keep their own config. Workspace instructions register before the skill catalog so their session-prefix message renders first. App packages use `pickSpineConfig()` to copy only these bundle-owned fields. ## Why a code bundle, not a shared YAML include A YAML include can deduplicate config but cannot own a bin or provide front-door defaults. App packages make stdout-safe ACP wiring the default, though a leaf can still add an unsafe logger. Bundle children register services in the root isolate-keyed store, so injected leaf siblings see them without load-order coupling. +The bounded retry policy may repeat a transiently failed request in a new numbered step. Retry status and failed partial chunks stay outside model history, each provider attempt can still incur billing, front doors derive usage across every logged step, and the reconstructed request preserves the prior prefix for provider cache reuse. + ## Model Experience -Indirectly, through `dsh-system-prompt`, `dsh-tool-skill`, `dsh-tool-bash`, and `dsh-tools`, which this bundle mounts without adding model-bound wrapper content. +Indirectly, through `dsh-system-prompt`, `dsh-tool-skill`, `dsh-tool-bash`, `dsh-tools`, and `dsh-llm-retry`, which this bundle mounts without adding model-bound wrapper content. #### KV Cache effect diff --git a/packages/examples/agent-spine-demo/package.json b/packages/examples/agent-spine-demo/package.json index 772d6c059b..2c73226f3c 100644 --- a/packages/examples/agent-spine-demo/package.json +++ b/packages/examples/agent-spine-demo/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-agent-spine-demo", - "description": "The default executor-less/UI-less agent spine as one Cordis bundle plugin (timer + llm + sessions + system-prompt + tools + skills + agents + tasks + invariants + tool-bash + workspace-context + tool-skill + tool-tasks + agent-loop)", + "description": "The default executor-less/UI-less agent spine as one Cordis bundle plugin (timer + llm + sessions + system-prompt + tools + skills + agents + bounded retry + tasks + invariants + tool-bash + workspace-context + tool-skill + tool-tasks + agent-loop)", "version": "0.0.1", "private": true, "type": "module", @@ -28,6 +28,7 @@ "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-home": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-llm-retry": "^0.0.1", "@deepseek-ai/dsh-workspace-context": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "@deepseek-ai/dsh-skill": "^0.0.1", @@ -48,6 +49,7 @@ "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-home": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-workspace-context": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-skill": "workspace:^", diff --git a/packages/examples/agent-spine-demo/src/index.ts b/packages/examples/agent-spine-demo/src/index.ts index 74ba5e0cc9..c8d5dd7d65 100644 --- a/packages/examples/agent-spine-demo/src/index.ts +++ b/packages/examples/agent-spine-demo/src/index.ts @@ -25,6 +25,7 @@ import * as workspaceContext from '@deepseek-ai/dsh-workspace-context' import * as toolSkill from '@deepseek-ai/dsh-tool-skill' import * as toolTasks from '@deepseek-ai/dsh-tool-tasks' import AgentLoop, { type Config as AgentLoopConfig } from '@deepseek-ai/dsh-agent-loop' +import * as llmRetry from '@deepseek-ai/dsh-llm-retry' import { resolveDshHome } from '@deepseek-ai/dsh-home' export const name = 'agent-spine-demo' @@ -77,6 +78,8 @@ export interface Config { toolBash?: toolBash.Config /** Generic background-task controls; set false to keep the task service without model-facing task tools. */ toolTasks?: toolTasks.Config | false + /** Bounded transient model-request retry policy. */ + llmRetry?: llmRetry.Config } /** The skill config schema exported for app packages that forward `skills`. */ @@ -93,6 +96,9 @@ export const ToolBashConfigSchema: z = toolBash.Config /** The task-control-tool config schema exported for app packages that forward `toolTasks`. */ export const ToolTasksConfigSchema: z = toolTasks.Config +/** The bounded LLM retry schema exported for app packages that forward `llmRetry`. */ +export const LlmRetryConfigSchema: z = llmRetry.Config + /** Intersect the owners' schemas so validation + defaulting stay identical. */ export const Config = z.intersect([ AgentLoop.Config, @@ -104,7 +110,8 @@ export const Config = z.intersect([ workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), toolBash: ToolBashConfigSchema, toolTasks: z.union([z.const(false), ToolTasksConfigSchema]), - }) as unknown as z>, + llmRetry: LlmRetryConfigSchema, + }) as unknown as z>, ]) as unknown as z /** @@ -123,6 +130,7 @@ export function pickSpineConfig(config: Omit): Omit block.type === 'text' ? block.text : '').join('\n') ?? '' } +class TransientOnceAdapter extends LlmAdapter { + requests = 0 + + async * stream(_options: GenerateOptions): AsyncIterable { + this.requests += 1 + if (this.requests === 1) throw new LlmError('temporary outage', 'SERVER') + yield* textResponse('recovered by bundled policy') + } +} + describe('dsh-agent-spine-demo bundle', () => { it('brings up the full default spine', async () => { const ctx = await mount({ workspaceContext: false }) @@ -117,6 +127,37 @@ describe('dsh-agent-spine-demo bundle', () => { await ctx.fiber.dispose() }) + it('loads and configures bounded request recovery for every bundled front door', async () => { + const adapter = new TransientOnceAdapter() + const ctx = await mount({ + workspaceContext: false, + llmRetry: { + maxTransientRetries: 1, + initialDelayMs: 1, + maxDelayMs: 1, + jitterRatio: 0, + }, + }) + ctx.llm.registerAdapter(['mock'], adapter) + const handle = await ctx.agents.create({ + sessionId: SessionId('bundled-retry-session'), + meta: { cwd: process.cwd() }, + agentOptions: { provider: 'mock', model: 'mock' }, + }) + + handle.agent.send([{ type: 'text', text: 'recover' }]) + await waitForIdle(ctx, handle.agent) + + expect(adapter.requests).toBe(2) + const retryEvents = handle.agent.session.events.filter(event => event.type === 'llm/retry') + expect(retryEvents).toHaveLength(1) + expect(retryEvents[0]?.data.retry).toBe(1) + expect(retryEvents[0]?.data.maxRetries).toBe(1) + expect(messageText(handle.agent.session.deriveMessages().at(-1))).toBe('recovered by bundled policy') + await handle.dispose() + await ctx.fiber.dispose() + }) + it('includes the skill registry, local provider, and skill tool without builtin skills', async () => { const ctx = await mount({ workspaceContext: false }) @@ -370,6 +411,7 @@ describe('dsh-agent-spine-demo bundle', () => { skills: { enabled: false }, toolBash: { enableRunInBackground: false }, toolTasks: false as const, + llmRetry: { maxTransientRetries: 1, jitterRatio: 0 }, } expect(agentCore.pickSpineConfig(appConfig)).toEqual({ @@ -381,6 +423,7 @@ describe('dsh-agent-spine-demo bundle', () => { skills: appConfig.skills, toolBash: appConfig.toolBash, toolTasks: appConfig.toolTasks, + llmRetry: appConfig.llmRetry, }) expect(agentCore.pickSpineConfig({ workspaceContext: false })).toEqual({ workspaceContext: false }) }) diff --git a/packages/examples/agent-spine-demo/tsconfig.json b/packages/examples/agent-spine-demo/tsconfig.json index 89cb2accd8..9b897d1674 100644 --- a/packages/examples/agent-spine-demo/tsconfig.json +++ b/packages/examples/agent-spine-demo/tsconfig.json @@ -47,6 +47,9 @@ { "path": "../../core/agent-loop" }, + { + "path": "../../llm/llm-retry" + }, { "path": "../../support/invariants" }, diff --git a/packages/examples/cli-demo/README.md b/packages/examples/cli-demo/README.md index 760a0f60e1..ebd104fc8d 100644 --- a/packages/examples/cli-demo/README.md +++ b/packages/examples/cli-demo/README.md @@ -18,6 +18,7 @@ The package mounts no console logger, readline UI, user-interaction service, or | `skills` | owner defaults | skill registry, local provider, and model-facing skill tool | | `toolBash` | owner defaults | model-facing bash config, including this producer's background opt-in | | `toolTasks` | owner defaults | generic `task_output` wait bounds | +| `llmRetry` | owner defaults | bounded transient model-request retry policy | | `persistenceRoot` | `./.sessions` | JSONL session root | | `workspaceContext` | required | workspace-instruction byte budget, or `false` to disable loading | @@ -40,7 +41,7 @@ Loader configs with bare package specifiers require `node --expose-internals` or ### Output formats - `text` writes the last assistant message containing text, followed by one newline. -- `json` writes one DSH-native result record: `{ type: "result", success, sessionId, turn, result, reason, usage? }`. `usage` sums every model step in the task turn. +- `json` writes one DSH-native result record: `{ type: "result", success, sessionId, turn, result, reason, usage? }`. `usage` sums each model step in the task turn once, including billed failed retry attempts that produced usage without a committed assistant message. - `stream-json` writes each canonical event from the top-level session's task turn as `{ type: "session_event", sessionId, event }`, then the same result record. Child-agent activity appears only through the parent tool events and results. Only `reason.kind === "completed"` exits successfully. Other durable turn endings still emit partial text or a result record, add a stderr diagnostic, and exit nonzero. Argument and boot failures leave stdout empty. SIGINT and SIGTERM cancel active work, await disposal, and exit 130 and 143 respectively. diff --git a/packages/examples/cli-demo/src/cli.ts b/packages/examples/cli-demo/src/cli.ts index 68c9598e6c..a2b61f85f6 100644 --- a/packages/examples/cli-demo/src/cli.ts +++ b/packages/examples/cli-demo/src/cli.ts @@ -219,7 +219,7 @@ export async function runOneShot(ctx: Context, options: OneShotOptions): Promise let targetTurn: number | undefined let reason: TurnEndReason | undefined let result = '' - let usage: TokenUsage | undefined + const usageByStep = new Map() let outputError: Error | undefined let resolveTurn!: () => void let rejectTurn!: (error: Error) => void @@ -254,9 +254,14 @@ export async function runOneShot(ctx: Context, options: OneShotOptions): Promise targetTurn = event.data.turn } observe(session.id, event) + if (event.type === 'assistant/chunk' + && event.data.turn === targetTurn + && event.data.chunk.type === 'usage') { + usageByStep.set(event.data.step, event.data.chunk.usage) + } if (event.type === 'assistant/message' && event.data.turn === targetTurn) { result = assistantText(event) ?? result - if (event.data.usage !== undefined) usage = addUsage(usage, event.data.usage) + if (event.data.usage !== undefined) usageByStep.set(event.data.step, event.data.usage) } if (event.type === 'turn/end' && event.data.turn === targetTurn) { reason = event.data.reason @@ -294,6 +299,7 @@ export async function runOneShot(ctx: Context, options: OneShotOptions): Promise } await ctx.sessions.flush(agent.session) if (outputError !== undefined) throw outputError + const usage = [...usageByStep.values()].reduce(addUsage, undefined) return { type: 'result', success: reason.kind === 'completed', @@ -365,7 +371,7 @@ export function formatTurnFailure(reason: TurnEndReason): string { switch (reason.kind) { case 'completed': return 'completed' case 'aborted': return reason.reason === undefined ? 'was aborted' : `was aborted: ${reason.reason}` - case 'error': return `failed at step ${reason.step}: ${reason.message}` + case 'error': return `failed at step ${reason.step}: ${'failure' in reason ? reason.failure.message : reason.message}` case 'disposed': return 'was disposed' case 'max-tokens': return 'reached the model output-token limit' case 'rejected': return `was rejected: ${reason.reason}` diff --git a/packages/examples/cli-demo/src/index.ts b/packages/examples/cli-demo/src/index.ts index e5c77af9ed..d545b24f42 100644 --- a/packages/examples/cli-demo/src/index.ts +++ b/packages/examples/cli-demo/src/index.ts @@ -42,6 +42,8 @@ export interface Config { toolBash?: NonNullable /** Generic background-task control-tool config forwarded through agent-spine-demo. */ toolTasks?: NonNullable + /** Bounded transient model-request retry policy forwarded through agent-spine-demo. */ + llmRetry?: NonNullable /** Controls automatic AGENTS.md/CLAUDE.md loading; configure a byte budget or set `false`. */ workspaceContext: agentCore.Config['workspaceContext'] } @@ -62,6 +64,7 @@ export const Config: z = z.object({ tools: ToolRegistry.Config, toolBash: agentCore.ToolBashConfigSchema, toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]), + llmRetry: agentCore.LlmRetryConfigSchema, workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), }) /* jscpd:ignore-end */ diff --git a/packages/examples/cli-demo/tests/cli.spec.ts b/packages/examples/cli-demo/tests/cli.spec.ts index fe61a42304..dc9de2e6ef 100644 --- a/packages/examples/cli-demo/tests/cli.spec.ts +++ b/packages/examples/cli-demo/tests/cli.spec.ts @@ -70,6 +70,15 @@ function toolResponse(usage: TokenUsage): StreamChunk[] { ] } +function failedResponse(usage: TokenUsage): StreamChunk[] { + return [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'text-delta', index: 0, text: 'discarded' }, + { type: 'usage', usage }, + { type: 'finish', reason: { kind: 'error', failure: { message: 'temporary', code: 'SERVER' } } }, + ] +} + function reasoningResponse(text: string): StreamChunk[] { return [ { type: 'block-start', index: 0, blockType: 'reasoning' }, @@ -98,6 +107,7 @@ async function harness(script: readonly ScriptEntry[]): Promise { persistenceRoot: root, skills: { local: { dshHome: join(skillHome, '.dsh'), agentsHome: join(skillHome, '.agents') } }, workspaceContext: false, + llmRetry: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 }, }) await new Promise(resolve => setTimeout(resolve, 80)) ctx.llm.registerAdapter(['mock'], new ScriptedAdapter(script)) @@ -323,6 +333,21 @@ describe('runOneShot and executeCli', () => { }) }) + it('counts a failed retry attempt once even though it has no assistant message', async () => { + const failed = { inputTokens: 11, outputTokens: 2, cacheReadTokens: 3 } + const recovered = { inputTokens: 7, outputTokens: 5, reasoningTokens: 4 } + const { ctx } = await harness([failedResponse(failed), textResponse('done', recovered)]) + + const result = await runOneShot(ctx, { task: 'task' }) + + expect(result.usage).toEqual({ + inputTokens: 18, + outputTokens: 7, + cacheReadTokens: 3, + reasoningTokens: 4, + }) + }) + it('keeps the prior text when a later assistant message has no text blocks', async () => { const { ctx } = await harness([ toolResponse({ inputTokens: 1, outputTokens: 1 }), @@ -463,6 +488,7 @@ describe('formatTurnFailure', () => { [{ kind: 'aborted' }, 'was aborted'], [{ kind: 'aborted', reason: 'stop' }, 'was aborted: stop'], [{ kind: 'error', step: 2, message: 'bad' }, 'failed at step 2: bad'], + [{ kind: 'error', step: 3, failure: { message: 'provider bad', code: 'SERVER' } }, 'failed at step 3: provider bad'], [{ kind: 'disposed' }, 'was disposed'], [{ kind: 'max-tokens' }, 'output-token limit'], [{ kind: 'rejected', reason: 'policy' }, 'was rejected: policy'], diff --git a/packages/examples/stdio-demo/README.md b/packages/examples/stdio-demo/README.md index 2d706e3008..a3a8332486 100644 --- a/packages/examples/stdio-demo/README.md +++ b/packages/examples/stdio-demo/README.md @@ -36,6 +36,7 @@ The leaf `cordis.yml` supplies only the **swappable backends** — an LLM adapte | `skills` | owner defaults | registry-cache, local-provider, and model-facing skill-tool config, routed through `dsh-agent-spine-demo` | | `toolBash` | owner defaults | model-facing bash config routed through `dsh-agent-spine-demo`, including bash's producer-local `enableRunInBackground` | | `toolTasks` | owner defaults | generic `task_output` wait bounds routed through `dsh-agent-spine-demo` | +| `llmRetry` | owner defaults | bounded transient model-request retry policy routed through `dsh-agent-spine-demo` | | `persistenceRoot` | `./.sessions` | the JSONL backend's root directory | | `welcome` | `ready.` | terminal banner / TUI subtitle | | `ui` | `{ mode: 'auto' }` | terminal mode (`auto` / `readline` / `tui`) and nested TUI presentation config | diff --git a/packages/examples/stdio-demo/src/index.ts b/packages/examples/stdio-demo/src/index.ts index 0bf66ab007..5e504e6804 100644 --- a/packages/examples/stdio-demo/src/index.ts +++ b/packages/examples/stdio-demo/src/index.ts @@ -99,6 +99,8 @@ export interface Config { toolBash?: NonNullable /** Generic background-task controls forwarded through agent-core; set false to omit their tool surface. */ toolTasks?: NonNullable + /** Bounded transient model-request retry policy forwarded through agent-core. */ + llmRetry?: NonNullable /** * If set, the pre-created agent RESUMES this persisted session id instead of * starting fresh. Sourced from an env var in the leaf `cordis.yml` @@ -126,6 +128,7 @@ export const Config: z = z.object({ skills: agentCore.SkillConfigSchema, toolBash: agentCore.ToolBashConfigSchema, toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]), + llmRetry: agentCore.LlmRetryConfigSchema, resumeSessionId: z.string(), workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(), }) diff --git a/packages/llm/README.md b/packages/llm/README.md index ac08ffafc2..405c2f18f4 100644 --- a/packages/llm/README.md +++ b/packages/llm/README.md @@ -6,7 +6,8 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a |---|---|---| | `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` | | `token-meter/` | Replay-aware request and surface token measurement | `ctx.tokenMeter` | +| `llm-retry/` | Bounded transient request retry policy | (listens to `agent/request-error`) | | `llm-deepseek/` | DeepSeek API adapter (hand-rolled fetch/SSE) | (registers on `ctx.llm`) | | `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) | -The interface lives at `llm/llm/`; adapters and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching the interface. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations and the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership. +The interface lives at `llm/llm/`; adapters, retry policy, and reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. A new provider adapter joins here and registers one or more provider routes on `ctx.llm` without touching the interface. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the contract-validation origin of the two shipping implementations and the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership. diff --git a/packages/llm/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md index 27bf4b626a..33af24b302 100644 --- a/packages/llm/llm-deepseek/README.md +++ b/packages/llm/llm-deepseek/README.md @@ -16,6 +16,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire baseURL: !!js process.env.DEEPSEEK_BASE_URL # default: https://api.deepseek.com thinking: enabled # optional; provider default is enabled reasoningEffort: high # optional; high | max — omitted ⇒ not sent + streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default models: # optional; defaults to V4 Flash and V4 Pro - id: deepseek-v4-flash name: DeepSeek V4 Flash @@ -29,6 +30,8 @@ The plugin registers the single provider route `deepseek`. A request selects it `thinking`/`reasoningEffort` are adapter-level request defaults serialized as the official top-level `thinking: {type}` / `reasoning_effort` wire fields. They live in adapter config (not `GenerateOptions`) to keep the core vocabulary provider-neutral. +`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; agent-level retry is a separate plugin policy. + ## App attribution Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. @@ -42,11 +45,11 @@ Every request carries the shared attribution header from dsh-llm's `attributionH ## Errors -Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `RATE_LIMIT` (429), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_` otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', code: }` chunks. +Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA` (a response whose provider details identify exhausted quota, balance, or credits), `RATE_LIMIT` (other 429s), `CONTEXT_WINDOW_EXCEEDED` (a 400 whose provider code, type, or message identifies context overflow), `INVALID_REQUEST` (other 400s), `SERVER` (5xx), `HTTP_` otherwise. Its serializable `failure` retains the HTTP status plus a valid positive `Retry-After` seconds/date delay and `x-request-id` / `x-deepseek-request-id` when present. Connection failures are `TRANSPORT`; protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or `MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s (e.g. `content_filter`, `insufficient_system_resource`) become `finish {kind: 'error', failure}` chunks. ## Testing -Unit suites run against a local `node:http` mock SSE server (no network). Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback. +Unit suites run against a local `node:http` mock SSE server (no network), including structured HTTP facts, malformed/truncated streams, caller abort, connection failure, and proof that idle timeout aborts the actual body. Real-API coverage lives in `tests/adapter.e2e.ts` (`pnpm run test:e2e`, key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both official effort levels, including the thinking+tools round trip with reasoning passback. ## Model Experience diff --git a/packages/llm/llm-deepseek/package.json b/packages/llm/llm-deepseek/package.json index 1461ad0f44..02e8cb85ac 100644 --- a/packages/llm/llm-deepseek/package.json +++ b/packages/llm/llm-deepseek/package.json @@ -23,6 +23,7 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.7" }, "dependencies": { @@ -30,6 +31,7 @@ }, "devDependencies": { "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index 918c8eee82..b34cc49087 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -5,8 +5,9 @@ * @module dsh-llm-deepseek/adapter */ -import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm' +import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout' import { serializeRequest } from './serialize.ts' import type { RequestDefaults } from './serialize.ts' import { parseSse } from './sse.ts' @@ -33,6 +34,31 @@ export interface DeepSeekAdapterOptions { defaults?: RequestDefaults /** Advisory models exposed to discovery consumers; requests remain unrestricted. */ models?: readonly DeepSeekCatalogModel[] + /** Maximum provider idle time while one stream read is outstanding. */ + streamIdleTimeoutMs?: number +} + +/** Default maximum idle interval while an adapter stream read is outstanding. */ +export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000 +const STREAM_IDLE_TIMEOUT_CODE = 'LLM_STREAM_IDLE_TIMEOUT' + +function retryAfterMs(value: string | null): number | undefined { + if (value === null) return undefined + if (/^\d+$/.test(value)) { + const delay = Number(value) * 1_000 + return Number.isFinite(delay) && delay > 0 ? delay : undefined + } + const delay = Date.parse(value) - Date.now() + return Number.isFinite(delay) && delay > 0 ? delay : undefined +} + +function requestId(headers: Headers): ReturnType | undefined { + const value = headers.get('x-request-id') ?? headers.get('x-deepseek-request-id') + return value === null || value.length === 0 ? undefined : ProviderRequestId(value) +} + +function errorMessage(value: unknown): string { + return value instanceof Error ? value.message : String(value) } /** @@ -43,9 +69,10 @@ export interface DeepSeekAdapterOptions { */ export function httpErrorCode(status: number, error?: WireError['error']): string { if (status === 401 || status === 403) return 'AUTH' + const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ') + if (isQuotaExceededError(detail)) return QUOTA_EXCEEDED_CODE if (status === 429) return 'RATE_LIMIT' if (status === 400) { - const detail = [error?.code, error?.type, error?.message].filter(Boolean).join(' ') if (isContextWindowExceededError(detail)) return CONTEXT_WINDOW_EXCEEDED_CODE return 'INVALID_REQUEST' } @@ -57,13 +84,22 @@ export function httpErrorCode(status: number, error?: WireError['error']): strin * The first real `LlmAdapter`. One instance serves every model name it was * registered under (the harness model name IS the wire model name). * - * Abort: `options.signal` is handed to fetch — both the initial request and - * the body stream reject on abort, which surfaces to the loop as a rejected - * step (the loop already contains step errors). + * One stable signal reaches both initial fetch and body reads. Caller aborts + * map to `ABORTED`; the configured per-read idle watchdog maps to `TIMEOUT`. */ export class DeepSeekAdapter extends LlmAdapter { + private readonly streamIdleTimeoutMs: number + constructor(private readonly options: DeepSeekAdapterOptions) { super() + this.streamIdleTimeoutMs = options.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS + if (!Number.isFinite(this.streamIdleTimeoutMs) + || this.streamIdleTimeoutMs <= 0 + || this.streamIdleTimeoutMs > MAX_TIMER_DELAY_MS) { + throw new Error( + `llm-deepseek: streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`, + ) + } } override providerInfo(provider: string): LlmProviderInfo { @@ -80,6 +116,48 @@ export class DeepSeekAdapter extends LlmAdapter { } async * stream(options: GenerateOptions): AsyncIterable { + const consumer = new AbortController() + const upstream = options.signal === undefined + ? consumer.signal + : AbortSignal.any([options.signal, consumer.signal]) + using watchdog = idleWatchdog(upstream, this.streamIdleTimeoutMs, STREAM_IDLE_TIMEOUT_CODE) + const iterator = this.request(options, watchdog.signal)[Symbol.asyncIterator]() + let exhausted = false + try { + while (true) { + const result = await watchdog.next(iterator) + if (result.done) { + exhausted = true + return + } + yield result.value + } + } catch (error: unknown) { + if (timeoutOf(watchdog.signal, STREAM_IDLE_TIMEOUT_CODE) !== undefined) { + throw new LlmError( + `DeepSeek stream idle timeout after ${this.streamIdleTimeoutMs}ms`, + 'TIMEOUT', + { cause: error }, + ) + } + if (options.signal?.aborted) { + throw new LlmError('DeepSeek request aborted by caller', 'ABORTED', { cause: error }) + } + if (error instanceof LlmError) throw error + throw new LlmError(`DeepSeek transport failed: ${errorMessage(error)}`, 'TRANSPORT', { cause: error }) + } finally { + consumer.abort('DeepSeek stream consumer stopped') + if (!exhausted && iterator.return !== undefined) { + try { + await iterator.return() + } catch (_abortedTransportTeardown) { + // The consumer controller already owns termination; a return-time abort cannot add a second outcome. + } + } + } + } + + private async * request(options: GenerateOptions, signal: AbortSignal): AsyncIterable { const body = serializeRequest(options, this.options.defaults ?? {}) // TODO(http): adopt the Cordis HTTP service when shared transport configuration @@ -96,7 +174,7 @@ export class DeepSeekAdapter extends LlmAdapter { : {}, }, body: JSON.stringify(body), - ...options.signal ? { signal: options.signal } : {}, + signal, }) if (!response.ok) { @@ -110,7 +188,13 @@ export class DeepSeekAdapter extends LlmAdapter { // Only swallow error-body parsing: the HTTP status still identifies the // failure, so malformed gateway JSON must not mask it. } - throw new LlmError(message, httpErrorCode(response.status, providerError)) + const delay = retryAfterMs(response.headers.get('retry-after')) + const id = requestId(response.headers) + throw new LlmError(message, httpErrorCode(response.status, providerError), { + status: response.status, + ...delay === undefined ? {} : { retryAfterMs: delay }, + ...id === undefined ? {} : { requestId: id }, + }) } if (!response.body) { throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE') diff --git a/packages/llm/llm-deepseek/src/index.ts b/packages/llm/llm-deepseek/src/index.ts index f9f223b6ff..c0d0df0f08 100644 --- a/packages/llm/llm-deepseek/src/index.ts +++ b/packages/llm/llm-deepseek/src/index.ts @@ -8,7 +8,8 @@ import type { Context } from 'cordis' import z from 'schemastery' import type {} from '@deepseek-ai/dsh-llm' -import { DeepSeekAdapter } from './adapter.ts' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' +import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts' import type { DeepSeekCatalogModel } from './adapter.ts' export { DeepSeekAdapter } from './adapter.ts' @@ -41,6 +42,8 @@ export interface Config { reasoningEffort?: 'high' | 'max' /** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */ models?: DeepSeekCatalogModel[] + /** Maximum provider idle time while one stream read is outstanding (default five minutes). */ + streamIdleTimeoutMs?: number } const catalogModel: z = z.object({ @@ -55,6 +58,7 @@ export const Config: z = z.object({ thinking: z.union(['enabled', 'disabled']), reasoningEffort: z.union(['high', 'max']), models: z.array(catalogModel).default(DEFAULT_MODELS), + streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS), }) /** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */ @@ -92,5 +96,6 @@ export function apply(ctx: Context, config: Config): void { reasoningEffort: config.reasoningEffort, }, models: resolveModels(config.models), + streamIdleTimeoutMs: config.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS, })) } diff --git a/packages/llm/llm-deepseek/src/translate.ts b/packages/llm/llm-deepseek/src/translate.ts index c66271246c..f0b5eaf789 100644 --- a/packages/llm/llm-deepseek/src/translate.ts +++ b/packages/llm/llm-deepseek/src/translate.ts @@ -35,7 +35,10 @@ export function mapFinishReason(reason: string): FinishReason { case 'length': return { kind: 'max-tokens' } default: // content_filter, insufficient_system_resource, future additions. - return { kind: 'error', message: `model stopped: ${reason}`, code: reason.toUpperCase() } + return { + kind: 'error', + failure: { message: `model stopped: ${reason}`, code: reason.toUpperCase() }, + } } } diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 954a0ecebd..2e23fd49f7 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -2,7 +2,14 @@ import { createServer } from 'node:http' import type { IncomingMessage, Server, ServerResponse } from 'node:http' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from 'cordis' -import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@deepseek-ai/dsh-llm' +import LlmService, { + CONTEXT_WINDOW_EXCEEDED_CODE, + LlmError, + ProviderRequestId, + QUOTA_EXCEEDED_CODE, + userAgent, +} from '@deepseek-ai/dsh-llm' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { SessionId } from '@deepseek-ai/dsh-session' import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek' import { DeepSeekAdapter } from '@deepseek-ai/dsh-llm-deepseek' @@ -12,7 +19,7 @@ import { assemble } from './assemble.ts' /** One scripted behavior for the next request the mock server receives. */ type Behavior = | { kind: 'sse'; events: string[]; delayMs?: number } - | { kind: 'http-error'; status: number; body: string; contentType?: string } + | { kind: 'http-error'; status: number; body: string; contentType?: string; headers?: Record } | { kind: 'close-early'; events: string[] } interface MockServer { @@ -30,6 +37,7 @@ const servers: Server[] = [] afterEach(async () => { await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve)))) vi.unstubAllEnvs() + vi.useRealTimers() }) /** Local chat-completions stand-in: replays scripted behaviors per request. */ @@ -48,7 +56,10 @@ async function mockServer(script: Behavior[]): Promise { return } if (behavior.kind === 'http-error') { - response.writeHead(behavior.status, { 'content-type': behavior.contentType ?? 'application/json' }) + response.writeHead(behavior.status, { + 'content-type': behavior.contentType ?? 'application/json', + ...behavior.headers, + }) response.end(behavior.body) return } @@ -202,6 +213,84 @@ describe('DeepSeekAdapter against a mock server', () => { expect(code).toBe(CONTEXT_WINDOW_EXCEEDED_CODE) }) + it('retains status, Retry-After seconds, and provider request id as structured facts', async () => { + const server = await mockServer([{ + kind: 'http-error', + status: 429, + body: JSON.stringify({ error: { message: 'slow down' } }), + headers: { 'retry-after': '2', 'x-request-id': 'req-429' }, + }]) + const ctx = await harness(server.url) + let thrown: unknown + try { + await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }) + } catch (error: unknown) { + thrown = error + } + expect(thrown).toBeInstanceOf(LlmError) + expect((thrown as LlmError).failure).toEqual({ + message: 'slow down', + code: 'RATE_LIMIT', + status: 429, + retryAfterMs: 2_000, + requestId: ProviderRequestId('req-429'), + }) + }) + + it('parses a future Retry-After HTTP date and the DeepSeek request-id fallback', async () => { + const now = 1_800_000_000_000 + const dateNow = vi.spyOn(Date, 'now').mockReturnValue(now) + try { + const server = await mockServer([{ + kind: 'http-error', + status: 503, + body: JSON.stringify({ error: { message: 'come back later' } }), + headers: { + 'retry-after': new Date(now + 3_000).toUTCString(), + 'x-deepseek-request-id': 'deepseek-503', + }, + }]) + const ctx = await harness(server.url) + await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })) + .rejects.toMatchObject({ + failure: { + message: 'come back later', + code: 'SERVER', + status: 503, + retryAfterMs: 3_000, + requestId: ProviderRequestId('deepseek-503'), + }, + }) + } finally { + dateNow.mockRestore() + } + }) + + it('omits zero, non-finite, invalid, and past Retry-After values', async () => { + const values = [ + '0', + '9'.repeat(400), + 'not-a-date', + new Date(0).toUTCString(), + ] + for (const value of values) { + const server = await mockServer([{ + kind: 'http-error', + status: 429, + body: JSON.stringify({ error: { message: 'retry later' } }), + headers: { 'retry-after': value }, + }]) + const ctx = await harness(server.url) + let thrown: LlmError | undefined + try { + await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }) + } catch (error: unknown) { + if (error instanceof LlmError) thrown = error + } + expect(thrown?.failure).toEqual({ message: 'retry later', code: 'RATE_LIMIT', status: 429 }) + } + }) + it('classifies only context-capacity HTTP 400 details as context overflow', () => { expect(httpErrorCode(400, { message: 'request too large for model context' })) .toBe(CONTEXT_WINDOW_EXCEEDED_CODE) @@ -210,6 +299,12 @@ describe('DeepSeekAdapter against a mock server', () => { expect(httpErrorCode(413, { code: 'context_length_exceeded' })).toBe('HTTP_413') }) + it('distinguishes terminal quota exhaustion from transient HTTP 429 throttling', () => { + expect(httpErrorCode(429, { code: 'insufficient_quota', message: 'account credits exhausted' })) + .toBe(QUOTA_EXCEEDED_CODE) + expect(httpErrorCode(429, { message: 'request rate limit exceeded' })).toBe('RATE_LIMIT') + }) + it('keeps the status-line message for JSON error bodies without a message', async () => { const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }]) const ctx = await harness(server.url) @@ -272,7 +367,76 @@ describe('DeepSeekAdapter against a mock server', () => { })() setTimeout(() => { controller.abort() }, 30) - await expect(pending).rejects.toThrow() + await expect(pending).rejects.toMatchObject({ code: 'ABORTED' }) + }) + + it('maps connection failures to TRANSPORT without losing the cause', async () => { + const cause = new TypeError('connection refused') + const fetchSpy = vi.spyOn(globalThis, 'fetch').mockRejectedValue(cause) + const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'https://example.invalid' }) + try { + const drain = async (): Promise => { + for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ } + } + await expect(drain()).rejects.toMatchObject({ code: 'TRANSPORT', cause }) + } finally { + fetchSpy.mockRestore() + } + }) + + it('renders a non-Error transport rejection without losing its cause', async () => { + const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation(() => { + const failed = Promise.withResolvers() + failed.reject('offline') + return failed.promise + }) + const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'https://example.invalid' }) + try { + const drain = async (): Promise => { + for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ } + } + await expect(drain()).rejects.toMatchObject({ + message: 'DeepSeek transport failed: offline', + code: 'TRANSPORT', + cause: 'offline', + }) + } finally { + fetchSpy.mockRestore() + } + }) + + it('aborts the underlying body when the stream stays idle past its watchdog', async () => { + vi.useFakeTimers() + let stopped = false + const fetchSpy = vi.spyOn(globalThis, 'fetch').mockImplementation((_input, init) => { + const signal = init?.signal + const body = new ReadableStream({ + start(controller) { + signal?.addEventListener('abort', () => { + stopped = true + controller.error(signal.reason) + }, { once: true }) + }, + }) + return Promise.resolve(new Response(body, { status: 200 })) + }) + const adapter = new DeepSeekAdapter({ + apiKey: 'k', + baseURL: 'https://example.invalid', + streamIdleTimeoutMs: 100, + }) + try { + const drain = (async () => { + for await (const _chunk of adapter.stream({ provider: 'deepseek', model: 'm', messages: [] })) { /* drain */ } + })() + const rejected = expect(drain).rejects.toMatchObject({ code: 'TIMEOUT' }) + await vi.advanceTimersByTimeAsync(0) + await vi.advanceTimersByTimeAsync(100) + await rejected + expect(stopped).toBe(true) + } finally { + fetchSpy.mockRestore() + } }) }) @@ -419,4 +583,30 @@ describe('plugin registration and config', () => { expect(adapter).toBeInstanceOf(DeepSeekAdapter) await expect(adapter.listModels('deepseek')).resolves.toEqual([]) }) + + it('rejects invalid idle watchdog bounds for direct and plugin composition', async () => { + expect(() => new DeepSeekAdapter({ + apiKey: 'k', + baseURL: 'http://127.0.0.1:1', + streamIdleTimeoutMs: Number.POSITIVE_INFINITY, + })).toThrow(/streamIdleTimeoutMs.*positive finite/) + expect(() => new DeepSeekAdapter({ + apiKey: 'k', + baseURL: 'http://127.0.0.1:1', + streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1, + })).toThrow(/streamIdleTimeoutMs.*no greater/) + + const ctx = new Context() + await ctx.plugin(LlmService) + await expect(ctx.plugin(LlmDeepSeek, { + apiKey: 'k', + baseURL: 'http://127.0.0.1:1', + streamIdleTimeoutMs: 0, + })).rejects.toThrow(/streamIdleTimeoutMs/) + await expect(ctx.plugin(LlmDeepSeek, { + apiKey: 'k', + baseURL: 'http://127.0.0.1:1', + streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1, + })).rejects.toThrow(/streamIdleTimeoutMs/) + }) }) diff --git a/packages/llm/llm-deepseek/tests/translate.spec.ts b/packages/llm/llm-deepseek/tests/translate.spec.ts index e62cebc4af..4ae833dc4c 100644 --- a/packages/llm/llm-deepseek/tests/translate.spec.ts +++ b/packages/llm/llm-deepseek/tests/translate.spec.ts @@ -232,8 +232,7 @@ describe('mapFinishReason', () => { (wire) => { expect(mapFinishReason(wire)).toEqual({ kind: 'error', - message: `model stopped: ${wire}`, - code: wire.toUpperCase(), + failure: { message: `model stopped: ${wire}`, code: wire.toUpperCase() }, }) }, ) diff --git a/packages/llm/llm-deepseek/tsconfig.json b/packages/llm/llm-deepseek/tsconfig.json index e9de391ba1..5e427d88b9 100644 --- a/packages/llm/llm-deepseek/tsconfig.json +++ b/packages/llm/llm-deepseek/tsconfig.json @@ -19,6 +19,9 @@ }, { "path": "../../llm/llm" + }, + { + "path": "../../util/timeout" } ] } diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index 06395d701c..bedfc5517d 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -19,7 +19,7 @@ Configure credentials and deployment-specific transport settings per provider. O reasoning: high - provider: anthropic apiKey: !!js process.env.ANTHROPIC_API_KEY - maxRetries: 2 + streamIdleTimeoutMs: 300000 - provider: openrouter apiKey: !!js process.env.OPENROUTER_API_KEY headers: @@ -30,7 +30,9 @@ Each provider name must exist in pi-ai's installed catalog and may appear only o The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. -Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `maxRetries`, and `maxRetryDelayMs`. They map to pi-ai's common stream options. Harness app attribution wins a conflicting configured header name. +Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name. + +The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`. ## Provider/model routing and replay @@ -43,7 +45,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state ## Vocabulary differences - pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output. -- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted'}` chunks. Provider-specific error text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`. +- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`. - pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map. - `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers. @@ -57,7 +59,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata ## Testing -Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. Real-API coverage remains key-gated under `pnpm run test:e2e`. +Unit tests use pi-ai catalog models redirected to local mock servers and cover provider/profile routing, one wire request per adapter call, idle-timeout response termination, caller abort, native API selection, endpoint overrides, attribution, conversion, replay-state validation, and cross-provider/model replay within one adapter instance. Real-API coverage remains key-gated under `pnpm run test:e2e`. ## Model Experience @@ -95,3 +97,4 @@ Recorded response content appends to the next request and does not invalidate it - **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field. - **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override. - **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes. +- **Retry policy is not an adapter option** — SDK retries are disabled so durable agent steps and `llm/retry` events own every visible attempt; direct `ctx.llm.stream()` calls remain single-attempt. diff --git a/packages/llm/llm-pi-ai/package.json b/packages/llm/llm-pi-ai/package.json index c922467deb..2a5aed1d76 100644 --- a/packages/llm/llm-pi-ai/package.json +++ b/packages/llm/llm-pi-ai/package.json @@ -23,6 +23,7 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.7" }, "dependencies": { @@ -32,6 +33,7 @@ "devDependencies": { "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-llm-deepseek": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/llm/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts index 7f40c67da3..25240109bb 100644 --- a/packages/llm/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -16,7 +16,9 @@ import type { } from '@earendil-works/pi-ai' import { attributionHeaders, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, LlmModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm' -import type { PiAiProviderProfile } from './config.ts' +import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout' +import { resolveProfiles } from './config.ts' +import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts' import { toPiContext } from './context.ts' import { toStreamChunks } from './stream.ts' @@ -48,8 +50,8 @@ function profileOptions(profile: PiAiProviderProfile): SimpleStreamOptions { ...profile.transport === undefined ? {} : { transport: profile.transport }, ...profile.timeoutMs === undefined ? {} : { timeoutMs: profile.timeoutMs }, ...profile.websocketConnectTimeoutMs === undefined ? {} : { websocketConnectTimeoutMs: profile.websocketConnectTimeoutMs }, - ...profile.maxRetries === undefined ? {} : { maxRetries: profile.maxRetries }, - ...profile.maxRetryDelayMs === undefined ? {} : { maxRetryDelayMs: profile.maxRetryDelayMs }, + // The agent recovery layer owns visible attempts; one adapter call is one SDK attempt. + maxRetries: 0, } } @@ -68,11 +70,11 @@ function requestHeaders(headers: Readonly> | undefined): * request, so models need not be registered during the Cordis lifecycle. */ export class PiAiAdapter extends LlmAdapter { - private readonly profiles: ReadonlyMap + private readonly profiles: ReadonlyMap constructor(options: PiAiAdapterOptions) { super() - this.profiles = new Map(options.profiles.map(profile => [profile.provider, profile])) + this.profiles = new Map(resolveProfiles(options.profiles).map(profile => [profile.provider, profile])) } override listModels(provider: string): Promise { @@ -97,12 +99,12 @@ export class PiAiAdapter extends LlmAdapter { } const model = resolveModel(profile, options.model) - // Pi-ai has no iterator-return cancellation hook. Chain an internal signal - // and abort it when this generator exits so early consumers stop the HTTP stream. - const controller = new AbortController() - const onCallerAbort = (): void => { controller.abort(options.signal?.reason) } - if (options.signal?.aborted) controller.abort(options.signal.reason) - else options.signal?.addEventListener('abort', onCallerAbort, { once: true }) + const consumer = new AbortController() + const upstream = options.signal === undefined + ? consumer.signal + : AbortSignal.any([options.signal, consumer.signal]) + const streamIdleTimeoutMs = profile.streamIdleTimeoutMs + using watchdog = idleWatchdog(upstream, streamIdleTimeoutMs, 'LLM_STREAM_IDLE_TIMEOUT') try { const events = streamSimple(model, toPiContext(options), { @@ -110,15 +112,44 @@ export class PiAiAdapter extends LlmAdapter { ...options.temperature === undefined ? {} : { temperature: options.temperature }, ...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens }, ...options.sessionId === undefined ? {} : { sessionId: String(options.sessionId) }, - signal: controller.signal, + signal: watchdog.signal, // Profile headers are deployment-owned; attribution names are // Harness-owned and therefore win collisions. headers: requestHeaders(profile.headers), }) - yield* toStreamChunks(events, model.contextWindow) + const iterator = toStreamChunks(events, model.contextWindow)[Symbol.asyncIterator]() + let exhausted = false + try { + while (true) { + const result = await watchdog.next(iterator) + const timeout = timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT') + if (timeout !== undefined) throw timeout + if (result.done) { + exhausted = true + return + } + yield result.value + } + } finally { + if (!exhausted) { + consumer.abort('pi-ai stream consumer stopped') + try { + await iterator.return(undefined) + } catch (_abortedSdkTeardown) { + // The stable signal already owns SDK termination; return-time abort cannot add an outcome. + } + } + } + } catch (error: unknown) { + if (timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT') !== undefined) { + throw new LlmError(`pi-ai stream idle timeout after ${streamIdleTimeoutMs}ms`, 'TIMEOUT', { cause: error }) + } + if (options.signal?.aborted) { + throw new LlmError('pi-ai request aborted by caller', 'ABORTED', { cause: error }) + } + throw error } finally { - options.signal?.removeEventListener('abort', onCallerAbort) - controller.abort('consumer stopped streaming') + consumer.abort('pi-ai stream consumer stopped') } } } diff --git a/packages/llm/llm-pi-ai/src/config.ts b/packages/llm/llm-pi-ai/src/config.ts index f7570aff64..d5b5d70867 100644 --- a/packages/llm/llm-pi-ai/src/config.ts +++ b/packages/llm/llm-pi-ai/src/config.ts @@ -7,6 +7,10 @@ import { getProviders } from '@earendil-works/pi-ai' import type { CacheRetention, ThinkingBudgets, ThinkingLevel, Transport } from '@earendil-works/pi-ai' import z from 'schemastery' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' + +/** Default maximum idle interval while an adapter stream read is outstanding. */ +export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000 /** Configuration for one pi-ai provider route. */ export interface PiAiProviderProfile { @@ -30,10 +34,14 @@ export interface PiAiProviderProfile { timeoutMs?: number /** WebSocket connection timeout in milliseconds. */ websocketConnectTimeoutMs?: number - /** Provider SDK retry count. */ - maxRetries?: number - /** Maximum provider-requested retry delay in milliseconds. */ - maxRetryDelayMs?: number + /** Maximum provider idle time while one stream read is outstanding. */ + streamIdleTimeoutMs?: number +} + +/** Validated profile with every adapter-owned default resolved. */ +export interface ResolvedPiAiProviderProfile extends PiAiProviderProfile { + /** Positive finite provider-idle interval after defaulting. */ + streamIdleTimeoutMs: number } /** Plugin configuration: the non-empty provider profiles this instance owns. */ @@ -60,8 +68,7 @@ const profile = z.object({ transport: z.union(['sse', 'websocket', 'websocket-cached', 'auto']), timeoutMs: z.natural(), websocketConnectTimeoutMs: z.natural(), - maxRetries: z.natural(), - maxRetryDelayMs: z.natural(), + streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS), }) /** Runtime schema for {@link Config}. */ @@ -75,11 +82,18 @@ export const Config: z = z.object({ * @param profiles - configured provider profiles. * @returns validated profiles in configuration order. */ -export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): PiAiProviderProfile[] { +export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): ResolvedPiAiProviderProfile[] { if (profiles.length === 0) throw new Error('llm-pi-ai: providers must contain at least one profile') const supported = new Set(getProviders()) const seen = new Set() return profiles.map((source) => { + const legacy = source as PiAiProviderProfile & { + maxRetries?: unknown + maxRetryDelayMs?: unknown + } + if ('maxRetries' in legacy || 'maxRetryDelayMs' in legacy) { + throw new Error('llm-pi-ai: maxRetries and maxRetryDelayMs were removed; compose agent recovery with dsh-llm-retry') + } if (source.provider.length === 0) throw new Error('llm-pi-ai: provider names must be non-empty') if (!supported.has(source.provider)) throw new Error(`llm-pi-ai: unknown pi-ai provider "${source.provider}"`) if (seen.has(source.provider)) throw new Error(`llm-pi-ai: duplicate provider profile "${source.provider}"`) @@ -89,9 +103,18 @@ export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): PiAiP if (source.baseURL !== undefined && source.baseURL.length === 0) { throw new Error(`llm-pi-ai: provider "${source.provider}" has an empty baseURL`) } + const streamIdleTimeoutMs = source.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS + if (!Number.isFinite(streamIdleTimeoutMs) + || streamIdleTimeoutMs <= 0 + || streamIdleTimeoutMs > MAX_TIMER_DELAY_MS) { + throw new Error( + `llm-pi-ai: provider "${source.provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`, + ) + } seen.add(source.provider) return { ...source, + streamIdleTimeoutMs, ...source.headers === undefined ? {} : { headers: { ...source.headers } }, ...source.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...source.thinkingBudgets } }, } diff --git a/packages/llm/llm-pi-ai/src/stream.ts b/packages/llm/llm-pi-ai/src/stream.ts index c1a85addf0..2c89d1e224 100644 --- a/packages/llm/llm-pi-ai/src/stream.ts +++ b/packages/llm/llm-pi-ai/src/stream.ts @@ -8,7 +8,7 @@ * @module dsh-llm-pi-ai/stream */ -import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, LlmError } from '@deepseek-ai/dsh-llm' +import { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmError, QUOTA_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm' import type { FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm' import { isContextOverflow } from '@earendil-works/pi-ai' import type { AssistantMessage, AssistantMessageEvent, Usage as PiUsage } from '@earendil-works/pi-ai' @@ -30,9 +30,12 @@ export function mapUsage(usage: PiUsage): TokenUsage { function classifyPiAiError(message: string): string { if (/\b(?:401|403)\b/.test(message)) return 'AUTH' + if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT' if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST' if (/\b5\d\d\b/.test(message)) return 'SERVER' + if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT' + if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message)) return 'TRANSPORT' return 'PI_AI_ERROR' } @@ -52,8 +55,10 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number) if (piAiOverflow || harnessOverflow) { return { kind: 'error', - message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`, - code: CONTEXT_WINDOW_EXCEEDED_CODE, + failure: { + message: message.errorMessage ?? `pi-ai detected context overflow for model "${message.model}"`, + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }, } } @@ -61,10 +66,13 @@ export function mapStopReason(message: AssistantMessage, contextWindow?: number) case 'stop': return { kind: 'stop' } case 'length': return { kind: 'max-tokens' } case 'toolUse': return { kind: 'tool-calls' } - case 'aborted': return { kind: 'aborted' } + case 'aborted': return { + kind: 'aborted', + failure: { message: message.errorMessage ?? 'pi-ai stream aborted', code: 'ABORTED' }, + } case 'error': { const text = message.errorMessage ?? 'pi-ai stream error' - return { kind: 'error', message: text, code: classifyPiAiError(text) } + return { kind: 'error', failure: { message: text, code: classifyPiAiError(text) } } } } } diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 51f78e7760..1b7445fa56 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -6,6 +6,7 @@ import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, userAgent } from '@ import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai' import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai' import { getModels } from '@earendil-works/pi-ai' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { resolveProfiles } from '../src/config.ts' import { assemble } from './assemble.ts' @@ -14,6 +15,8 @@ interface MockServer { paths: string[] requests: unknown[] headers: IncomingMessage['headers'][] + readonly closedResponses: number + responseClosed: Promise } const servers: Server[] = [] @@ -23,11 +26,23 @@ afterEach(async () => { await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve)))) }) -async function mockServer(script: { status?: number; events?: string[]; body?: string; delayMs?: number }[]): Promise { +async function mockServer(script: { + status?: number + events?: string[] + body?: string + delayMs?: number + headers?: Record +}[]): Promise { const paths: string[] = [] const requests: unknown[] = [] const headers: IncomingMessage['headers'][] = [] + let closedResponses = 0 + const responseClosed = Promise.withResolvers() const server = createServer((request: IncomingMessage, response: ServerResponse) => { + response.on('close', () => { + closedResponses += 1 + responseClosed.resolve(undefined) + }) let body = '' request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') }) request.on('end', () => { @@ -36,7 +51,7 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s headers.push(request.headers) const behavior = script.shift() ?? { status: 500, body: 'script exhausted' } if (behavior.status !== undefined && behavior.status !== 200) { - response.writeHead(behavior.status, { 'content-type': 'application/json' }) + response.writeHead(behavior.status, { 'content-type': 'application/json', ...behavior.headers }) response.end(behavior.body ?? '{}') return } @@ -56,7 +71,14 @@ async function mockServer(script: { status?: number; events?: string[]; body?: s await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)) const address = server.address() if (address === null || typeof address === 'string') throw new Error('no port') - return { url: `http://127.0.0.1:${address.port}`, paths, requests, headers } + return { + url: `http://127.0.0.1:${address.port}`, + paths, + requests, + headers, + responseClosed: responseClosed.promise, + get closedResponses() { return closedResponses }, + } } const textEvents = [ @@ -107,8 +129,7 @@ describe('PiAiAdapter provider routing', () => { transport: 'sse', timeoutMs: 5000, websocketConnectTimeoutMs: 3000, - maxRetries: 0, - maxRetryDelayMs: 10, + streamIdleTimeoutMs: 10_000, thinkingBudgets: { high: 2048 }, }) await assemble(ctx, { @@ -161,13 +182,35 @@ describe('PiAiAdapter provider routing', () => { const ctx = new Context() await ctx.plugin(LlmService) await ctx.plugin(LlmPiAi, { - providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1`, maxRetries: 0 }], + providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1` }], }) const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] }) expect(result.finish.kind).toBe('error') expect(server.paths).toEqual(['/v1/responses']) }) + it('forces one wire request for an SDK-retryable provider failure', async () => { + const server = await mockServer([ + { + status: 429, + headers: { 'retry-after-ms': '1' }, + body: JSON.stringify({ error: { message: 'retryable provider failure' } }), + }, + { status: 500, body: JSON.stringify({ error: { message: 'hidden SDK retry' } }) }, + { status: 500, body: JSON.stringify({ error: { message: 'second hidden SDK retry' } }) }, + ]) + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(LlmPiAi, { + providers: [{ provider: 'openai', apiKey: 'test-key', baseURL: `${server.url}/v1` }], + }) + + const result = await assemble(ctx, { provider: 'openai', model: 'gpt-4.1', messages: [] }) + + expect(result.finish).toMatchObject({ kind: 'error' }) + expect(server.paths).toEqual(['/v1/responses']) + }) + it('uses OpenAI Responses against an Azure project v1 path with its API key header', async () => { const server = await mockServer([{ status: 401, body: JSON.stringify({ error: { message: 'expected mock failure' } }) }]) const ctx = new Context() @@ -178,7 +221,6 @@ describe('PiAiAdapter provider routing', () => { apiKey: 'test-key', baseURL: `${server.url}/api/projects/openai/openai/v1`, headers: { 'api-key': 'test-key', Authorization: '' }, - maxRetries: 0, }], }) const result = await assemble(ctx, { provider: 'openai', model: 'gpt-5.5', messages: [] }) @@ -195,9 +237,10 @@ describe('PiAiAdapter provider routing', () => { [500, 'SERVER'], ] as const)('maps HTTP %s failures to %s', async (status, code) => { const server = await mockServer([{ status, body: JSON.stringify({ error: { message: `provider ${status}` } }) }]) - const ctx = await harness(server.url, { maxRetries: 0 }) + const ctx = await harness(server.url) const result = await assemble(ctx, { model: 'deepseek-v4-flash', messages: [] }) - expect(result.finish).toMatchObject({ kind: 'error', code }) + expect(result.finish).toMatchObject({ kind: 'error', failure: { code } }) + expect(server.paths).toEqual(['/chat/completions']) }) it('uses the resolved catalog context window for usage-based overflow detection', async () => { @@ -218,10 +261,29 @@ describe('PiAiAdapter provider routing', () => { expect(result.finish).toEqual({ kind: 'error', - message: `pi-ai detected context overflow for model "${model.id}"`, - code: CONTEXT_WINDOW_EXCEEDED_CODE, + failure: { + message: `pi-ai detected context overflow for model "${model.id}"`, + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }, }) }) + + it('stops the SDK request when the adapter idle watchdog expires', async () => { + const server = await mockServer([{ events: textEvents, delayMs: 200 }]) + const ctx = await harness(server.url, { streamIdleTimeoutMs: 20 }) + + await expect(assemble(ctx, { model: 'deepseek-v4-flash', messages: [] })) + .rejects.toMatchObject({ code: 'TIMEOUT' }) + await Promise.race([ + server.responseClosed, + new Promise((_resolve, reject) => { + setTimeout(() => { reject(new Error('SDK request did not close after idle timeout')) }, 100) + }), + ]) + + expect(server.paths).toEqual(['/chat/completions']) + expect(server.closedResponses).toBe(1) + }) }) describe('provider profile lifecycle', () => { @@ -280,16 +342,31 @@ describe('provider profile lifecycle', () => { expect(() => resolveProfiles([{ provider: 'openai', baseURL: '' }])).toThrow(/empty baseURL/) }) - it('rejects negative or fractional stream tunables at schema validation', () => { + it.each(['maxRetries', 'maxRetryDelayMs'] as const)( + 'rejects removed profile field %s instead of silently restoring hidden SDK retries', + async (field) => { + const legacy = { provider: 'openai', [field]: 2 } + expect(() => resolveProfiles([legacy as never])).toThrow(/removed.*agent recovery/i) + const ctx = new Context() + await ctx.plugin(LlmService) + await expect(ctx.plugin(LlmPiAi, { providers: [legacy as never] })) + .rejects.toThrow(/removed.*agent recovery/i) + }, + ) + + it('rejects invalid stream tunables at plugin load', async () => { const invalid = [ { timeoutMs: -1 }, { websocketConnectTimeoutMs: -1 }, - { maxRetries: -1 }, - { maxRetries: 0.5 }, - { maxRetryDelayMs: -1 }, + { streamIdleTimeoutMs: 0 }, + { streamIdleTimeoutMs: Number.NaN }, + { streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 }, ] for (const entry of invalid) { - expect(() => new LlmPiAi.Config({ providers: [{ provider: 'openai', ...entry }] })).toThrow() + const ctx = new Context() + await ctx.plugin(LlmService) + await expect(ctx.plugin(LlmPiAi, { providers: [{ provider: 'openai', ...entry }] })) + .rejects.toThrow() } }) @@ -301,11 +378,59 @@ describe('provider profile lifecycle', () => { })()).rejects.toMatchObject({ code: 'NO_ADAPTER' }) expect(new LlmError('x', 'X')).toBeInstanceOf(Error) }) + + it('validates direct-constructor profiles at the embedding boundary', () => { + expect(() => new PiAiAdapter({ + profiles: [{ provider: 'openai', streamIdleTimeoutMs: 0 }], + })).toThrow(/streamIdleTimeoutMs.*positive finite/) + expect(() => new PiAiAdapter({ + profiles: [{ provider: 'openai', streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 }], + })).toThrow(/streamIdleTimeoutMs.*no greater/) + }) }) describe('abort wiring', () => { + it('preserves an unknown pre-dispatch adapter Error exactly', async () => { + const original = new Error('SDK context conversion exploded') + const message = Object.defineProperty({}, 'role', { + get() { throw original }, + }) + const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] }) + const drain = async (): Promise => { + for await (const _chunk of adapter.stream({ + provider: 'deepseek', + model: 'deepseek-v4-flash', + messages: [message as never], + })) { /* drain */ } + } + + await expect(drain()).rejects.toBe(original) + }) + + it('lets a concurrent caller abort classify a pre-dispatch adapter failure', async () => { + const controller = new AbortController() + const original = new Error('conversion lost its caller') + const message = Object.defineProperty({}, 'role', { + get() { + controller.abort('caller cancelled during conversion') + throw original + }, + }) + const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] }) + const drain = async (): Promise => { + for await (const _chunk of adapter.stream({ + provider: 'deepseek', + model: 'deepseek-v4-flash', + messages: [message as never], + signal: controller.signal, + })) { /* drain */ } + } + + await expect(drain()).rejects.toMatchObject({ code: 'ABORTED', cause: original }) + }) + it('resolves catalog endpoints without an override before honoring pre-abort', async () => { - const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key', maxRetries: 0 }] }) + const adapter = new PiAiAdapter({ profiles: [{ provider: 'deepseek', apiKey: 'test-key' }] }) const controller = new AbortController() controller.abort('already stopped') const chunks = [] diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index a2f37ce511..e33f0bf09a 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -485,20 +485,32 @@ describe('toStreamChunks', () => { ))) expect(chunks).toEqual([ { type: 'usage', usage: { inputTokens: 1, outputTokens: 0 } }, - { type: 'finish', reason: { kind: 'error', message: 'boom', code: 'PI_AI_ERROR' } }, + { type: 'finish', reason: { kind: 'error', failure: { message: 'boom', code: 'PI_AI_ERROR' } } }, ]) }) it('maps aborted error events to aborted finish', async () => { const error = assistant({ stopReason: 'aborted' }) const chunks = await collect(toStreamChunks(feed({ type: 'error', reason: 'aborted', error }))) - expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'aborted' } }) + expect(chunks.at(-1)).toEqual({ + type: 'finish', + reason: { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } }, + }) }) it('rejects a stream that ends without done or error', async () => { await expect(collect(toStreamChunks(feed({ type: 'start', partial: assistant() })))) .rejects.toThrow(/without done\/error/) }) + + it('preserves an unknown SDK iterator Error exactly', async () => { + const original = Object.assign(new Error('SDK transport exploded'), { code: 'ECONNRESET' }) + async function* failedSdkStream(): AsyncGenerator { + throw original + } + + await expect(collect(toStreamChunks(failedSdkStream()))).rejects.toBe(original) + }) }) describe('mapStopReason / mapUsage', () => { @@ -506,46 +518,52 @@ describe('mapStopReason / mapUsage', () => { ['stop', { kind: 'stop' }], ['length', { kind: 'max-tokens' }], ['toolUse', { kind: 'tool-calls' }], - ['aborted', { kind: 'aborted' }], + ['aborted', { kind: 'aborted', failure: { message: 'pi-ai stream aborted', code: 'ABORTED' } }], ] as const)('maps %s', (stopReason, expected) => { expect(mapStopReason(assistant({ stopReason }))).toEqual(expected) }) it('defaults the error message when pi-ai omits it', () => { expect(mapStopReason(assistant({ stopReason: 'error' }))) - .toEqual({ kind: 'error', message: 'pi-ai stream error', code: 'PI_AI_ERROR' }) + .toEqual({ kind: 'error', failure: { message: 'pi-ai stream error', code: 'PI_AI_ERROR' } }) }) it('maps routable HTTP-ish error messages to stable codes', () => { expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 401: bad key' }))) - .toMatchObject({ kind: 'error', code: 'AUTH' }) + .toMatchObject({ kind: 'error', failure: { code: 'AUTH' } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: rate limit' }))) - .toMatchObject({ kind: 'error', code: 'RATE_LIMIT' }) + .toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } }) + expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: insufficient_quota' }))) + .toMatchObject({ kind: 'error', failure: { code: 'QUOTA' } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 500: backend down' }))) - .toMatchObject({ kind: 'error', code: 'SERVER' }) + .toMatchObject({ kind: 'error', failure: { code: 'SERVER' } }) + expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'provider timed out' }))) + .toMatchObject({ kind: 'error', failure: { code: 'TIMEOUT' } }) + expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'ECONNRESET socket closed' }))) + .toMatchObject({ kind: 'error', failure: { code: 'TRANSPORT' } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 400: input exceeds the model context window limit', - }))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE }) + }))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 400: request too large for model context', - }))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE }) + }))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value', - }))).toMatchObject({ kind: 'error', code: 'INVALID_REQUEST' }) + }))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } }) }) it('uses pi-ai provider-specific overflow classification without losing rate-limit exclusions', () => { expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'prompt is too long: 213462 tokens > 200000 maximum', - }))).toMatchObject({ kind: 'error', code: CONTEXT_WINDOW_EXCEEDED_CODE }) + }))).toMatchObject({ kind: 'error', failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'ThrottlingException: Too many tokens, rate limit reached', - }))).toMatchObject({ kind: 'error', code: 'RATE_LIMIT' }) + }))).toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } }) }) it('uses the resolved context window for silent and length-stop overflows', () => { @@ -553,15 +571,17 @@ describe('mapStopReason / mapUsage', () => { expect(mapStopReason(silent)).toEqual({ kind: 'stop' }) expect(mapStopReason(silent, 100)).toEqual({ kind: 'error', - message: 'pi-ai detected context overflow for model "deepseek-v4-flash"', - code: CONTEXT_WINDOW_EXCEEDED_CODE, + failure: { + message: 'pi-ai detected context overflow for model "deepseek-v4-flash"', + code: CONTEXT_WINDOW_EXCEEDED_CODE, + }, }) const truncated = assistant({ stopReason: 'length', usage: usage(80, 0, 19) }) expect(mapStopReason(truncated)).toEqual({ kind: 'max-tokens' }) expect(mapStopReason(truncated, 100)).toMatchObject({ kind: 'error', - code: CONTEXT_WINDOW_EXCEEDED_CODE, + failure: { code: CONTEXT_WINDOW_EXCEEDED_CODE }, }) }) diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index 107c12d264..06154a8a5e 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -70,7 +70,7 @@ function textOf(result: AssembledResult): string { function expectFinish(result: AssembledResult, expected: 'stop' | 'tool-calls'): void { if (result.finish.kind === 'error') { - throw new Error(`provider request failed (${result.finish.code ?? 'unknown'}): ${result.finish.message}`) + throw new Error(`provider request failed (${result.finish.failure.code}): ${result.finish.failure.message}`) } expect(result.finish.kind).toBe(expected) } diff --git a/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts b/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts new file mode 100644 index 0000000000..e85c44c110 --- /dev/null +++ b/packages/llm/llm-pi-ai/tests/sdk-options.spec.ts @@ -0,0 +1,35 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' + +const streamSimple = vi.hoisted(() => vi.fn()) + +vi.mock('@earendil-works/pi-ai', async (importOriginal) => { + const actual = await importOriginal() + return { ...actual, streamSimple } +}) + +import { PiAiAdapter } from '../src/adapter.ts' + +afterEach(() => { streamSimple.mockReset() }) + +describe('pi-ai SDK retry boundary', () => { + it('pins one SDK attempt even when the installed provider currently defaults to zero retries', async () => { + const failure = new Error('mock SDK boundary') + streamSimple.mockReturnValue({ + async * [Symbol.asyncIterator](): AsyncGenerator { + throw failure + }, + }) + const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai', apiKey: 'test-key' }] }) + const drain = async (): Promise => { + for await (const _chunk of adapter.stream({ + provider: 'openai', + model: 'gpt-4.1', + messages: [], + })) { /* drain */ } + } + + await expect(drain()).rejects.toBe(failure) + expect(streamSimple).toHaveBeenCalledOnce() + expect(streamSimple.mock.calls[0]?.[2]).toMatchObject({ maxRetries: 0 }) + }) +}) diff --git a/packages/llm/llm-pi-ai/tsconfig.json b/packages/llm/llm-pi-ai/tsconfig.json index e9de391ba1..5e427d88b9 100644 --- a/packages/llm/llm-pi-ai/tsconfig.json +++ b/packages/llm/llm-pi-ai/tsconfig.json @@ -19,6 +19,9 @@ }, { "path": "../../llm/llm" + }, + { + "path": "../../util/timeout" } ] } diff --git a/packages/llm/llm-retry/README.md b/packages/llm/llm-retry/README.md new file mode 100644 index 0000000000..baebc2d0b3 --- /dev/null +++ b/packages/llm/llm-retry/README.md @@ -0,0 +1,39 @@ +# `@deepseek-ai/dsh-llm-retry` + +Function plugin that retries selected transient model-request failures on the agent loop's closed-step recovery seam. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered step. + +The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. Delay bounds must fit Node's supported timer range. A valid provider `retryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead. + +Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward. + +```yaml +- name: '@deepseek-ai/dsh-llm-retry' + config: + maxTransientRetries: 2 + initialDelayMs: 500 + maxDelayMs: 10000 + jitterRatio: 0.1 + retryableCodes: [RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT] +``` + +## Model Experience + +### Transient request recovery + +#### What the model sees + +No retry event, delay, or failure prose is model-visible. After a retry, the next numbered step reconstructs the same explicit provider/model request from durable session history; failed chunks never enter derived messages. + +#### Token effect + +Each retry is a new provider request and may repeat input-token billing. The finite budget caps attempts; `llm/retry` itself contributes no tokens. + +#### KV Cache effect + +The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface status event does not change cache identity. + +## Known Limitations and Deferred Work + +- **Agent steps are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably. +- **Finite plugin budgets add** — this policy counts only configured transient codes; context-overflow compaction counts only its own code. A future policy with overlapping codes must document and test registration-order behavior. +- **`llm/retry` records scheduling, not completion** — later step and turn events establish success, exhaustion, or cancellation. diff --git a/packages/llm/llm-retry/package.json b/packages/llm/llm-retry/package.json new file mode 100644 index 0000000000..fce40dbe33 --- /dev/null +++ b/packages/llm/llm-retry/package.json @@ -0,0 +1,47 @@ +{ + "name": "@deepseek-ai/dsh-llm-retry", + "description": "Bounded transient LLM request retry policy for the DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts", + "lib/types/**/*.d.ts.map", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "@deepseek-ai/dsh-timeout": "^0.0.1", + "cordis": "^4.0.0-rc.7" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@cordisjs/plugin-include": "workspace:^", + "@cordisjs/plugin-loader": "workspace:^", + "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-agent-loop": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^", + "@deepseek-ai/dsh-session-persistence-sqlite": "workspace:^", + "@deepseek-ai/dsh-system-prompt": "workspace:^", + "@deepseek-ai/dsh-timeout": "workspace:^", + "@deepseek-ai/dsh-tools": "workspace:^", + "cordis": "^4.0.0-rc.7" + } +} diff --git a/packages/llm/llm-retry/src/index.ts b/packages/llm/llm-retry/src/index.ts new file mode 100644 index 0000000000..d4c5b47b3d --- /dev/null +++ b/packages/llm/llm-retry/src/index.ts @@ -0,0 +1,211 @@ +/** + * Bounded transient model-request retry policy on the agent loop's closed-step + * recovery seam. Each scheduled retry is durable before its cancellable wait. + * + * @module @deepseek-ai/dsh-llm-retry + */ + +import type { Context } from 'cordis' +import z from 'schemastery' +import type { Agent, RequestError, RequestErrorDecision } from '@deepseek-ai/dsh-agent' +import type { LlmFailure } from '@deepseek-ai/dsh-llm' +import type {} from '@deepseek-ai/dsh-session' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' + +declare module '@deepseek-ai/dsh-session' { + interface SessionEventMap { + /** Durable, non-surface record of one transient retry scheduled after a closed failed step. */ + 'llm/retry': { + turn: number + step: number + retry: number + maxRetries: number + delayMs: number + failure: LlmFailure + } + } +} + +export const name = 'llm-retry' +export const inject = ['agents'] + +const DEFAULT_MAX_TRANSIENT_RETRIES = 2 +const DEFAULT_INITIAL_DELAY_MS = 500 +const DEFAULT_MAX_DELAY_MS = 10_000 +const DEFAULT_JITTER_RATIO = 0.1 +const DEFAULT_RETRYABLE_CODES = Object.freeze(['RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT']) + +/** Deployment-owned limits and classification for transient request recovery. */ +export interface Config { + /** Maximum transient retries after the first request (default 2). */ + maxTransientRetries?: number + /** Initial local exponential-backoff delay in milliseconds (default 500). */ + initialDelayMs?: number + /** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */ + maxDelayMs?: number + /** Symmetric random multiplier range around one (default 0.1). */ + jitterRatio?: number + /** Stable failure codes eligible for this policy. */ + retryableCodes?: string[] +} + +/** Runtime schema for {@link Config}. */ +export const Config: z = z.object({ + maxTransientRetries: z.number().step(1).min(0).default(DEFAULT_MAX_TRANSIENT_RETRIES), + initialDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_INITIAL_DELAY_MS), + maxDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_MAX_DELAY_MS), + jitterRatio: z.number().min(0).max(1).default(DEFAULT_JITTER_RATIO), + retryableCodes: z.array(z.string()).default([...DEFAULT_RETRYABLE_CODES]), +}) + +interface ResolvedConfig { + readonly maxTransientRetries: number + readonly initialDelayMs: number + readonly maxDelayMs: number + readonly jitterRatio: number + readonly retryableCodes: ReadonlySet +} + +function resolveConfig(config: Config): ResolvedConfig { + const maxTransientRetries = config.maxTransientRetries ?? DEFAULT_MAX_TRANSIENT_RETRIES + const initialDelayMs = config.initialDelayMs ?? DEFAULT_INITIAL_DELAY_MS + const maxDelayMs = config.maxDelayMs ?? DEFAULT_MAX_DELAY_MS + const jitterRatio = config.jitterRatio ?? DEFAULT_JITTER_RATIO + const codes = config.retryableCodes ?? [...DEFAULT_RETRYABLE_CODES] + + if (!Number.isInteger(maxTransientRetries) || maxTransientRetries < 0) { + throw new Error('llm-retry: maxTransientRetries must be a non-negative integer') + } + if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0 || initialDelayMs > MAX_TIMER_DELAY_MS) { + throw new Error(`llm-retry: initialDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) + } + if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0 || maxDelayMs > MAX_TIMER_DELAY_MS) { + throw new Error(`llm-retry: maxDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) + } + if (initialDelayMs > maxDelayMs) { + throw new Error('llm-retry: initialDelayMs must be less than or equal to maxDelayMs') + } + if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) { + throw new Error('llm-retry: jitterRatio must be between 0 and 1') + } + if (codes.length === 0) { + throw new Error('llm-retry: retryableCodes must not be empty') + } + if (codes.some(code => code.length === 0)) { + throw new Error('llm-retry: retryableCodes must contain only non-empty strings') + } + if (new Set(codes).size !== codes.length) { + throw new Error('llm-retry: retryableCodes must not contain duplicates') + } + + return Object.freeze({ + maxTransientRetries, + initialDelayMs, + maxDelayMs, + jitterRatio, + retryableCodes: new Set(codes), + }) +} + +/** Non-serializable seams used to make timing policy deterministic in tests. */ +export interface RetryInternals { + /** Random sample in the inclusive zero-to-one range used for jitter. */ + random?: () => number +} + +function localDelay(config: ResolvedConfig, retry: number, random: () => number): number { + const exponent = Math.min(retry - 1, 1024) + const exponential = Math.min(config.initialDelayMs * 2 ** exponent, config.maxDelayMs) + const jitter = 1 - config.jitterRatio + 2 * config.jitterRatio * random() + return Math.min(exponential * jitter, config.maxDelayMs) +} + +function cancellableDelay(delayMs: number, signal: AbortSignal): Promise { + if (signal.aborted) return Promise.resolve(false) + return new Promise((resolve) => { + const timer = setTimeout(() => { + signal.removeEventListener('abort', onAbort) + resolve(true) + }, delayMs) + function onAbort(): void { + clearTimeout(timer) + resolve(false) + } + signal.addEventListener('abort', onAbort, { once: true }) + }) +} + +/** + * Install bounded transient request recovery. + * @param ctx - plugin context that owns the listener and active waits. + * @param config - retry budget, delay bounds, jitter, and eligible codes. + * @param internals - non-serializable deterministic seams for tests. + */ +export function apply(ctx: Context, config: Config = {}, internals: RetryInternals = {}): void { + const resolved = resolveConfig(config) + const random = internals.random ?? Math.random + const lifetime = new AbortController() + const active = new Set>() + + async function backoff( + agent: Agent, + turn: number, + step: number, + failure: LlmFailure, + retry: number, + delayMs: number, + signal: AbortSignal, + ): Promise { + const fusedSignal = AbortSignal.any([signal, lifetime.signal]) + if (fusedSignal.aborted) return { action: 'fail' } + agent.session.append('llm/retry', { + turn, + step, + retry, + maxRetries: resolved.maxTransientRetries, + delayMs, + failure, + }) + if (!await cancellableDelay(delayMs, fusedSignal)) return { action: 'fail' } + return { action: 'retry' } + } + + const disposeListener = ctx.on('agent/request-error', ( + agent: Agent, + turn: number, + step: number, + _error: RequestError, + failure: LlmFailure, + priorFailures: readonly LlmFailure[], + signal: AbortSignal, + next: () => Promise, + ) => { + // A waterfall may have captured this callback before its registration was + // removed. Lifetime cancellation must prevent that stale callback from + // entering a downstream policy after disposal. + if (lifetime.signal.aborted) return Promise.resolve({ action: 'fail' }) + if (!resolved.retryableCodes.has(failure.code)) return next() + const priorTransientFailures = priorFailures.filter(item => resolved.retryableCodes.has(item.code)).length + if (priorTransientFailures >= resolved.maxTransientRetries) return next() + + const retry = priorTransientFailures + 1 + let delayMs: number + if (failure.retryAfterMs !== undefined && Number.isFinite(failure.retryAfterMs) && failure.retryAfterMs > 0) { + if (failure.retryAfterMs > resolved.maxDelayMs) return next() + delayMs = failure.retryAfterMs + } else { + delayMs = localDelay(resolved, retry, random) + } + + const tracked = backoff(agent, turn, step, failure, retry, delayMs, signal) + .finally(() => active.delete(tracked)) + active.add(tracked) + return tracked + }) + + ctx.effect(() => async () => { + disposeListener() + lifetime.abort(new Error('llm-retry plugin disposed')) + await Promise.allSettled([...active]) + }, 'llm-retry: abort and drain backoffs') +} diff --git a/packages/llm/llm-retry/tests/loader-composition.spec.ts b/packages/llm/llm-retry/tests/loader-composition.spec.ts new file mode 100644 index 0000000000..c00175b20d --- /dev/null +++ b/packages/llm/llm-retry/tests/loader-composition.spec.ts @@ -0,0 +1,124 @@ +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import Loader from '@cordisjs/plugin-loader' +import Include from '@cordisjs/plugin-include' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import AgentLoop from '@deepseek-ai/dsh-agent-loop' +import LlmService, { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import * as retry from '../src/index.ts' + +let root: string | undefined +let context: Context | undefined + +class TransientOnceAdapter extends LlmAdapter { + requests = 0 + + async * stream(_options: GenerateOptions): AsyncIterable { + this.requests += 1 + if (this.requests === 1) throw new LlmError('temporary outage', 'SERVER') + yield { type: 'block-start', index: 0, blockType: 'text' } + yield { type: 'text-delta', index: 0, text: 'recovered' } + yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } } + yield { type: 'finish', reason: { kind: 'stop' } } + } +} + +function waitForIdle(ctx: Context, agent: Agent): Promise { + return new Promise((resolve) => { + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'idle') { + dispose() + resolve() + } + }) + }) +} + +afterEach(async () => { + await context?.fiber.dispose() + context = undefined + if (root !== undefined) await rm(root, { recursive: true, force: true }) + root = undefined +}) + +async function loadYaml(lines: readonly string[]): Promise { + root = await mkdtemp(join(tmpdir(), 'dsh-llm-retry-loader-')) + const configPath = join(root, 'cordis.yml') + await writeFile(configPath, [...lines, ''].join('\n')) + + context = new Context() + context.baseUrl = pathToFileURL(root).href + '/' + await context.plugin(Loader) + context.loader.builtins.include = Include + const modules = new Map([ + ['@deepseek-ai/dsh-llm', LlmService], + ['@deepseek-ai/dsh-session', SessionStore], + ['@deepseek-ai/dsh-system-prompt', SystemPrompt], + ['@deepseek-ai/dsh-tools', ToolRegistry], + ['@deepseek-ai/dsh-agent', AgentRegistry], + ['@deepseek-ai/dsh-llm-retry', retry], + ['@deepseek-ai/dsh-agent-loop', AgentLoop], + ]) + context.loader.internal = { + version: 'v2', + async import(specifier: string) { + if (!modules.has(specifier)) throw new Error(`unexpected Loader import: ${specifier}`) + return modules.get(specifier) + }, + } as unknown as NonNullable + await context.loader.create({ + name: 'cordis:include', + config: { path: pathToFileURL(configPath).href }, + }) + await context.loader.await() + return context +} + +describe('real Loader composition', () => { + it('loads the flat policy and records recovery through the shipping loop', async () => { + const loaded = await loadYaml([ + "- name: '@deepseek-ai/dsh-llm'", + "- name: '@deepseek-ai/dsh-session'", + "- name: '@deepseek-ai/dsh-system-prompt'", + "- name: '@deepseek-ai/dsh-tools'", + "- name: '@deepseek-ai/dsh-agent'", + "- name: '@deepseek-ai/dsh-llm-retry'", + ' config:', + ' maxTransientRetries: 1', + ' initialDelayMs: 1', + ' maxDelayMs: 1', + ' jitterRatio: 0', + ' retryableCodes: [RATE_LIMIT, SERVER]', + "- name: '@deepseek-ai/dsh-agent-loop'", + ]) + + const unloaded = [...loaded.loader.entries()] + .filter(entry => entry.fiber === undefined && !entry.disabled) + .map(entry => entry.options.name) + expect(unloaded).toEqual([]) + expect(loaded.agents).toBeInstanceOf(AgentRegistry) + + const adapter = new TransientOnceAdapter() + loaded.llm.registerAdapter(['mock'], adapter) + const agent = loaded.agentLoop.create(SessionId('loader-retry'), { provider: 'mock', model: 'mock' }) + const idle = waitForIdle(loaded, agent) + agent.send([{ type: 'text', text: 'recover' }]) + await idle + + expect(adapter.requests).toBe(2) + expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1) + expect(agent.session.deriveMessages().at(-1)).toMatchObject({ + role: 'assistant', + content: [{ type: 'text', text: 'recovered' }], + }) + }) +}) diff --git a/packages/llm/llm-retry/tests/persistence.spec.ts b/packages/llm/llm-retry/tests/persistence.spec.ts new file mode 100644 index 0000000000..3e5a9c8dcd --- /dev/null +++ b/packages/llm/llm-retry/tests/persistence.spec.ts @@ -0,0 +1,57 @@ +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' +import SessionPersistenceSqlite from '@deepseek-ai/dsh-session-persistence-sqlite' +import type {} from '../src/index.ts' + +const dirs: string[] = [] + +afterEach(async () => { + for (const dir of dirs.splice(0)) await rm(dir, { recursive: true, force: true }) +}) + +async function backend(kind: 'jsonl' | 'sqlite'): Promise { + const ctx = new Context() + await ctx.plugin(SessionStore) + if (kind === 'jsonl') { + const root = await mkdtemp(join(tmpdir(), 'dsh-llm-retry-jsonl-')) + dirs.push(root) + await ctx.plugin(SessionPersistenceJsonl, { root }) + } else { + await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) + } + return ctx +} + +describe.each(['jsonl', 'sqlite'] as const)('%s retry-event persistence', (kind) => { + it('round-trips the event losslessly without adding a model message', async () => { + const ctx = await backend(kind) + try { + const session = ctx.sessions.create(SessionId(`retry-${kind}`)) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) + session.append('step/start', { turn: 1, step: 1 }) + session.append('step/end', { turn: 1, step: 1 }) + const event = session.append('llm/retry', { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 750, + failure: { message: 'provider busy', code: 'RATE_LIMIT', status: 429 }, + }) + session.append('turn/end', { turn: 1, reason: { kind: 'aborted', reason: 'cancelled in backoff' } }) + + expect(session.deriveMessages()).toEqual([]) + await ctx.sessions.flush(session) + const loaded = await ctx.sessionPersistence.load(session.id) + + expect(loaded.events.find(item => item.type === 'llm/retry')).toEqual(event) + } finally { + await ctx.fiber.dispose() + } + }) +}) diff --git a/packages/llm/llm-retry/tests/retry.spec.ts b/packages/llm/llm-retry/tests/retry.spec.ts new file mode 100644 index 0000000000..bc10e28922 --- /dev/null +++ b/packages/llm/llm-retry/tests/retry.spec.ts @@ -0,0 +1,453 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import type { Fiber } from 'cordis' +import LlmService, { CallId, LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' +import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' +import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import type { SessionEvent } from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import type { Agent, RequestErrorDecision } from '@deepseek-ai/dsh-agent' +import AgentLoop from '@deepseek-ai/dsh-agent-loop' +import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' +import * as retry from '../src/index.ts' + +type ScriptEntry = Error | Iterable | AsyncIterable + +class ScriptedAdapter extends LlmAdapter { + readonly requests: GenerateOptions[] = [] + + constructor(private readonly entries: ScriptEntry[]) { + super() + } + + async * stream(options: GenerateOptions): AsyncIterable { + this.requests.push(options) + const entry = this.entries.shift() + if (entry === undefined) throw new Error('retry test script exhausted') + if (entry instanceof Error) throw entry + yield* entry + } +} + +async function* partialToolFailure(error: Error): AsyncGenerator { + const id = CallId('discarded-call') + yield { type: 'block-start', index: 0, blockType: 'text' } + yield { type: 'text-delta', index: 0, text: 'discarded partial output' } + yield { type: 'block-end', index: 0, block: { type: 'text', text: 'discarded partial output' } } + yield { type: 'block-start', index: 1, blockType: 'tool-call' } + yield { type: 'tool-call-delta', index: 1, id, name: 'danger', argumentsDelta: '{}' } + yield { type: 'block-end', index: 1, block: { type: 'tool-call', id, name: 'danger', arguments: '{}' } } + throw error +} + +function textResponse(text: string): StreamChunk[] { + return [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'text-delta', index: 0, text }, + { type: 'block-end', index: 0, block: { type: 'text', text } }, + { type: 'finish', reason: { kind: 'stop' } }, + ] +} + +async function harness( + adapter: LlmAdapter, + config: retry.Config = {}, + beforeRetry?: (ctx: Context) => void, + internals: retry.RetryInternals = {}, +): Promise<{ ctx: Context; retryFiber: Fiber }> { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + beforeRetry?.(ctx) + const resolvedConfig = Object.assign({ + maxTransientRetries: 2, + initialDelayMs: 500, + maxDelayMs: 10_000, + jitterRatio: 0, + }, config) + const retryFiber = await ctx.plugin(Object.assign((inner: Context) => { + retry.apply(inner, resolvedConfig, internals) + }, { inject: retry.inject })) + await ctx.plugin(AgentLoop, { agents: [] }) + ctx.llm.registerAdapter(['mock'], adapter) + return { ctx, retryFiber } +} + +function waitForIdle(ctx: Context, agent: Agent): Promise { + return new Promise((resolve) => { + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'idle') { + dispose() + resolve() + } + }) + }) +} + +function waitForRetry(ctx: Context, agent: Agent, retryNumber: number): Promise> { + return new Promise((resolve) => { + const dispose = ctx.on('session/event', (session, event) => { + if (session === agent.session && event.type === 'llm/retry' && event.data.retry === retryNumber) { + dispose() + resolve(event) + } + }) + }) +} + +let context: Context | undefined + +afterEach(async () => { + vi.useRealTimers() + await context?.fiber.dispose() + context = undefined +}) + +describe('bounded transient retry policy', () => { + it('records the scheduled delay before opening a fresh request attempt', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([ + new LlmError('busy', 'RATE_LIMIT', { status: 429 }), + textResponse('done'), + ]) + ;({ ctx: context } = await harness(adapter)) + const agent = context.agentLoop.create(SessionId('retry-success'), { + provider: 'mock', + model: 'mock', + }) + const scheduled = new Promise>((resolve) => { + const dispose = context?.on('session/event', (session, event) => { + if (session === agent.session && event.type === 'llm/retry') { + dispose?.() + resolve(event) + } + }) + }) + + agent.send([{ type: 'text', text: 'go' }]) + const event = await scheduled + + expect(event.data).toEqual({ + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 500, + failure: { message: 'busy', code: 'RATE_LIMIT', status: 429 }, + }) + expect(adapter.requests).toHaveLength(1) + await vi.advanceTimersByTimeAsync(499) + expect(adapter.requests).toHaveLength(1) + + const idle = waitForIdle(context, agent) + await vi.advanceTimersByTimeAsync(1) + await idle + + expect(adapter.requests).toHaveLength(2) + expect(agent.session.events.filter(item => item.type === 'step/start').map(item => item.data.step)) + .toEqual([1, 2]) + expect(agent.session.deriveMessages().at(-1)).toEqual({ + role: 'assistant', + content: [{ type: 'text', text: 'done' }], + provenance: { provider: 'mock', model: 'mock' }, + }) + }) + + it('leaves partial failed chunks on their step without committing a message or tool side effect', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([ + partialToolFailure(new LlmError('stream interrupted', 'TRANSPORT')), + textResponse('recovered'), + ]) + ;({ ctx: context } = await harness(adapter)) + let toolExecutions = 0 + context.tools.register(defineTool({ + name: 'danger', + description: 'must not run for a failed provider attempt', + parameters: {}, + async execute() { + toolExecutions += 1 + return [{ type: 'text', text: 'unexpected' }] + }, + })) + const agent = context.agentLoop.create(SessionId('retry-partial'), { provider: 'mock', model: 'mock' }) + const scheduled = waitForRetry(context, agent, 1) + + agent.send([{ type: 'text', text: 'go' }]) + await scheduled + const idle = waitForIdle(context, agent) + await vi.advanceTimersByTimeAsync(500) + await idle + + const failedChunks = agent.session.events.filter(event => + event.type === 'assistant/chunk' && event.data.step === 1, + ) + expect(failedChunks).toHaveLength(6) + expect(agent.session.events.filter(event => event.type === 'assistant/message').map(event => event.data.step)) + .toEqual([2]) + expect(agent.session.events.some(event => event.type === 'tool/call')).toBe(false) + expect(toolExecutions).toBe(0) + expect(agent.session.deriveMessages().at(-1)).toMatchObject({ + role: 'assistant', + content: [{ type: 'text', text: 'recovered' }], + provenance: { provider: 'mock', model: 'mock' }, + }) + }) + + it('applies bounded exponential jitter and stops after the configured budget', async () => { + vi.useFakeTimers() + const samples = [0, 1] + const adapter = new ScriptedAdapter([ + new LlmError('busy one', 'SERVER'), + new LlmError('busy two', 'SERVER'), + new LlmError('busy three', 'SERVER'), + ]) + ;({ ctx: context } = await harness(adapter, { jitterRatio: 0.1 }, undefined, { + random: () => samples.shift() ?? 0.5, + })) + const agent = context.agentLoop.create(SessionId('retry-exhausted'), { provider: 'mock', model: 'mock' }) + const first = waitForRetry(context, agent, 1) + + agent.send([{ type: 'text', text: 'go' }]) + expect((await first).data.delayMs).toBe(450) + + const second = waitForRetry(context, agent, 2) + await vi.advanceTimersByTimeAsync(450) + expect((await second).data.delayMs).toBe(1_100) + + const idle = waitForIdle(context, agent) + await vi.advanceTimersByTimeAsync(1_100) + await idle + + expect(adapter.requests).toHaveLength(3) + expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(2) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'error', failure: { message: 'busy three', code: 'SERVER' } } }, + }) + }) + + it('uses a bounded provider Retry-After verbatim and delegates an over-cap instruction', async () => { + vi.useFakeTimers() + const accepted = new ScriptedAdapter([ + new LlmError('wait', 'RATE_LIMIT', { retryAfterMs: 2_000 }), + textResponse('done'), + ]) + ;({ ctx: context } = await harness(accepted, { jitterRatio: 1 })) + const acceptedAgent = context.agentLoop.create(SessionId('retry-after-accepted'), { provider: 'mock', model: 'mock' }) + const scheduled = waitForRetry(context, acceptedAgent, 1) + acceptedAgent.send([{ type: 'text', text: 'go' }]) + expect((await scheduled).data.delayMs).toBe(2_000) + const acceptedIdle = waitForIdle(context, acceptedAgent) + await vi.advanceTimersByTimeAsync(2_000) + await acceptedIdle + expect(accepted.requests).toHaveLength(2) + + await context.fiber.dispose() + const rejected = new ScriptedAdapter([ + new LlmError('wait too long', 'RATE_LIMIT', { retryAfterMs: 10_001 }), + ]) + ;({ ctx: context } = await harness(rejected)) + const rejectedAgent = context.agentLoop.create(SessionId('retry-after-rejected'), { provider: 'mock', model: 'mock' }) + const rejectedIdle = waitForIdle(context, rejectedAgent) + rejectedAgent.send([{ type: 'text', text: 'go' }]) + await rejectedIdle + expect(rejected.requests).toHaveLength(1) + expect(rejectedAgent.session.events.some(event => event.type === 'llm/retry')).toBe(false) + }) + + it('delegates non-transient failures without scheduling a timer', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')]) + ;({ ctx: context } = await harness(adapter)) + const agent = context.agentLoop.create(SessionId('retry-auth'), { provider: 'mock', model: 'mock' }) + const idle = waitForIdle(context, agent) + agent.send([{ type: 'text', text: 'go' }]) + await idle + expect(adapter.requests).toHaveLength(1) + expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false) + expect(vi.getTimerCount()).toBe(0) + }) + + it('aborts and drains a captured backoff before plugin disposal completes', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([ + new LlmError('temporary', 'TRANSPORT'), + textResponse('must not run'), + ]) + const mounted = await harness(adapter) + context = mounted.ctx + const agent = context.agentLoop.create(SessionId('retry-hmr'), { provider: 'mock', model: 'mock' }) + const scheduled = waitForRetry(context, agent, 1) + agent.send([{ type: 'text', text: 'go' }]) + await scheduled + const idle = waitForIdle(context, agent) + + await mounted.retryFiber.dispose() + await idle + await vi.advanceTimersByTimeAsync(60_000) + + expect(adapter.requests).toHaveLength(1) + expect(agent.session.events.filter(event => event.type === 'step/start')).toHaveLength(1) + expect(vi.getTimerCount()).toBe(0) + }) + + it('does not make plugin disposal wait for a delegated recovery policy', async () => { + const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')]) + const mounted = await harness(adapter) + context = mounted.ctx + const downstream = Promise.withResolvers() + const entered = Promise.withResolvers() + context.on('agent/request-error', () => { + entered.resolve(undefined) + return downstream.promise + }) + const agent = context.agentLoop.create(SessionId('retry-delegated-disposal'), { + provider: 'mock', + model: 'mock', + }) + const idle = waitForIdle(context, agent) + agent.send([{ type: 'text', text: 'go' }]) + await entered.promise + + const disposing = mounted.retryFiber.dispose() + let timer: ReturnType | undefined + const outcome = await Promise.race([ + disposing.then(() => 'disposed' as const), + new Promise<'blocked'>((resolve) => { timer = setTimeout(() => { resolve('blocked') }, 100) }), + ]) + if (timer !== undefined) clearTimeout(timer) + downstream.resolve({ action: 'fail' }) + await disposing + await idle + + expect(outcome).toBe('disposed') + expect(adapter.requests).toHaveLength(1) + }) + + it('fails a captured callback after disposal without entering downstream policy', async () => { + const adapter = new ScriptedAdapter([new LlmError('bad key', 'AUTH')]) + const captured = Promise.withResolvers() + let invokeCaptured: (() => Promise) | undefined + const mounted = await harness(adapter, {}, (ctx) => { + ctx.on('agent/request-error', (_agent, _turn, _step, _error, _failure, _history, _signal, next) => { + return new Promise((resolve) => { + invokeCaptured = async () => { resolve(await next()) } + captured.resolve(undefined) + }) + }) + }) + context = mounted.ctx + let downstreamCalls = 0 + context.on('agent/request-error', async (_agent, _turn, _step, _error, _failure, _history, _signal, next) => { + downstreamCalls += 1 + return next() + }) + const agent = context.agentLoop.create(SessionId('retry-captured-disposal'), { + provider: 'mock', + model: 'mock', + }) + const idle = waitForIdle(context, agent) + agent.send([{ type: 'text', text: 'go' }]) + await captured.promise + + await mounted.retryFiber.dispose() + if (invokeCaptured === undefined) throw new Error('request-error waterfall did not capture retry callback') + await invokeCaptured() + await idle + + expect(downstreamCalls).toBe(0) + expect(adapter.requests).toHaveLength(1) + }) + + it('lets turn cancellation win during backoff without opening another step', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([ + new LlmError('temporary', 'TIMEOUT'), + textResponse('must not run'), + ]) + ;({ ctx: context } = await harness(adapter)) + const agent = context.agentLoop.create(SessionId('retry-cancel'), { provider: 'mock', model: 'mock' }) + const scheduled = waitForRetry(context, agent, 1) + agent.send([{ type: 'text', text: 'go' }]) + await scheduled + const idle = waitForIdle(context, agent) + agent.cancel('user cancelled during retry') + await idle + + expect(adapter.requests).toHaveLength(1) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'aborted', reason: 'user cancelled during retry' } }, + }) + expect(vi.getTimerCount()).toBe(0) + }) + + it('lets an earlier recovery listener cancel before retry policy runs', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([ + new LlmError('temporary', 'SERVER'), + textResponse('must not run'), + ]) + ;({ ctx: context } = await harness(adapter, {}, (ctx) => { + ctx.on('agent/request-error', async (agent, _turn, _step, _error, _failure, _history, _signal, next) => { + agent.cancel('cancelled by earlier recovery policy') + return next() + }) + })) + const agent = context.agentLoop.create(SessionId('retry-pre-cancel'), { provider: 'mock', model: 'mock' }) + const idle = waitForIdle(context, agent) + + agent.send([{ type: 'text', text: 'go' }]) + await idle + + expect(adapter.requests).toHaveLength(1) + expect(agent.session.events.some(event => event.type === 'llm/retry')).toBe(false) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'aborted', reason: 'cancelled by earlier recovery policy' } }, + }) + }) + + it('handles synchronous cancellation from the retry status event', async () => { + vi.useFakeTimers() + const adapter = new ScriptedAdapter([ + new LlmError('temporary', 'SERVER'), + textResponse('must not run'), + ]) + ;({ ctx: context } = await harness(adapter)) + const agent = context.agentLoop.create(SessionId('retry-event-cancel'), { provider: 'mock', model: 'mock' }) + context.on('session/event', (session, event) => { + if (session === agent.session && event.type === 'llm/retry') agent.cancel('cancelled by retry observer') + }) + const idle = waitForIdle(context, agent) + + agent.send([{ type: 'text', text: 'go' }]) + await idle + + expect(adapter.requests).toHaveLength(1) + expect(agent.session.events.filter(event => event.type === 'llm/retry')).toHaveLength(1) + expect(vi.getTimerCount()).toBe(0) + }) + + it.each([ + [{ maxTransientRetries: -1 }, /maxTransientRetries/], + [{ maxTransientRetries: 1.5 }, /maxTransientRetries/], + [{ initialDelayMs: 0 }, /initialDelayMs/], + [{ maxDelayMs: Number.POSITIVE_INFINITY }, /maxDelayMs/], + [{ initialDelayMs: MAX_TIMER_DELAY_MS + 1 }, /initialDelayMs/], + [{ maxDelayMs: MAX_TIMER_DELAY_MS + 1 }, /maxDelayMs/], + [{ initialDelayMs: 20, maxDelayMs: 10 }, /less than or equal/], + [{ jitterRatio: 1.1 }, /jitterRatio/], + [{ retryableCodes: [] }, /must not be empty/], + [{ retryableCodes: ['SERVER', 'SERVER'] }, /duplicates/], + [{ retryableCodes: [''] }, /non-empty strings/], + ] as const)('fails direct composition for invalid config %#', (config, message) => { + expect(() => { retry.apply(new Context(), config as retry.Config) }).toThrow(message) + }) +}) diff --git a/packages/llm/llm-retry/tsconfig.json b/packages/llm/llm-retry/tsconfig.json new file mode 100644 index 0000000000..44310af6e9 --- /dev/null +++ b/packages/llm/llm-retry/tsconfig.json @@ -0,0 +1,33 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../../vendor/schemastery" + }, + { + "path": "../../llm/llm" + }, + { + "path": "../../core/session" + }, + { + "path": "../../core/agent" + }, + { + "path": "../../util/timeout" + } + ] +} diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index f9142dbc52..ff5bb6e0a2 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -13,7 +13,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa - `ctx.llm.listModels(provider: string): Promise` Discover the models one registered provider currently advertises. - `ctx.llm.stream(options: GenerateOptions): AsyncIterable` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`. -`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification does not replace the adapter's original coded `Error`. +`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`. Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`. @@ -21,12 +21,12 @@ Provider and model metadata is a discovery surface, not a routing whitelist. `re | Event | Mode | Purpose | |---|---|---| -| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) | +| `llm/stream` | waterfall | Intercept/wrap every streaming model call for caching, logging, or routing | ### Extension points - Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata; their defaults use the route id as its name and advertise no models. -- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc. +- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead. ### Content-block vocabulary (`types.ts`) @@ -47,8 +47,9 @@ Every product adapter sends application identity on provider HTTP requests. `att - `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`. - `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history. - `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams. -- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) is the programmatic failure contract. +- `LlmError` — extends `HarnessError`; its stable `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) matches its frozen serializable `failure.code`. The payload may also retain validated status, `Retry-After`, and branded provider request id facts; policy remains outside the error. - `CONTEXT_WINDOW_EXCEEDED_CODE` — the provider-neutral code both DeepSeek adapters use when a request exceeds the model context window, regardless of thrown-HTTP versus in-band finish delivery. `isContextWindowExceededError(detail)` is their shared conservative classifier for OpenAI-compatible provider detail. +- `QUOTA_EXCEEDED_CODE` — the non-transient provider-neutral code for exhausted account quota, balance, credits, budget, or usage limits. `isQuotaExceededError(detail)` keeps those failures distinct from request-rate limits. ### Real adapters @@ -64,7 +65,7 @@ Pass-through; the registry preserves the assembled request prefix, while the sel ## Known Limitations and Deferred Work -- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains the call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. +- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains a single-attempt call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. `@deepseek-ai/dsh-llm-retry` is an optional policy plugin loaded by the shared example spine. - **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/implemented/simplification/2026-07-04-drop-inert-request-knobs.md)). - **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/implemented/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)). - **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw. diff --git a/packages/llm/llm/src/adapter-failure.ts b/packages/llm/llm/src/adapter-failure.ts index 745cbbdc64..b2189fdaf9 100644 --- a/packages/llm/llm/src/adapter-failure.ts +++ b/packages/llm/llm/src/adapter-failure.ts @@ -5,10 +5,10 @@ */ import { HarnessError } from './error.ts' -import type { StreamChunk } from './types.ts' +import type { LlmFailure, StreamChunk } from './types.ts' -/** Errors proven to originate in one model call's final adapter boundary. */ -export type AdapterFailureScope = WeakSet +/** Errors and normalized facts proven to originate in one model call's final adapter boundary. */ +export type AdapterFailureScope = WeakMap /** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */ const adapterFailureScopes = new WeakMap, AdapterFailureScope>() @@ -47,10 +47,70 @@ export function markLlmAdapterFailure( const error = value instanceof Error ? value as Error & { code?: string } : new HarnessError(String(value), 'UNKNOWN', { cause: value }) - failures.add(error) + const carried = error instanceof HarnessError ? ownFailureSnapshot(error) : undefined + const failure = carried !== undefined && carried.code === error.code ? carried : Object.freeze({ + message: errorMessage(error), + code: harnessErrorCode(error), + }) + failures.set(error, failure) return error } +/** Snapshot an own data property without invoking an SDK-defined accessor. */ +function ownFailureSnapshot(error: Error): LlmFailure | undefined { + try { + const descriptor = Object.getOwnPropertyDescriptor(error, 'failure') + return descriptor !== undefined && 'value' in descriptor + ? failureSnapshot(descriptor.value) + : undefined + } catch (_sdkPropertyTrap) { + return undefined + } +} + +/** Validate and detach an arbitrary serializable failure payload. */ +function failureSnapshot(value: unknown): LlmFailure | undefined { + if (typeof value !== 'object' || value === null) return undefined + try { + const candidate = value as Partial + const message = candidate.message + const code = candidate.code + const status = candidate.status + const retryAfterMs = candidate.retryAfterMs + const requestId = candidate.requestId + if (typeof message !== 'string' || message.length === 0 + || typeof code !== 'string' || code.length === 0 + || (status !== undefined && (!Number.isInteger(status) || status < 100 || status > 599)) + || (retryAfterMs !== undefined && (!Number.isFinite(retryAfterMs) || retryAfterMs <= 0)) + || (requestId !== undefined && (typeof requestId !== 'string' || requestId.length === 0))) return undefined + return Object.freeze({ + message, + code, + ...status === undefined ? {} : { status }, + ...retryAfterMs === undefined ? {} : { retryAfterMs }, + ...requestId === undefined ? {} : { requestId }, + }) + } catch (_sdkFailureGetter) { + return undefined + } +} + +/** Read an SDK error message without letting an accessor replace the primary failure. */ +function errorMessage(error: Error): string { + try { + const message: unknown = error.message + if (typeof message === 'string' && message.length > 0) return message + } catch (_sdkMessageGetter) { + // The fallback below preserves a serializable failure beside the original Error. + } + return 'LLM adapter failed' +} + +/** Trust only Harness-owned codes; third-party SDK codes are not our taxonomy. */ +function harnessErrorCode(error: Error): string { + return error instanceof HarnessError ? error.code : 'UNKNOWN' +} + /** * Whether a failure came from final adapter dispatch, iterator construction, * or iteration for the call represented by the exact returned stream handle. @@ -65,3 +125,18 @@ export function isLlmAdapterFailure( const failures = adapterFailureScopes.get(stream) return value instanceof Error && failures !== undefined && failures.has(value) } + +/** + * Retrieve normalized provider facts only for an Error tagged by this exact + * model call's final adapter boundary. + * @param stream - the exact stream returned to the consumer. + * @param value - the caught failure. + * @returns the immutable facts for that call, or `undefined` for middleware, nested, or consumer failures. + */ +export function llmFailureOf( + stream: AsyncIterable, + value: unknown, +): LlmFailure | undefined { + const failures = adapterFailureScopes.get(stream) + return value instanceof Error ? failures?.get(value) : undefined +} diff --git a/packages/llm/llm/src/brand.ts b/packages/llm/llm/src/brand.ts index ee1cf786b1..259dc49bce 100644 --- a/packages/llm/llm/src/brand.ts +++ b/packages/llm/llm/src/brand.ts @@ -1,5 +1,6 @@ /** - * dsh-llm's owned branded id: `CallId` (tool-call correlation). + * dsh-llm's owned branded ids: tool-call correlation and provider request + * diagnostics. * * The `Branded` primitive itself lives in `@deepseek-ai/dsh-brand` (a * zero-dependency type-only package) so every owner of a cross-boundary id can @@ -25,3 +26,15 @@ export type CallId = Branded<'CallId'> export function CallId(id: string): CallId { return id as CallId } + +/** Provider-issued request identifier retained for diagnostics across package boundaries. */ +export type ProviderRequestId = Branded<'ProviderRequestId'> + +/** + * Brand a provider-issued request identifier. + * @param id - the opaque provider-issued string. + * @returns the same string, branded; no validation is performed. + */ +export function ProviderRequestId(id: string): ProviderRequestId { + return id as ProviderRequestId +} diff --git a/packages/llm/llm/src/error.ts b/packages/llm/llm/src/error.ts index 8c1c736492..4ff60c657e 100644 --- a/packages/llm/llm/src/error.ts +++ b/packages/llm/llm/src/error.ts @@ -24,6 +24,9 @@ export class HarnessError extends Error { /** Canonical provider-neutral code for a model request rejected because its context window was exceeded. */ export const CONTEXT_WINDOW_EXCEEDED_CODE = 'CONTEXT_WINDOW_EXCEEDED' +/** Canonical provider-neutral code for an exhausted account quota or balance. */ +export const QUOTA_EXCEEDED_CODE = 'QUOTA' + /** Structured codes and plain phrases that explicitly name a context bound being exceeded. */ const STRUCTURED_CONTEXT_OVERFLOW = new RegExp( String.raw`(?:^|[^a-z0-9])context[\s_-](?:length|window)[\s_-]` @@ -62,6 +65,19 @@ export function isContextWindowExceededError(detail: string): boolean { || EXCEEDS_MODEL_CONTEXT.test(detail) } +/** + * Recognize provider wording that identifies an exhausted account quota rather + * than a transient request-rate limit. + * @param detail - provider error code/type/message text joined into one string. + * @returns true only for terminal quota, balance, credit, budget, or usage-limit wording. + */ +export function isQuotaExceededError(detail: string): boolean { + return /\binsufficient[\s_-]+(?:quota|balance|credits?)\b/i.test(detail) + || /\b(?:quota|usage[\s_-]+limit)[\s_-]+(?:exceeded|exhausted|reached)\b/i.test(detail) + || /\b(?:balance|credits?)[\s_-]+(?:exhausted|depleted)\b/i.test(detail) + || /\bout[\s_-]+of[\s_-]+(?:credits?|budget)\b/i.test(detail) +} + /** * Narrow an arbitrary thrown value to a HarnessError (for `instanceof` at seams). * @param value - the caught value (`unknown` in catch clauses). diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index f276aa9f92..dfebb059be 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -7,7 +7,8 @@ */ import { Context, Service } from 'cordis' -import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts' +import type { GenerateOptions, LlmFailure, LlmModelInfo, LlmProviderInfo, Message, StreamChunk } from './types.ts' +import type { ProviderRequestId } from './brand.ts' import { deepFreeze } from './call-config.ts' import { HarnessError } from './error.ts' import { bindAdapterFailureScope, markLlmAdapterFailure } from './adapter-failure.ts' @@ -21,7 +22,7 @@ export * from './types.ts' export { BlockAssembler } from './assembler.ts' export { callConfigEquals, deepFreeze } from './call-config.ts' export type { LlmCallConfig } from './call-config.ts' -export { isLlmAdapterFailure } from './adapter-failure.ts' +export { isLlmAdapterFailure, llmFailureOf } from './adapter-failure.ts' declare module 'cordis' { interface Context { @@ -44,14 +45,53 @@ declare module 'cordis' { } } +/** Structured provider facts and cause accepted by {@link LlmError}. */ +export interface LlmErrorOptions extends ErrorOptions { + /** Valid HTTP status observed at the provider boundary. */ + status?: number + /** Positive finite provider-requested delay in milliseconds. */ + retryAfterMs?: number + /** Non-empty opaque provider request id. */ + requestId?: ProviderRequestId +} + /** * Typed error for LLM-related failures. Extends {@link HarnessError}, so the * `code` string (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`) is shared taxonomy. */ export class LlmError extends HarnessError { - constructor(message: string, code: string, options?: ErrorOptions) { + /** Serializable facts retained beside this live Error. */ + readonly failure: LlmFailure + + /** + * @param message - non-empty human-readable failure summary. + * @param code - non-empty stable provider-neutral machine code. + * @param options - optional cause and validated serializable provider facts. + */ + constructor(message: string, code: string, options?: LlmErrorOptions) { + if (typeof message !== 'string' || message.length === 0) throw new Error('LlmError message must be a non-empty string') + if (typeof code !== 'string' || code.length === 0) throw new Error('LlmError code must be a non-empty string') + if (options?.status !== undefined + && (!Number.isInteger(options.status) || options.status < 100 || options.status > 599)) { + throw new Error('LlmError status must be an integer from 100 through 599') + } + if (options?.retryAfterMs !== undefined + && (!Number.isFinite(options.retryAfterMs) || options.retryAfterMs <= 0)) { + throw new Error('LlmError retryAfterMs must be a positive finite number') + } + if (options?.requestId !== undefined + && (typeof options.requestId !== 'string' || options.requestId.length === 0)) { + throw new Error('LlmError requestId must be a non-empty string') + } super(message, code, options) this.name = 'LlmError' + this.failure = Object.freeze({ + message, + code, + ...options?.status === undefined ? {} : { status: options.status }, + ...options?.retryAfterMs === undefined ? {} : { retryAfterMs: options.retryAfterMs }, + ...options?.requestId === undefined ? {} : { requestId: options.requestId }, + }) } } @@ -262,7 +302,7 @@ export class LlmService extends Service { * @returns the chunk stream, possibly wrapped by `llm/stream` listeners. */ stream(options: GenerateOptions): AsyncIterable { - const failures: AdapterFailureScope = new WeakSet() + const failures: AdapterFailureScope = new WeakMap() const stream = this.ctx.waterfall(this, 'llm/stream', options, () => this.adapterStream(options, failures)) return bindAdapterFailureScope(stream, failures) } diff --git a/packages/llm/llm/src/types.ts b/packages/llm/llm/src/types.ts index b8054c2046..f8412aee2b 100644 --- a/packages/llm/llm/src/types.ts +++ b/packages/llm/llm/src/types.ts @@ -5,7 +5,21 @@ */ import type { Branded } from '@deepseek-ai/dsh-brand' -import type { CallId } from './brand.ts' +import type { CallId, ProviderRequestId } from './brand.ts' + +/** Serializable provider-boundary facts; policy decides whether they are retryable. */ +export interface LlmFailure { + /** Human-readable provider or transport failure. */ + readonly message: string + /** Stable provider-neutral machine-routing code. */ + readonly code: string + /** HTTP status observed at the provider boundary, when available. */ + readonly status?: number + /** Provider-requested delay in milliseconds, when valid and available. */ + readonly retryAfterMs?: number + /** Opaque provider-issued request identifier for diagnostics. */ + readonly requestId?: ProviderRequestId +} /** Plain text visible to the end user. */ export interface TextBlock { @@ -98,8 +112,8 @@ export interface FinishReasonMap { 'stop': { kind: 'stop' } 'tool-calls': { kind: 'tool-calls' } 'max-tokens': { kind: 'max-tokens' } - 'aborted': { kind: 'aborted' } - 'error': { kind: 'error'; message: string; code?: string } + 'aborted': { kind: 'aborted'; failure: LlmFailure } + 'error': { kind: 'error'; failure: LlmFailure } } /** Any known finish reason, derived from {@link FinishReasonMap}; switch on `kind` and fall through unknowns (merge-extensible). */ diff --git a/packages/llm/llm/tests/properties.spec.ts b/packages/llm/llm/tests/properties.spec.ts index 0d65b545d1..31d07c1a47 100644 --- a/packages/llm/llm/tests/properties.spec.ts +++ b/packages/llm/llm/tests/properties.spec.ts @@ -41,7 +41,10 @@ const chunkArb: fc.Arbitrary = indexArb.chain(index => fc.oneof( fc.constant({ type: 'usage', usage: { inputTokens: 1, outputTokens: 1 } }), fc.constant({ type: 'finish', reason: { kind: 'stop' } }), fc.constant({ type: 'finish', reason: { kind: 'tool-calls' } }), - fc.string().map((message): StreamChunk => ({ type: 'finish', reason: { kind: 'error', message } })), + fc.string({ minLength: 1 }).map((message): StreamChunk => ({ + type: 'finish', + reason: { kind: 'error', failure: { message, code: 'UNKNOWN' } }, + })), )) /** A stream is an arbitrary list of chunks (we do NOT force a terminal finish). */ diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index 6e14d749ba..e491ad2dc8 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -4,9 +4,12 @@ import LlmService, { GenerateOptions, HarnessError, isContextWindowExceededError, + isQuotaExceededError, isLlmAdapterFailure, LlmAdapter, LlmError, + llmFailureOf, + ProviderRequestId, StreamChunk, } from '@deepseek-ai/dsh-llm' import type { LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm' @@ -80,6 +83,17 @@ describe('LlmService', () => { expect(isContextWindowExceededError('context window size must be positive')).toBe(false) }) + it('distinguishes exhausted account quota from transient rate limiting', () => { + for (const detail of [ + 'insufficient_quota', + 'account balance depleted', + 'usage-limit-exceeded', + 'out of credits', + ]) expect(isQuotaExceededError(detail)).toBe(true) + expect(isQuotaExceededError('HTTP 429: rate limit reached')).toBe(false) + expect(isQuotaExceededError('quota resets in one minute')).toBe(false) + }) + it('routes stream() to the registered adapter', async () => { const ctx = new Context() await ctx.plugin(LlmService) @@ -168,6 +182,151 @@ describe('LlmService', () => { expect(caught).toBe(original) expect(isLlmAdapterFailure(stream, caught)).toBe(true) + expect(llmFailureOf(stream, caught)).toEqual({ + message: `${boundary} failed`, + code: 'BOUNDARY_FAILED', + }) + }) + + it('keeps structured provider facts beside a frozen third-party Error', async () => { + const original = new LlmError('provider busy', 'RATE_LIMIT', { + status: 429, + retryAfterMs: 1_500, + requestId: ProviderRequestId('req-7'), + }) + Object.freeze(original) + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original)) + + const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] }) + let caught: unknown + try { + for await (const _chunk of stream) { /* drain */ } + } catch (error: unknown) { + caught = error + } + + expect(caught).toBe(original) + expect(llmFailureOf(stream, caught)).toEqual({ + message: 'provider busy', + code: 'RATE_LIMIT', + status: 429, + retryAfterMs: 1_500, + requestId: ProviderRequestId('req-7'), + }) + }) + + it('does not trust retry facts carried by an unknown third-party Error', async () => { + const carried = { message: 'busy', code: 'SERVER', status: 503 } + const original = Object.assign(new Error('busy'), { failure: carried }) + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original)) + + const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] }) + await expect((async () => { + for await (const _chunk of stream) { /* drain */ } + })()).rejects.toBe(original) + const facts = llmFailureOf(stream, original) + carried.status = 500 + + expect(facts).toEqual({ message: 'busy', code: 'UNKNOWN' }) + expect(Object.isFrozen(facts)).toBe(true) + expect(facts).not.toBe(carried) + }) + + it('keeps an unknown SDK Error exact without trusting its private code or accessors', async () => { + const original = Object.assign(new Error('socket closed'), { code: 'ECONNRESET' }) + Object.defineProperty(original, 'failure', { + get() { throw new Error('SDK failure accessor must not run') }, + }) + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original)) + + const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] }) + await expect((async () => { + for await (const _chunk of stream) { /* drain */ } + })()).rejects.toBe(original) + + expect(original.code).toBe('ECONNRESET') + expect(llmFailureOf(stream, original)).toEqual({ message: 'socket closed', code: 'UNKNOWN' }) + }) + + it('keeps an SDK Error exact when its message accessor is hostile', async () => { + const original = Object.defineProperty(new Error(), 'message', { + get() { throw new Error('SDK message accessor trap') }, + }) + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original)) + const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] }) + + await expect((async () => { + for await (const _chunk of stream) { /* drain */ } + })()).rejects.toBe(original) + expect(llmFailureOf(stream, original)).toEqual({ message: 'LLM adapter failed', code: 'UNKNOWN' }) + }) + + it('falls back safely when SDK objects trap failure inspection or expose malformed facts', async () => { + const propertyTrap = new Proxy(new HarnessError('descriptor trapped', 'SERVER'), { + getOwnPropertyDescriptor(target, property) { + if (property === 'failure') throw new Error('SDK descriptor trap') + return Reflect.getOwnPropertyDescriptor(target, property) + }, + }) + const throwingFacts = Object.create(null) as Record + Object.defineProperty(throwingFacts, 'message', { + get() { throw new Error('SDK fact getter trap') }, + }) + const carrying = (message: string, failure: unknown): HarnessError => Object.defineProperty( + new HarnessError(message, 'SERVER'), + 'failure', + { value: failure }, + ) + const factGetter = carrying('fact getter failed', throwingFacts) + const malformed = carrying('malformed facts', { message: 'provider busy', code: 'SERVER', requestId: 1 }) + const primitive = carrying('primitive facts', 1) + const nullFacts = carrying('null facts', null) + const mismatched = carrying('mismatched facts', { message: 'busy', code: 'RATE_LIMIT' }) + + for (const [original, expectedMessage] of [ + [propertyTrap, 'descriptor trapped'], + [factGetter, 'fact getter failed'], + [malformed, 'malformed facts'], + [primitive, 'primitive facts'], + [nullFacts, 'null facts'], + [mismatched, 'mismatched facts'], + ] as const) { + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original)) + const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] }) + + await expect((async () => { + for await (const _chunk of stream) { /* drain */ } + })()).rejects.toBe(original) + expect(llmFailureOf(stream, original)).toEqual({ message: expectedMessage, code: 'SERVER' }) + } + }) + + it('retains a stable code from a HarnessError without requiring LlmError facts', async () => { + const original = new HarnessError('stable adapter failure', 'ADAPTER_STABLE') + const ctx = new Context() + await ctx.plugin(LlmService) + ctx.llm.registerAdapter(['test-provider'], new ThrowingAdapter(original)) + const stream = ctx.llm.stream({ provider: 'test-provider', model: 'test-model', messages: [] }) + + await expect((async () => { + for await (const _chunk of stream) { /* drain */ } + })()).rejects.toBe(original) + expect(llmFailureOf(stream, original)).toEqual({ + message: 'stable adapter failure', + code: 'ADAPTER_STABLE', + }) + expect(llmFailureOf(stream, 'not an Error')).toBeUndefined() + expect(llmFailureOf({ [Symbol.asyncIterator]: () => stream[Symbol.asyncIterator]() }, original)).toBeUndefined() }) it('keeps a nested adapter failure scoped to the nested model call', async () => { @@ -586,6 +745,15 @@ describe('LlmService', () => { expect(err.code).toBe('CUSTOM_CODE') }) + it('rejects non-serializable structured failure facts at construction', () => { + expect(() => new LlmError('busy', 'RATE_LIMIT', { status: 42 })).toThrow(/status/) + expect(() => new LlmError('busy', 'RATE_LIMIT', { retryAfterMs: Number.NaN })).toThrow(/retryAfterMs/) + expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: ProviderRequestId('') })).toThrow(/requestId/) + expect(() => new LlmError(1 as never, 'RATE_LIMIT')).toThrow(/message/) + expect(() => new LlmError('busy', 1 as never)).toThrow(/code/) + expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: 1 as never })).toThrow(/requestId/) + }) + it('LlmError extends the shared HarnessError base', async () => { const { HarnessError, isHarnessError } = await import('@deepseek-ai/dsh-llm') const cause = new Error('root cause') diff --git a/packages/support/llm-replay/tests/llm-replay.spec.ts b/packages/support/llm-replay/tests/llm-replay.spec.ts index 59ffc485db..846085deec 100644 --- a/packages/support/llm-replay/tests/llm-replay.spec.ts +++ b/packages/support/llm-replay/tests/llm-replay.spec.ts @@ -142,7 +142,7 @@ describe('deriveReplayScript', () => { it('keeps a finish-error chunk in the derived entry (replays naturally)', () => { const errChunks: StreamChunk[] = [ { type: 'block-start', index: 0, blockType: 'text' }, - { type: 'finish', reason: { kind: 'error', message: 'boom', code: 'X' } }, + { type: 'finish', reason: { kind: 'error', failure: { message: 'boom', code: 'X' } } }, ] const events = errChunks.map((c, i) => chunkEvent(i + 1, 1, 1, c)) expect(deriveReplayScript(events)).toEqual([{ kind: 'chunks', chunks: errChunks }]) diff --git a/packages/ui/acp/README.md b/packages/ui/acp/README.md index 44748add2a..b11165bd26 100644 --- a/packages/ui/acp/README.md +++ b/packages/ui/acp/README.md @@ -30,7 +30,7 @@ The `initialize` handshake reports a fixed server identity (`agentInfo: { name: | `session/load` | `ctx.agents.resume(...)` | reserves the id, verifies the persisted cwd, resumes, and replays user, assistant, and tool events | | `session/prompt` | `agent.send()` | supports ACP `text` and `resource_link` blocks; rejects image/audio/embedded resource and empty prompts; one in-flight prompt PER session (independent); settles on the OWNING turn's end (a turn that ends in `error` rejects the RPC) | | `session/cancel` | `agent.cancel()` | the queue-aware cancel: aborts a running step, clears queued + steering work, and drops a turn about to start, then settles the prompt `cancelled` — for ONLY that session (a cancel never touches another session's stream or prompt) | -| `session/update` | `session/event` | streams user replay, assistant text/reasoning, and tool render intents | +| `session/update` | `session/event` | streams user replay, assistant text/reasoning, retry/failure attempt markers, and tool render intents | | `elicitation/create` | `ctx.userInteraction.ask()` | maps `ask_user_question` questions to ACP form elicitations; option descriptions are shown in enum titles, `multi_select` uses ACP array enums, optionless requests use a required `custom` field, and a non-empty custom answer overrides any selected choice | | `session/request_permission` | `approval/request` listener | answers one-shot allow/reject requests for bridge-owned calls; foreign or call-less requests delegate and fail closed if unanswered — see "Permission prompts" | | `session/set_config_option` | agent-scoped request target / `ctx.permission.set()` | per-session provider+model and permission-preset switching over [session config options](https://agentclientprotocol.com/protocol/session-config-options) — see "Session config options" | @@ -47,6 +47,8 @@ When `ctx.permission` is composed, the bridge also advertises a `permission` sel The shared [`ctx.tasks` runtime](../../tasks/tasks/) fences access to predictable task ids by the owning session; ACP sessions therefore cannot read or stop one another's background work. +ACP updates are append-only, so `llm/retry` emits a visible separator that marks preceding partial model output discarded before the next attempt streams. A terminal model-request failure emits the same discarded-output warning; replay derives both markers from the durable events. + ## Per-session cwd `session/new` records the request's absolute cwd in the session header. Before constructing an agent, `session/load` uses persisted metadata to require an absolute request cwd that matches the stored one. Bash defaults to that workspace; an explicit relative workdir resolves against it, and multiple sessions may use different workspaces. `additionalDirectories` remains unsupported. diff --git a/packages/ui/acp/package.json b/packages/ui/acp/package.json index 84e4dcbda9..94cdd10743 100644 --- a/packages/ui/acp/package.json +++ b/packages/ui/acp/package.json @@ -30,6 +30,7 @@ "@deepseek-ai/dsh-agent": "^0.0.1", "@deepseek-ai/dsh-bash": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-llm-retry": "^0.0.1", "@deepseek-ai/dsh-permission": "^0.0.1", "@deepseek-ai/dsh-sandbox": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", @@ -50,6 +51,7 @@ "@deepseek-ai/dsh-fs-policy": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-permission": "workspace:^", "@deepseek-ai/dsh-sandbox": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", diff --git a/packages/ui/acp/src/index.ts b/packages/ui/acp/src/index.ts index fe6b2630c7..6d549af8e1 100644 --- a/packages/ui/acp/src/index.ts +++ b/packages/ui/acp/src/index.ts @@ -44,6 +44,7 @@ import { } from '@agentclientprotocol/sdk' import type { ContentBlock, LlmCallConfig, LlmModelInfo, LlmProviderInfo } from '@deepseek-ai/dsh-llm' import { assertNever, CallId } from '@deepseek-ai/dsh-llm' +import type {} from '@deepseek-ai/dsh-llm-retry' import type { Agent } from '@deepseek-ai/dsh-agent' import { SessionId } from '@deepseek-ai/dsh-session' // Side-effect type import: resolves `ctx.get('permission')` to the service. @@ -481,7 +482,7 @@ export function apply(ctx: Context, config: AcpConfig): void { reason: TurnEndReason, ): void => { if (reason.kind === 'error') { - inflight.reject(internalError(`turn failed: ${reason.message}`)) + inflight.reject(internalError(`turn failed: ${'failure' in reason ? reason.failure.message : reason.message}`)) } else { inflight.resolve(turnEndToStopReason(reason)) } @@ -1035,6 +1036,7 @@ function validateMcpServers(params: { mcpServers?: unknown[] }): void { * identical update stream from the same event log. * * - `assistant/chunk` text-delta/reasoning-delta → message/thought chunks + * - `llm/retry` and terminal model failure → visible discarded-attempt markers * - `user/message` → `user_message_chunk` during load replay only — so a * loaded transcript reconstructs the USER side of each turn without echoing * a live `session/prompt` back to the client @@ -1081,6 +1083,13 @@ export function streamSessionEventUpdate( } return } + case 'llm/retry': { + const text = '\n\n[Previous model attempt discarded; retrying ' + + `${event.data.retry}/${event.data.maxRetries} in ${event.data.delayMs}ms: ` + + `${event.data.failure.message}]\n\n` + notify({ sessionId, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text } } }) + return + } case 'user/message': { if (!includeUserMessages) return // Replay the user's prompt so a loaded session shows both sides of each @@ -1108,7 +1117,16 @@ export function streamSessionEventUpdate( notify({ sessionId, update: { sessionUpdate: 'plan', ...todosToPlan(event.data.todos) } }) return } - // turn/step boundaries, context/message, steering, + case 'turn/end': { + if (event.data.reason.kind !== 'error') return + const message = 'failure' in event.data.reason + ? event.data.reason.failure.message + : event.data.reason.message + const text = `\n\n[Model attempt failed; any partial output above is discarded: ${message}]\n\n` + notify({ sessionId, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text } } }) + return + } + // non-error turn/step boundaries, context/message, steering, // assistant/message — no direct ACP client update. default: return diff --git a/packages/ui/acp/tests/harness.ts b/packages/ui/acp/tests/harness.ts index 1796cbf868..b8b47b80d5 100644 --- a/packages/ui/acp/tests/harness.ts +++ b/packages/ui/acp/tests/harness.ts @@ -100,7 +100,7 @@ export function errorResponse(message: string): StreamChunk[] { return [ { type: 'block-start', index: 0, blockType: 'text' }, { type: 'text-delta', index: 0, text: 'partial' }, - { type: 'finish', reason: { kind: 'error', message, code: 'PROVIDER_ERROR' } }, + { type: 'finish', reason: { kind: 'error', failure: { message, code: 'PROVIDER_ERROR' } } }, ] } diff --git a/packages/ui/acp/tests/stream-update.spec.ts b/packages/ui/acp/tests/stream-update.spec.ts index 415e9afb33..e451099744 100644 --- a/packages/ui/acp/tests/stream-update.spec.ts +++ b/packages/ui/acp/tests/stream-update.spec.ts @@ -65,6 +65,33 @@ describe('streamSessionEventUpdate', () => { .toEqual([]) }) + it('marks retry and terminal failure boundaries in the append-only update stream', () => { + expect(updatesFor(evt('llm/retry', { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 500, + failure: { message: 'backend busy', code: 'SERVER' }, + }))).toEqual([{ + sessionUpdate: 'agent_message_chunk', + content: { + type: 'text', + text: '\n\n[Previous model attempt discarded; retrying 1/2 in 500ms: backend busy]\n\n', + }, + }]) + expect(updatesFor(evt('turn/end', { + turn: 1, + reason: { kind: 'error', step: 2, failure: { message: 'still busy', code: 'SERVER' } }, + }))).toEqual([{ + sessionUpdate: 'agent_message_chunk', + content: { + type: 'text', + text: '\n\n[Model attempt failed; any partial output above is discarded: still busy]\n\n', + }, + }]) + }) + it('maps tool/call to an in_progress tool_call with kind other and parsed rawInput (generic fallback, no presenter)', () => { const updates = updatesFor(evt('tool/call', { turn: 1, step: 1, callId: CallId('c1'), name: 'bash', arguments: '{"command":"ls"}' })) expect(updates).toEqual([{ diff --git a/packages/ui/acp/tests/turns.spec.ts b/packages/ui/acp/tests/turns.spec.ts index 15c2415449..e3e8d679ea 100644 --- a/packages/ui/acp/tests/turns.spec.ts +++ b/packages/ui/acp/tests/turns.spec.ts @@ -49,6 +49,15 @@ describe('acp bridge — turn outcomes', () => { .rejects.toThrow(/turn failed: provider boom/) }) + it('rejects an ordinary plugin turn failure through the same ACP boundary', async () => { + harness = await makeBridgeHarness({ storageDir, script: [textResponse('must not run')] }) + harness.ctx.on('agent/pre-step', () => { throw new Error('plugin pre-step failed') }) + const sessionId = await newSession(harness) + + await expect(harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] })) + .rejects.toThrow(/turn failed: plugin pre-step failed/) + }) + it('streams a tool call as tool_call then tool_call_update', async () => { harness = await makeBridgeHarness({ storageDir, diff --git a/packages/ui/acp/tsconfig.json b/packages/ui/acp/tsconfig.json index 387e0d0c53..de4ccd3650 100644 --- a/packages/ui/acp/tsconfig.json +++ b/packages/ui/acp/tsconfig.json @@ -20,6 +20,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../llm/llm-retry" + }, { "path": "../../core/session" }, diff --git a/packages/ui/stdio/README.md b/packages/ui/stdio/README.md index eda7d00b8b..23cdb65ab1 100644 --- a/packages/ui/stdio/README.md +++ b/packages/ui/stdio/README.md @@ -11,7 +11,7 @@ This package owns the terminal channel only. It injects `agents` and `userIntera | `welcome` | `ready.` | Banner printed before the first prompt | | `sessionId` | `main` | Exact agent/session identity driven by stdin and observed for EOF shutdown | -The plugin seeds display labels from the live agent registry, then tracks `agent/created` and `agent/disposed` so HMR and externally managed agents render consistently. While an initial exact identity is pending, it buffers nonblank input until `agent/session-start` and observes live `agent-loop/config-start-failed`; a matching failure drops queued lines, reports the loss, and lets piped EOF finish instead of hanging. The composing app must mount this front door before its config-created agent. Disposal closes readline and unregisters every listener/provider through Cordis effects. +The plugin seeds display labels from the live agent registry, then tracks `agent/created` and `agent/disposed` so HMR and externally managed agents render consistently. While an initial exact identity is pending, it buffers nonblank input until `agent/session-start` and observes live `agent-loop/config-start-failed`; a matching failure drops queued lines, reports the loss, and lets piped EOF finish instead of hanging. When a composed retry policy closes a failed step, the append-only transcript inserts an explicit discarded-attempt marker before later chunks; terminal request failure marks any preceding partial output discarded. The composing app must mount this front door before its config-created agent. Disposal closes readline and unregisters every listener/provider through Cordis effects. ```yaml - id: stdio diff --git a/packages/ui/stdio/package.json b/packages/ui/stdio/package.json index e1bffdf171..783f304823 100644 --- a/packages/ui/stdio/package.json +++ b/packages/ui/stdio/package.json @@ -25,6 +25,7 @@ "@deepseek-ai/dsh-agent": "^0.0.1", "@deepseek-ai/dsh-agent-loop": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-llm-retry": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "@deepseek-ai/dsh-user-interaction": "^0.0.1", "cordis": "^4.0.0-rc.7" @@ -42,6 +43,7 @@ "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-agent-loop": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-user-interaction": "workspace:^", "cordis": "^4.0.0-rc.7" diff --git a/packages/ui/stdio/src/index.ts b/packages/ui/stdio/src/index.ts index 6f73d948bf..3491c354a2 100644 --- a/packages/ui/stdio/src/index.ts +++ b/packages/ui/stdio/src/index.ts @@ -16,6 +16,7 @@ import type { Context } from 'cordis' import z from 'schemastery' import type { Agent } from '@deepseek-ai/dsh-agent' import type {} from '@deepseek-ai/dsh-agent-loop' +import type {} from '@deepseek-ai/dsh-llm-retry' import { SessionId } from '@deepseek-ai/dsh-session' import { UserInteractionError, @@ -119,6 +120,10 @@ export function createStdioChat(ctx: Context, config: Config, runtime: StdioRunt // append order keeps `inReasoning` transitions deterministic across chunk and // boundary events. let inReasoning = false + const resetReasoning = (): void => { + if (inReasoning) output.write('\x1B[0m') + inReasoning = false + } ctx.on('session/event', (session, event) => { if (event.type === 'assistant/chunk') { const { chunk } = event.data @@ -135,22 +140,31 @@ export function createStdioChat(ctx: Context, config: Config, runtime: StdioRunt } else if (event.type === 'turn/start') { const label = target?.session === session ? 'main' : session.id output.write(`\n[${label} turn ${event.data.turn}] `) + } else if (event.type === 'llm/retry') { + resetReasoning() + output.write( + `\n [previous model attempt discarded; retry ${event.data.retry}/${event.data.maxRetries}` + + ` in ${event.data.delayMs}ms: ${event.data.failure.message}]\n `, + ) } else if (event.type === 'turn/end') { - if (inReasoning) output.write('\x1B[0m') - inReasoning = false + resetReasoning() + if (event.data.reason.kind === 'error') { + const message = 'failure' in event.data.reason + ? event.data.reason.failure.message + : event.data.reason.message + output.write(`\n [model attempt failed; any partial output above is discarded: ${message}]`) + } output.write('\n> ') } else if (event.type === 'tool/call') { const { name: toolName, arguments: args } = event.data - if (inReasoning) output.write('\x1B[0m') - inReasoning = false + resetReasoning() output.write(`\n [tool call] ${toolName}(${args})`) } else if (event.type === 'tool/result') { const { content } = event.data const text = content.filter(block => block.type === 'text').map(block => block.text).join('') output.write(`\n [tool result] ${text}\n `) } else if (event.type === 'todo/write') { - if (inReasoning) output.write('\x1B[0m') - inReasoning = false + resetReasoning() const glyph = (status: string): string => status === 'completed' ? '[x]' : status === 'in_progress' ? '[~]' : '[ ]' const lines = event.data.todos.map(todo => ` ${glyph(todo.status)} ${todo.content}`).join('\n') diff --git a/packages/ui/stdio/tests/stdio.spec.ts b/packages/ui/stdio/tests/stdio.spec.ts index a3069462ff..4babc8c117 100644 --- a/packages/ui/stdio/tests/stdio.spec.ts +++ b/packages/ui/stdio/tests/stdio.spec.ts @@ -290,6 +290,51 @@ describe('createStdioChat rendering', () => { expect(out.text()).toContain('\x1B[2mmid\x1B[0m') }) + it('marks failed partial output at retry and terminal failure boundaries', async () => { + const { ctx, out } = await setup() + const session = makeSession('main') + ctx.emit('session/event', session, chunkEvent({ type: 'reasoning-delta', index: 0, text: 'partial' })) + ctx.emit('session/event', session, { + type: 'llm/retry', + seq: 1, + time: 0, + data: { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 500, + failure: { message: 'backend busy', code: 'SERVER' }, + }, + }) + ctx.emit('session/event', session, { + type: 'turn/end', + seq: 3, + time: 0, + data: { turn: 2, reason: { kind: 'error', step: 1, message: 'loop defect' } }, + }) + ctx.emit('session/event', session, chunkEvent({ type: 'text-delta', index: 0, text: 'also partial' })) + ctx.emit('session/event', session, { + type: 'turn/end', + seq: 2, + time: 0, + data: { + turn: 1, + reason: { kind: 'error', step: 2, failure: { message: 'still busy', code: 'SERVER' } }, + }, + }) + + expect(out.text()).toContain( + '\x1B[2mpartial\x1B[0m\n [previous model attempt discarded; retry 1/2 in 500ms: backend busy]', + ) + expect(out.text()).toContain( + 'also partial\n [model attempt failed; any partial output above is discarded: still busy]\n> ', + ) + expect(out.text()).toContain( + '[model attempt failed; any partial output above is discarded: loop defect]\n> ', + ) + }) + it('drops the target object on agent/disposed', async () => { const { ctx, out } = await setup() const agent = makeAgent('main') diff --git a/packages/ui/stdio/tsconfig.json b/packages/ui/stdio/tsconfig.json index e0c578ed32..ca69d43ffd 100644 --- a/packages/ui/stdio/tsconfig.json +++ b/packages/ui/stdio/tsconfig.json @@ -26,6 +26,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../llm/llm-retry" + }, { "path": "../user-interaction" } diff --git a/packages/ui/tui/README.md b/packages/ui/tui/README.md index 6d2c7858e4..61e6987104 100644 --- a/packages/ui/tui/README.md +++ b/packages/ui/tui/README.md @@ -6,7 +6,7 @@ The implemented [TUI feature Agent Note](../../../.agents/notes/implemented/feat This package owns interactive terminal presentation and input only. It injects `agents`, `tools`, and `userInteraction`, then drives an agent created or resumed by app or developer code. Agent lifecycle, persistence, and the model-facing [`ask_user_question`](../tool-ask-user/README.md) tool remain separate composition entries. -The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions as keyboard-driven overlays. Surface replacement events rebuild the transcript so compacted history does not reappear. +The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions as keyboard-driven overlays. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Surface replacement events rebuild the transcript so compacted history does not reappear. Before model output, session events, tool presenters, questions, configuration, or diagnostics reach pi-tui's ANSI-aware renderers or the terminal title, the TUI renders C0 and C1 controls other than line feeds as visible `\xNN` text. Those sources cannot add terminal control sequences; the TUI and pi-tui retain ownership of terminal rendering and styling. diff --git a/packages/ui/tui/package.json b/packages/ui/tui/package.json index fd4f187e35..5e7c8d060c 100644 --- a/packages/ui/tui/package.json +++ b/packages/ui/tui/package.json @@ -25,6 +25,7 @@ "@deepseek-ai/dsh-agent": "^0.0.1", "@deepseek-ai/dsh-agent-loop": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-llm-retry": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "@deepseek-ai/dsh-tools": "^0.0.1", "@deepseek-ai/dsh-user-interaction": "^0.0.1", @@ -39,6 +40,7 @@ "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-agent-loop": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", + "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-tool-cordis": "workspace:^", diff --git a/packages/ui/tui/src/index.ts b/packages/ui/tui/src/index.ts index 1c3fc1315c..39f2a5c40c 100644 --- a/packages/ui/tui/src/index.ts +++ b/packages/ui/tui/src/index.ts @@ -35,7 +35,8 @@ import type { Context } from 'cordis' import z from 'schemastery' import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent' import type {} from '@deepseek-ai/dsh-agent-loop' -import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm' +import type { ContentBlock, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm' +import type {} from '@deepseek-ai/dsh-llm-retry' import { SessionId, type Session, type SessionEvent, type TodoItem } from '@deepseek-ai/dsh-session' import type { FileDiff, @@ -612,15 +613,38 @@ function formatCwd(cwd: string | undefined): string { return displayText(cwd) } -function sessionTokens(session: Session): { input: number; output: number } { - let input = 0 - let output = 0 - for (const event of session.events) { - if (event.type !== 'assistant/message' || event.data.usage === undefined) continue - input += event.data.usage.inputTokens - output += event.data.usage.outputTokens +interface SessionTokenTotals { + input: number + output: number + readonly byStep: Map +} + +function recordTokenUsage(totals: SessionTokenTotals, turn: number, step: number, usage: TokenUsage): void { + const key = `${turn}:${step}` + const previous = totals.byStep.get(key) + if (previous !== undefined) { + totals.input -= previous.inputTokens + totals.output -= previous.outputTokens } - return { input, output } + totals.byStep.set(key, usage) + totals.input += usage.inputTokens + totals.output += usage.outputTokens +} + +function recordEventUsage(totals: SessionTokenTotals, event: SessionEvent): void { + if (event.type === 'assistant/chunk' && event.data.chunk.type === 'usage') { + recordTokenUsage(totals, event.data.turn, event.data.step, event.data.chunk.usage) + } else if (event.type === 'assistant/message' && event.data.usage !== undefined) { + recordTokenUsage(totals, event.data.turn, event.data.step, event.data.usage) + } +} + +function sessionTokens(session: Session): SessionTokenTotals { + const totals: SessionTokenTotals = { input: 0, output: 0, byStep: new Map() } + for (const event of session.events) { + recordEventUsage(totals, event) + } + return totals } class FooterComponent implements Component { @@ -899,6 +923,14 @@ export function createTuiChat( return card } + const clearStreaming = (): void => { + if (streaming === undefined) return + const index = chat.children.indexOf(streaming) + /* v8 ignore next -- streaming is assigned only after the same component is added, and every removal clears it. */ + if (index >= 0) chat.children.splice(index, 1) + streaming = undefined + } + const renderEvent = (event: SessionEvent, options: { addHistory: boolean; renderChunks: boolean }): void => { switch (event.type) { case 'user/message': { @@ -941,15 +973,19 @@ export function createTuiChat( } break case 'assistant/message': { - if (streaming !== undefined) { - const index = chat.children.indexOf(streaming) - if (index >= 0) chat.children.splice(index, 1) - streaming = undefined - } + clearStreaming() const component = new AssistantMessageComponent(event.data.content, showReasoning, palette, mdTheme) if (component.children.length > 0) chat.addChild(component) break } + case 'llm/retry': { + clearStreaming() + appendNotice( + `Retrying model request (${event.data.retry}/${event.data.maxRetries}) in ${event.data.delayMs}ms: ${event.data.failure.message}`, + 'warning', + ) + break + } case 'tool/call': chat.addChild(new Spacer(1)) chat.addChild(parsedTool(event)) @@ -970,9 +1006,13 @@ export function createTuiChat( todo.update(event.data.todos) break case 'turn/end': + clearStreaming() if (event.data.reason.kind === 'error') { const key = `${event.data.turn}:${event.data.reason.step}` - if (!liveErrors.delete(key)) appendNotice(event.data.reason.message, 'error') + const message = 'failure' in event.data.reason + ? event.data.reason.failure.message + : event.data.reason.message + if (!liveErrors.delete(key)) appendNotice(message, 'error') } else if (event.data.reason.kind === 'aborted') { appendNotice(event.data.reason.reason ?? 'Turn cancelled.', 'warning') } else if (event.data.reason.kind === 'max-tokens') { @@ -1245,10 +1285,7 @@ export function createTuiChat( const disposeSessionEvents = ctx.on('session/event', (session, event) => { if (session !== agent.session) return - if (event.type === 'assistant/message' && event.data.usage !== undefined) { - tokens.input += event.data.usage.inputTokens - tokens.output += event.data.usage.outputTokens - } + recordEventUsage(tokens, event) if ('surfaceOp' in event && typeof event.surfaceOp === 'object') { rebuildTranscript(false) return diff --git a/packages/ui/tui/tests/harness.ts b/packages/ui/tui/tests/harness.ts index 9994833308..96d0570921 100644 --- a/packages/ui/tui/tests/harness.ts +++ b/packages/ui/tui/tests/harness.ts @@ -120,10 +120,11 @@ export function appendAssistant( session: Session, content: ContentBlock[], usage?: { inputTokens: number; outputTokens: number }, + position: { turn: number; step: number } = { turn: 1, step: 0 }, ): void { session.append('assistant/message', { - turn: 1, - step: 0, + turn: position.turn, + step: position.step, provenance: { provider: 'mock', model: 'deepseek-v4-flash' }, content, ...usage === undefined ? {} : { usage }, diff --git a/packages/ui/tui/tests/snapshots/retry-cancelled.expected.txt b/packages/ui/tui/tests/snapshots/retry-cancelled.expected.txt new file mode 100644 index 0000000000..f257b1e9f6 --- /dev/null +++ b/packages/ui/tui/tests/snapshots/retry-cancelled.expected.txt @@ -0,0 +1,48 @@ +terminal 96x36 buffer=normal length=36 base=0 viewport=0 +lifecycle started=1 stopped=0 progress=inactive +title "DSH snapshot" +cursor hidden column=1 viewportRow=15 bufferRow=15 +buffer +0| "╭──────────────────────────────────────────────────────────────────────────────────────────────╮" + style 0-95 fg=bright-blue +1| "│ DEEPSEEK HARNESS │" + style 0-0 fg=bright-blue + style 2-9 fg=bright-blue bold + style 11-17 bold + style 95-95 fg=bright-blue +2| "│ Snapshot agent ready. │" + style 0-0 fg=bright-blue + style 2-22 fg=bright-black + style 95-95 fg=bright-blue +3| "│ deepseek-v4-flash • main-session │" + style 0-0 fg=bright-blue + style 2-35 dim + style 95-95 fg=bright-blue +4| "╰──────────────────────────────────────────────────────────────────────────────────────────────╯" + style 0-95 fg=bright-blue +5| +6| "▌ " + style 0-0 fg=bright-blue +7| "▌ You " + style 0-0 fg=bright-blue + style 2-4 fg=bright-blue bold +8| "▌ Start then cancel. " + style 0-0 fg=bright-blue +9| "▌ " + style 0-0 fg=bright-blue +10| +11| " Retrying model request (1/2) in 1000ms: temporary transport failure " + style 1-67 fg=yellow +12| +13| " cancelled during retry delay " + style 1-28 fg=yellow +14| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +15| " " + style 1-1 inverse +16| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +17| "/workspace/project ↑0 ↓0 idle reasoning:on tools:compact" + style 0-24 dim + style 63-95 dim +18-35| diff --git a/packages/ui/tui/tests/snapshots/retry-exhausted.expected.txt b/packages/ui/tui/tests/snapshots/retry-exhausted.expected.txt new file mode 100644 index 0000000000..b1cf16d42a --- /dev/null +++ b/packages/ui/tui/tests/snapshots/retry-exhausted.expected.txt @@ -0,0 +1,45 @@ +terminal 96x36 buffer=normal length=36 base=0 viewport=0 +lifecycle started=1 stopped=0 progress=inactive +title "DSH snapshot" +cursor hidden column=1 viewportRow=13 bufferRow=13 +buffer +0| "╭──────────────────────────────────────────────────────────────────────────────────────────────╮" + style 0-95 fg=bright-blue +1| "│ DEEPSEEK HARNESS │" + style 0-0 fg=bright-blue + style 2-9 fg=bright-blue bold + style 11-17 bold + style 95-95 fg=bright-blue +2| "│ Snapshot agent ready. │" + style 0-0 fg=bright-blue + style 2-22 fg=bright-black + style 95-95 fg=bright-blue +3| "│ deepseek-v4-flash • main-session │" + style 0-0 fg=bright-blue + style 2-35 dim + style 95-95 fg=bright-blue +4| "╰──────────────────────────────────────────────────────────────────────────────────────────────╯" + style 0-95 fg=bright-blue +5| +6| "▌ " + style 0-0 fg=bright-blue +7| "▌ You " + style 0-0 fg=bright-blue + style 2-4 fg=bright-blue bold +8| "▌ Let the bounded policy exhaust. " + style 0-0 fg=bright-blue +9| "▌ " + style 0-0 fg=bright-blue +10| +11| " provider still unavailable " + style 1-26 fg=red +12| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +13| " " + style 1-1 inverse +14| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +15| "/workspace/project ↑0 ↓0 idle reasoning:on tools:compact" + style 0-24 dim + style 63-95 dim +16-35| diff --git a/packages/ui/tui/tests/snapshots/retry-recovered.expected.txt b/packages/ui/tui/tests/snapshots/retry-recovered.expected.txt new file mode 100644 index 0000000000..7d692e4365 --- /dev/null +++ b/packages/ui/tui/tests/snapshots/retry-recovered.expected.txt @@ -0,0 +1,49 @@ +terminal 96x36 buffer=normal length=36 base=0 viewport=0 +lifecycle started=1 stopped=0 progress=inactive +title "DSH snapshot" +cursor hidden column=1 viewportRow=16 bufferRow=16 +buffer +0| "╭──────────────────────────────────────────────────────────────────────────────────────────────╮" + style 0-95 fg=bright-blue +1| "│ DEEPSEEK HARNESS │" + style 0-0 fg=bright-blue + style 2-9 fg=bright-blue bold + style 11-17 bold + style 95-95 fg=bright-blue +2| "│ Snapshot agent ready. │" + style 0-0 fg=bright-blue + style 2-22 fg=bright-black + style 95-95 fg=bright-blue +3| "│ deepseek-v4-flash • main-session │" + style 0-0 fg=bright-blue + style 2-35 dim + style 95-95 fg=bright-blue +4| "╰──────────────────────────────────────────────────────────────────────────────────────────────╯" + style 0-95 fg=bright-blue +5| +6| "▌ " + style 0-0 fg=bright-blue +7| "▌ You " + style 0-0 fg=bright-blue + style 2-4 fg=bright-blue bold +8| "▌ Recover this request. " + style 0-0 fg=bright-blue +9| "▌ " + style 0-0 fg=bright-blue +10| +11| " Retrying model request (1/2) in 500ms: provider rate limit " + style 1-58 fg=yellow +12| +13| " Assistant " + style 1-9 fg=bright-magenta bold +14| " Recovered on the next bounded attempt. " +15| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +16| " " + style 1-1 inverse +17| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +18| "/workspace/project ↑0 ↓0 idle reasoning:on tools:compact" + style 0-24 dim + style 63-95 dim +19-35| diff --git a/packages/ui/tui/tests/snapshots/retry-scheduled.expected.txt b/packages/ui/tui/tests/snapshots/retry-scheduled.expected.txt new file mode 100644 index 0000000000..5cd27db26b --- /dev/null +++ b/packages/ui/tui/tests/snapshots/retry-scheduled.expected.txt @@ -0,0 +1,45 @@ +terminal 96x36 buffer=normal length=36 base=0 viewport=0 +lifecycle started=1 stopped=0 progress=inactive +title "DSH snapshot" +cursor hidden column=1 viewportRow=13 bufferRow=13 +buffer +0| "╭──────────────────────────────────────────────────────────────────────────────────────────────╮" + style 0-95 fg=bright-blue +1| "│ DEEPSEEK HARNESS │" + style 0-0 fg=bright-blue + style 2-9 fg=bright-blue bold + style 11-17 bold + style 95-95 fg=bright-blue +2| "│ Snapshot agent ready. │" + style 0-0 fg=bright-blue + style 2-22 fg=bright-black + style 95-95 fg=bright-blue +3| "│ deepseek-v4-flash • main-session │" + style 0-0 fg=bright-blue + style 2-35 dim + style 95-95 fg=bright-blue +4| "╰──────────────────────────────────────────────────────────────────────────────────────────────╯" + style 0-95 fg=bright-blue +5| +6| "▌ " + style 0-0 fg=bright-blue +7| "▌ You " + style 0-0 fg=bright-blue + style 2-4 fg=bright-blue bold +8| "▌ Recover this request. " + style 0-0 fg=bright-blue +9| "▌ " + style 0-0 fg=bright-blue +10| +11| " Retrying model request (1/2) in 500ms: provider rate limit " + style 1-58 fg=yellow +12| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +13| " " + style 1-1 inverse +14| "────────────────────────────────────────────────────────────────────────────────────────────────" + style 0-95 dim +15| "/workspace/project ↑0 ↓0 idle reasoning:on tools:compact" + style 0-24 dim + style 63-95 dim +16-35| diff --git a/packages/ui/tui/tests/tui.snapshot.ts b/packages/ui/tui/tests/tui.snapshot.ts index 609574e758..b8afc452cb 100644 --- a/packages/ui/tui/tests/tui.snapshot.ts +++ b/packages/ui/tui/tests/tui.snapshot.ts @@ -4,6 +4,7 @@ import { fileURLToPath } from 'node:url' import { afterAll, describe, expect, it } from 'vitest' import type { Context } from 'cordis' import { CallId, type ContentBlock } from '@deepseek-ai/dsh-llm' +import type {} from '@deepseek-ai/dsh-llm-retry' import type { Session } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry, { type ToolDefinition, type ToolResultView } from '@deepseek-ai/dsh-tools' @@ -24,6 +25,10 @@ const REFRESHING = process.env.DSH_SNAPSHOT === 'refresh' const CHECKPOINTS = [ 'conversation-streaming', + 'retry-scheduled', + 'retry-recovered', + 'retry-cancelled', + 'retry-exhausted', 'code-mode-pending', 'dynamic-workflow-pending', 'cordis-tools-pending', @@ -222,6 +227,82 @@ describe('TUI terminal-state snapshots', () => { await disposeSnapshot(harness) }) + it('pins failed-stream retraction, scheduled retry, and eventual success', async () => { + const harness = await setupSnapshot() + await renderAfter(harness, () => { + appendUser(harness.session, 'Recover this request.') + harness.session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text: 'discarded partial output' }, + }) + harness.session.append('llm/retry', { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 500, + failure: { message: 'provider rate limit', code: 'RATE_LIMIT', status: 429 }, + }) + }) + await checkpoint('retry-scheduled', harness.terminal, { includeScrollback: true }) + + await renderAfter(harness, () => { + harness.session.append('assistant/message', { + turn: 1, + step: 2, + provenance: { provider: 'mock', model: 'deepseek-v4-flash' }, + content: [{ type: 'text', text: 'Recovered on the next bounded attempt.' }], + }, { surfaceOp: 'append' }) + harness.session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + }) + await checkpoint('retry-recovered', harness.terminal, { includeScrollback: true }) + await disposeSnapshot(harness) + }) + + it('pins cancellation during a scheduled retry delay', async () => { + const harness = await setupSnapshot() + await renderAfter(harness, () => { + appendUser(harness.session, 'Start then cancel.') + harness.session.append('llm/retry', { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 1_000, + failure: { message: 'temporary transport failure', code: 'TRANSPORT' }, + }) + harness.session.append('turn/end', { + turn: 1, + reason: { kind: 'aborted', reason: 'cancelled during retry delay' }, + }) + }) + await checkpoint('retry-cancelled', harness.terminal, { includeScrollback: true }) + await disposeSnapshot(harness) + }) + + it('pins terminal exhaustion after retracting a failed partial stream', async () => { + const harness = await setupSnapshot() + await renderAfter(harness, () => { + appendUser(harness.session, 'Let the bounded policy exhaust.') + harness.session.append('assistant/chunk', { + turn: 1, + step: 3, + chunk: { type: 'text-delta', index: 0, text: 'discarded terminal partial output' }, + }) + harness.session.append('turn/end', { + turn: 1, + reason: { + kind: 'error', + step: 3, + failure: { message: 'provider still unavailable', code: 'SERVER', status: 503 }, + }, + }) + }) + await checkpoint('retry-exhausted', harness.terminal, { includeScrollback: true }) + await disposeSnapshot(harness) + }) + it('pins Code Mode run_code with its production presenter', async () => { const harness = await setupSnapshot({ configureContext: configureAdvancedTools }) const call = { diff --git a/packages/ui/tui/tests/tui.spec.ts b/packages/ui/tui/tests/tui.spec.ts index 27200e0fa9..813b5dd677 100644 --- a/packages/ui/tui/tests/tui.spec.ts +++ b/packages/ui/tui/tests/tui.spec.ts @@ -7,6 +7,7 @@ import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import type { ToolDefinition } from '@deepseek-ai/dsh-tools' import UserInteractionService from '@deepseek-ai/dsh-user-interaction' +import type {} from '@deepseek-ai/dsh-llm-retry' import { createTuiChat, mountTui, @@ -244,7 +245,12 @@ describe('pi-tui chat lifecycle and transcript', () => { expect(result.terminal.output).toContain('live thought') result.terminal.send('\x12') await tick() - appendAssistant(result.session, [{ type: 'text', text: 'final live answer' }], { inputTokens: 500, outputTokens: 8 }) + appendAssistant( + result.session, + [{ type: 'text', text: 'final live answer' }], + { inputTokens: 500, outputTokens: 8 }, + { turn: 2, step: 0 }, + ) await tick() expect(result.terminal.output).toContain('Working') @@ -276,6 +282,68 @@ describe('pi-tui chat lifecycle and transcript', () => { expect(result.terminal.drainInput).toHaveBeenCalledWith(100, 20) }) + it('counts failed and recovered request usage once per step', async () => { + const result = await setup() + result.session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'usage', usage: { inputTokens: 10, outputTokens: 2 } }, + }) + result.session.append('llm/retry', { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 500, + failure: { message: 'temporary', code: 'SERVER' }, + }) + result.session.append('assistant/chunk', { + turn: 1, + step: 2, + chunk: { type: 'usage', usage: { inputTokens: 7, outputTokens: 3 } }, + }) + appendAssistant( + result.session, + [{ type: 'text', text: 'recovered' }], + { inputTokens: 7, outputTokens: 3 }, + { turn: 1, step: 2 }, + ) + await tick() + + expect(result.terminal.output).toContain('↑17 ↓5') + await dispose(result) + }) + + it('retracts a failed live stream and renders its durable retry status', async () => { + const result = await setup() + result.session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text: 'discarded partial answer' }, + }) + result.session.append('llm/retry', { + turn: 1, + step: 1, + retry: 1, + maxRetries: 2, + delayMs: 500, + failure: { message: 'rate limited', code: 'RATE_LIMIT', status: 429 }, + }) + result.session.append('llm/retry', { + turn: 1, + step: 2, + retry: 2, + maxRetries: 2, + delayMs: 1_000, + failure: { message: 'failed before chunks', code: 'SERVER', status: 503 }, + }) + await tick() + + expect(result.terminal.output).toContain('Retrying model request (1/2) in 500ms: rate limited') + expect(result.terminal.output).toContain('Retrying model request (2/2) in 1000ms: failed before chunks') + await dispose(result) + }) + it('renders the ANSI palette and every markdown/content style', async () => { const result = await setup({ config: { color: true }, @@ -453,10 +521,15 @@ describe('pi-tui chat lifecycle and transcript', () => { events.session.append('turn/end', { turn: 6, reason: { kind: 'max-tokens' } }) events.session.append('turn/end', { turn: 7, reason: { kind: 'rejected', reason: 'policy' } }) events.session.append('turn/end', { turn: 8, reason: { kind: 'interrupted' } }) + events.session.append('turn/end', { + turn: 9, + reason: { kind: 'error', step: 1, failure: { message: 'structured provider failure', code: 'SERVER' } }, + }) events.ctx.emit('agent/disposed', events.agent) await tick() expect(events.terminal.output).toContain('live failure') expect(events.terminal.output).toContain('durable failure') + expect(events.terminal.output).toContain('structured provider failure') expect(events.terminal.output).toContain('stopped') expect(events.terminal.output).toContain('output-token limit') expect(events.terminal.output).toContain('Turn rejected') diff --git a/packages/ui/tui/tsconfig.json b/packages/ui/tui/tsconfig.json index 3a09f80ad8..0c9e89e9e5 100644 --- a/packages/ui/tui/tsconfig.json +++ b/packages/ui/tui/tsconfig.json @@ -26,6 +26,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../llm/llm-retry" + }, { "path": "../../core/tools" }, diff --git a/packages/util/timeout/README.md b/packages/util/timeout/README.md index b5923a3a73..cb6f0aa558 100644 --- a/packages/util/timeout/README.md +++ b/packages/util/timeout/README.md @@ -9,13 +9,15 @@ It is a **library, not a service or plugin**: no `ctx`, registers nothing, holds ## Surface ```ts -import { clampTimeout, deadline, timeoutOf, TimeoutReason } from '@deepseek-ai/dsh-timeout' +import { clampTimeout, deadline, idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf, TimeoutReason } from '@deepseek-ai/dsh-timeout' ``` | Export | Role | |---|---| | `clampTimeout(requested, def, max, name?)` | Validate the caller's optional positive-finite hint, fill from `def`, cap at `max`. Throws (with `name`) on a non-positive/non-finite hint. | | `deadline(upstream, timeoutMs, code)` | Fuse `upstream` cancellation with a timeout into one `AbortSignal` (`AbortSignal.any`); the timeout carries a `TimeoutReason`. `[Symbol.dispose]` clears the timer. | +| `idleWatchdog(upstream, timeoutMs, code)` | Keep one stable fused signal and arm only while its guarded async-iterator `next()` is outstanding. Resolution disarms; later demand rearms; disposal clears; concurrent demand rejects. | +| `MAX_TIMER_DELAY_MS` | Largest delay Node schedules without clamping it to one millisecond (`2_147_483_647`). Timer-owning config must not exceed it. | | `timeoutOf(signal \| { reason }, code?)` | Recover the `TimeoutReason` from an aborted signal/error, else `undefined` — the timeout-vs-cancel classifier. Pass `code` to match only THIS deadline's timer (see nesting below). | | `TimeoutReason` | The internal reason (`code` + `timeoutMs`) stamped on a timeout abort. Not a public error — providers translate it into their own error/field. | @@ -44,6 +46,8 @@ The signal only *notifies* — the caller MUST attach its own termination (`d.si Pass your own `code` to `timeoutOf` so classification composes under nesting: when the `upstream` you were handed is *itself* a deadline signal (a future `tools/execute` middleware arming a per-call deadline), `AbortSignal.any` preserves the outer `TimeoutReason` if the outer timer fires first. Scoping to your `code` makes a foreign timeout read as an ordinary upstream cancel — the correct classification from your capability's view — instead of your own timeout firing when your local timer never expired. +For a streamed transport, create one `idleWatchdog`, pass its stable `signal` into the transport, and call `watchdog.next(iterator)` for each provider read. The interval must be positive, finite, and no greater than `MAX_TIMER_DELAY_MS`; Node otherwise clamps it to one millisecond. It measures only outstanding demand, so no timer runs while downstream code renders or otherwise waits before asking for the next chunk. The primitive still only notifies, so the transport must observe the stable signal; the DeepSeek and pi-ai adapters prove that timeout closes their real response body or SDK request. + ## What does NOT get a timeout Local file `read`/`write`/`edit` take no `timeoutMs`: a syscall is best-effort-abortable at most, a timeout could not force `fsync`/`rename` to stop, and adding one would be an implicit default that violates explicit-over-implicit. See [`fs/`](../../fs/README.md). @@ -61,3 +65,4 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **Notification only** — a deadline cannot stop work that ignores its signal; every capability still needs its own socket/process/task termination path. - **`timeoutMs <= 0` is internal vocabulary** — it disables the local timer only after an owning backend has resolved policy, never as a public model/plugin knob. - **The first abort reason wins classification** — when an upstream cancellation beats the local timer, this layer cannot later report that its own timeout would also have elapsed. +- **An idle watchdog is not a total deadline** — it rearms per outstanding iterator demand and deliberately excludes consumer think time. diff --git a/packages/util/timeout/src/index.ts b/packages/util/timeout/src/index.ts index 47c5d87c84..a9bd47eb08 100644 --- a/packages/util/timeout/src/index.ts +++ b/packages/util/timeout/src/index.ts @@ -21,6 +21,15 @@ export class TimeoutReason extends Error { } } +/** Largest delay Node schedules without clamping it to one millisecond. */ +export const MAX_TIMER_DELAY_MS = 2_147_483_647 + +function assertTimerDelay(timeoutMs: number, name: string): void { + if (!Number.isFinite(timeoutMs) || timeoutMs <= 0 || timeoutMs > MAX_TIMER_DELAY_MS) { + throw new Error(`${name} must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`) + } +} + /** * Validate a caller's optional timeout hint, use the backend default, then cap * it. Supplied values must be positive and finite; zero is not a public @@ -53,6 +62,20 @@ export interface Deadline { [Symbol.dispose](): void } +/** Rearmable timeout around one outstanding async-iterator demand. */ +export interface IdleWatchdog { + /** Stable signal aborted by upstream cancellation or this watchdog's timeout. */ + readonly signal: AbortSignal + /** + * Await one iterator demand while the idle timer is armed. + * @param iterator - iterator whose next value represents provider progress. + * @returns the iterator's next result. + */ + next(iterator: AsyncIterator): Promise> + /** Clear an armed timer; safe to call once at the owning stream's exit. */ + [Symbol.dispose](): void +} + /** * Fuse upstream cancellation with an identifiable timeout. `timeoutMs <= 0` is * the internal no-timer sentinel; the returned disposer clears an armed timer. @@ -74,6 +97,8 @@ export function deadline( return { signal: upstream ?? new AbortController().signal, [Symbol.dispose]() {} } } + assertTimerDelay(timeoutMs, 'deadline timeoutMs') + const timer = new AbortController() const id = setTimeout(() => { timer.abort(new TimeoutReason(code, timeoutMs)) }, timeoutMs) return { @@ -85,6 +110,57 @@ export function deadline( } } +/** + * Create a rearmable idle watchdog for an async iterator. The timer exists only + * while {@link IdleWatchdog.next} is outstanding, so consumer think time does + * not count as provider idle time. The returned signal is stable for the whole + * call and only notifies; the iterator must observe it to terminate its work. + * + * @param upstream - caller cancellation fused into the stable signal. + * @param timeoutMs - positive finite idle interval in milliseconds. + * @param code - capability-owned code carried by the timeout reason. + * @returns a stable signal, guarded next operation, and timer disposer. + */ +export function idleWatchdog( + upstream: AbortSignal | undefined, + timeoutMs: number, + code: string, +): IdleWatchdog { + assertTimerDelay(timeoutMs, 'idleWatchdog timeoutMs') + const timeout = new AbortController() + const signal = upstream === undefined + ? timeout.signal + : AbortSignal.any([upstream, timeout.signal]) + let timer: ReturnType | undefined + let outstanding = false + let disposed = false + + return { + signal, + async next(iterator: AsyncIterator): Promise> { + if (disposed) throw new Error('idleWatchdog is disposed') + if (outstanding) throw new Error('idleWatchdog next is already outstanding') + outstanding = true + timer = setTimeout(() => { + timeout.abort(new TimeoutReason(code, timeoutMs)) + }, timeoutMs) + try { + return await iterator.next() + } finally { + clearTimeout(timer) + timer = undefined + outstanding = false + } + }, + [Symbol.dispose](): void { + if (disposed) return + disposed = true + if (timer !== undefined) clearTimeout(timer) + timer = undefined + }, + } +} + /** * Recover a timeout reason from a reason-bearing object. Supplying `code` * distinguishes this deadline from a nested upstream deadline; a foreign code diff --git a/packages/util/timeout/tests/timeout.spec.ts b/packages/util/timeout/tests/timeout.spec.ts index dd4da3adde..11779c915f 100644 --- a/packages/util/timeout/tests/timeout.spec.ts +++ b/packages/util/timeout/tests/timeout.spec.ts @@ -1,5 +1,12 @@ import { afterEach, describe, expect, it, vi } from 'vitest' -import { clampTimeout, deadline, timeoutOf, TimeoutReason } from '@deepseek-ai/dsh-timeout' +import { + clampTimeout, + deadline, + idleWatchdog, + MAX_TIMER_DELAY_MS, + timeoutOf, + TimeoutReason, +} from '@deepseek-ai/dsh-timeout' describe('TimeoutReason', () => { it('is an Error carrying the code and elapsed ms', () => { @@ -67,6 +74,13 @@ describe('deadline — timeout arm', () => { expect(d.signal.aborted).toBe(false) expect(timeoutOf(d.signal)).toBeUndefined() }) + + it('rejects delays that Node would clamp to one millisecond', () => { + expect(() => deadline(undefined, MAX_TIMER_DELAY_MS + 1, 'BASH_TIMEOUT')) + .toThrow(`no greater than ${MAX_TIMER_DELAY_MS}`) + expect(() => deadline(undefined, Number.POSITIVE_INFINITY, 'BASH_TIMEOUT')) + .toThrow(`no greater than ${MAX_TIMER_DELAY_MS}`) + }) }) describe('deadline — fuse with upstream', () => { @@ -182,3 +196,74 @@ describe('deadline — nested deadlines', () => { expect(timeoutOf(inner.signal)?.code).toBe('OUTER_TIMEOUT') // but IS a timeout, unscoped }) }) + +describe('idleWatchdog', () => { + afterEach(() => { vi.useRealTimers() }) + + it('arms only while next is outstanding and rearms the same signal for later demand', async () => { + vi.useFakeTimers() + const first = Promise.withResolvers>() + const second = Promise.withResolvers>() + const iterator: AsyncIterator = { + next: vi.fn() + .mockImplementationOnce(() => first.promise) + .mockImplementationOnce(() => second.promise), + } + using watchdog = idleWatchdog(undefined, 100, 'LLM_STREAM_IDLE_TIMEOUT') + const stableSignal = watchdog.signal + + const firstNext = watchdog.next(iterator) + await vi.advanceTimersByTimeAsync(99) + expect(stableSignal.aborted).toBe(false) + first.resolve({ done: false, value: 1 }) + await expect(firstNext).resolves.toEqual({ done: false, value: 1 }) + + await vi.advanceTimersByTimeAsync(10_000) + expect(stableSignal.aborted).toBe(false) + expect(watchdog.signal).toBe(stableSignal) + + const secondNext = watchdog.next(iterator) + await vi.advanceTimersByTimeAsync(100) + expect(timeoutOf(stableSignal, 'LLM_STREAM_IDLE_TIMEOUT')).toMatchObject({ timeoutMs: 100 }) + second.reject(stableSignal.reason) + await expect(secondNext).rejects.toBe(stableSignal.reason) + }) + + it('keeps an earlier upstream abort distinct from its own timeout', async () => { + vi.useFakeTimers() + const upstream = new AbortController() + using watchdog = idleWatchdog(upstream.signal, 100, 'LLM_STREAM_IDLE_TIMEOUT') + upstream.abort('caller cancelled') + expect(watchdog.signal.aborted).toBe(true) + expect(timeoutOf(watchdog.signal, 'LLM_STREAM_IDLE_TIMEOUT')).toBeUndefined() + await vi.advanceTimersByTimeAsync(1_000) + expect(watchdog.signal.reason).toBe('caller cancelled') + }) + + it('clears an outstanding arm on disposal', async () => { + vi.useFakeTimers() + const pending = Promise.withResolvers>() + const watchdog = idleWatchdog(undefined, 100, 'LLM_STREAM_IDLE_TIMEOUT') + void watchdog.next({ next: () => pending.promise }) + watchdog[Symbol.dispose]() + await vi.advanceTimersByTimeAsync(1_000) + expect(watchdog.signal.aborted).toBe(false) + pending.resolve({ done: true, value: undefined }) + await expect(watchdog.next({ next: () => Promise.resolve({ done: true, value: undefined }) })) + .rejects.toThrow(/disposed/) + watchdog[Symbol.dispose]() + }) + + it('rejects invalid bounds and concurrent iterator demand', async () => { + expect(() => idleWatchdog(undefined, 0, 'IDLE')).toThrow(/positive finite/) + expect(() => idleWatchdog(undefined, Number.NaN, 'IDLE')).toThrow(/positive finite/) + expect(() => idleWatchdog(undefined, MAX_TIMER_DELAY_MS + 1, 'IDLE')) + .toThrow(`no greater than ${MAX_TIMER_DELAY_MS}`) + const pending = Promise.withResolvers>() + using watchdog = idleWatchdog(undefined, 100, 'IDLE') + const iterator = { next: () => pending.promise } + void watchdog.next(iterator) + await expect(watchdog.next(iterator)).rejects.toThrow(/already outstanding/) + pending.resolve({ done: true, value: undefined }) + }) +}) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index dba3435f60..96e66247c0 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -388,6 +388,9 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-llm-retry': + specifier: workspace:^ + version: link:../../llm/llm-retry '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session @@ -716,6 +719,9 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-llm-retry': + specifier: workspace:^ + version: link:../../llm/llm-retry '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session @@ -1132,6 +1138,9 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../llm + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout cordis: specifier: ^4.0.0-rc.7 version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) @@ -1151,10 +1160,56 @@ importers: '@deepseek-ai/dsh-llm-deepseek': specifier: workspace:^ version: link:../llm-deepseek + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout cordis: specifier: ^4.0.0-rc.7 version: 4.0.0-rc.7(@cordisjs/plugin-include@1.0.4)(@cordisjs/plugin-loader@1.0.0-rc.5) + packages/llm/llm-retry: + dependencies: + schemastery: + specifier: ^3.18.0 + version: 3.18.0 + devDependencies: + '@cordisjs/plugin-include': + specifier: workspace:^ + version: link:../../../vendor/include + '@cordisjs/plugin-loader': + specifier: workspace:^ + version: link:../../../vendor/loader + '@deepseek-ai/dsh-agent': + specifier: workspace:^ + version: link:../../core/agent + '@deepseek-ai/dsh-agent-loop': + specifier: workspace:^ + version: link:../../core/agent-loop + '@deepseek-ai/dsh-llm': + specifier: workspace:^ + version: link:../llm + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + '@deepseek-ai/dsh-session-persistence-jsonl': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence-jsonl + '@deepseek-ai/dsh-session-persistence-sqlite': + specifier: workspace:^ + version: link:../../session-persistence/session-persistence-sqlite + '@deepseek-ai/dsh-system-prompt': + specifier: workspace:^ + version: link:../../core/system-prompt + '@deepseek-ai/dsh-timeout': + specifier: workspace:^ + version: link:../../util/timeout + '@deepseek-ai/dsh-tools': + specifier: workspace:^ + version: link:../../core/tools + cordis: + specifier: ^4.0.0-rc.7 + version: 4.0.0-rc.7(@cordisjs/plugin-include@vendor+include)(@cordisjs/plugin-loader@vendor+loader) + packages/llm/token-meter: dependencies: schemastery: @@ -1948,6 +2003,9 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-llm-retry': + specifier: workspace:^ + version: link:../../llm/llm-retry '@deepseek-ai/dsh-permission': specifier: workspace:^ version: link:../permission @@ -2080,6 +2138,9 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-llm-retry': + specifier: workspace:^ + version: link:../../llm/llm-retry '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session @@ -2132,6 +2193,9 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm + '@deepseek-ai/dsh-llm-retry': + specifier: workspace:^ + version: link:../../llm/llm-retry '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session @@ -2528,6 +2592,9 @@ importers: '@deepseek-ai/dsh-llm-pi-ai': specifier: workspace:^ version: link:../../packages/llm/llm-pi-ai + '@deepseek-ai/dsh-llm-retry': + specifier: workspace:^ + version: link:../../packages/llm/llm-retry '@deepseek-ai/dsh-paths': specifier: workspace:^ version: link:../../packages/util/paths diff --git a/python/sdk-runtime/package.json b/python/sdk-runtime/package.json index 2d3560ec5c..c9075f26a3 100644 --- a/python/sdk-runtime/package.json +++ b/python/sdk-runtime/package.json @@ -34,6 +34,7 @@ "@deepseek-ai/dsh-token-meter": "workspace:^", "@deepseek-ai/dsh-llm-deepseek": "workspace:^", "@deepseek-ai/dsh-llm-pi-ai": "workspace:^", + "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-permission": "workspace:^", "@deepseek-ai/dsh-paths": "workspace:^", "@deepseek-ai/dsh-repeat-tool-guard": "workspace:^", diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 15780b17e0..84ce9991dc 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -34,6 +34,7 @@ export const LINK_MAP: Record = { ContinuationStop: 'core.md', GenerateOptions: 'core.md', LlmCallConfig: 'core.md', + LlmFailure: 'llm-streaming.md', LlmModelInfo: 'core.md', LlmProviderInfo: 'core.md', Message: 'core.md', diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 202f2efb6e..ccd573dd9f 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -6,6 +6,7 @@ { "doc": "docs/core-data-structures/core.md", "symbol": "AssistantProvenance", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "Message", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "MessageSourceMap", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "LlmFailure", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "FinishReasonMap", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "LlmProviderInfo", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "LlmModelInfo", "source": "packages/llm/llm/src/types.ts" }, @@ -32,6 +33,7 @@ { "doc": "docs/core-data-structures/system-prompt.md", "symbol": "ToolProviderResult", "source": "packages/core/system-prompt/src/index.ts" }, { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "StreamChunk", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "LlmFailure", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "TokenUsage", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "ContentBlockMap", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/llm-streaming.md", "symbol": "AppIdentity", "source": "packages/llm/llm/src/attribution.ts" }, diff --git a/tsconfig.build.json b/tsconfig.build.json index c056aa43db..53c3bfa2d0 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -36,6 +36,7 @@ { "path": "./packages/ui/tool-ask-user" }, { "path": "./packages/context/workspace-context" }, { "path": "./packages/core/agent-loop" }, + { "path": "./packages/llm/llm-retry" }, { "path": "./packages/examples/agent-spine-demo" }, { "path": "./packages/examples/cli-demo" }, { "path": "./packages/bash/bash" }, diff --git a/tsconfig.json b/tsconfig.json index 52337c6a71..d69c831a9c 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -47,6 +47,7 @@ { "path": "./packages/ui/tool-ask-user" }, { "path": "./packages/context/workspace-context" }, { "path": "./packages/core/agent-loop" }, + { "path": "./packages/llm/llm-retry" }, { "path": "./packages/examples/agent-spine-demo" }, { "path": "./packages/examples/cli-demo" }, { "path": "./packages/bash/bash" }, diff --git a/website/zh-CN/api/harness/events.md b/website/zh-CN/api/harness/events.md index 9cad9215fb..218ce99dba 100644 --- a/website/zh-CN/api/harness/events.md +++ b/website/zh-CN/api/harness/events.md @@ -77,7 +77,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w - `step` — the step at which the failure surfaced. - `error` — the failure, verbatim. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L311) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L312) ### agent/post-step @@ -226,12 +226,13 @@ Replace the frozen call configuration. Model-visible content must use logged cha * @param turn - the open turn number. * @param step - the failed step number. * @param error - the original model-request failure. - * @param retryAttempt - zero-based number of prior recovery retries. + * @param failure - serializable facts normalized at the final adapter boundary. + * @param priorFailures - immutable failures that already authorized another request in this consecutive sequence. * @param signal - the turn abort signal. * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. * @mode waterfall */ -'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, retryAttempt: number, signal: AbortSignal, next: () => Promise): Promise +'agent/request-error'(this: Scoped, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], signal: AbortSignal, next: () => Promise): Promise ``` Recover a model-request failure after its failed step has closed. `retry` opens a new numbered step; `fail` preserves the original request error. Call `next()` to delegate to the next recovery listener or the default. @@ -240,10 +241,11 @@ Recover a model-request failure after its failed step has closed. `retry` opens - `turn` — the open turn number. - `step` — the failed step number. - `error` — the original model-request failure. -- `retryAttempt` — zero-based number of prior recovery retries. +- `failure` — serializable facts normalized at the final adapter boundary. +- `priorFailures` — immutable failures that already authorized another request in this consecutive sequence. - `signal` — the turn abort signal. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L278) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L279) ### agent/session-prefix @@ -373,7 +375,7 @@ Override whether the turn continues. The default continues after tool calls or s - `turn` — the turn being continued or stopped. - `defaultDecision` — what the loop would do absent an override. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L288) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L289) ### agent/turn-stop @@ -397,7 +399,7 @@ Monotonic terminal-stop checkpoint after continuation and steering are folded; a - `agent` — the agent whose composed continuation outcome may be stopped. - `turn` — the turn at its terminal-stop checkpoint. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L298) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/core/agent/src/types.ts#L299) ## agent-loop/* @@ -544,7 +546,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t - `options` — the full request. A LOOP-built request arrives deep-frozen (mutation throws): its content is a pure function of the session log (the reconstructability Agent Note), so listeners read it, never rewrite it. A hand-built one-shot (compaction summarize) is the caller's own object and stays mutable here. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L43) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L44) ## session/* diff --git a/website/zh-CN/api/harness/llm.md b/website/zh-CN/api/harness/llm.md index e8e4d53a16..548e462b43 100644 --- a/website/zh-CN/api/harness/llm.md +++ b/website/zh-CN/api/harness/llm.md @@ -6,7 +6,7 @@ The abstract `llm` service: an adapter registry plus a streaming model-call surface, interceptable via the `llm/stream` waterfall. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L97) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L137) ### ctx.llm.registerAdapter(providers, adapter) @@ -29,7 +29,7 @@ Register an adapter for the given provider routes. Throws `LlmError` with code ` **Returns** the disposer that unregisters all of them. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L112) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L152) ### ctx.llm.listProviders() @@ -45,7 +45,7 @@ Describe provider routes with a registered adapter. **Returns** detached provider metadata in registration order. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L143) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L183) ### ctx.llm.listModels(provider) @@ -65,7 +65,7 @@ Discover models advertised by one registered provider. Catalog membership is adv **Returns** detached model metadata in adapter-preferred order. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L153) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L193) ### ctx.llm.stream(options) @@ -91,4 +91,4 @@ Stream one model call as raw chunks (token-level deltas). Throws `LlmError` with **Returns** the chunk stream, possibly wrapped by `llm/stream` listeners. -[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L264) +[Source](https://github.com/deepseek-harness/deepseek-harness/blob/master/packages/llm/llm/src/index.ts#L304) From 3293d56a066865faec7614f2be7cae3f707d7bbc Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 18:38:22 +0800 Subject: [PATCH 07/15] fix: clarify provider retry delay contract --- .../2026-06-21-bounded-llm-request-recovery.md | 4 ++-- docs/core-data-structures/core.md | 18 +----------------- docs/core-data-structures/llm-streaming.md | 6 ++++-- packages/cordis/tool-cordis/src/api-catalog.ts | 2 +- packages/core/agent-loop/src/loop.ts | 4 +++- .../tests/contract-regressions.spec.ts | 2 +- .../agent-loop/tests/request-recovery.spec.ts | 4 ++-- packages/llm/llm-deepseek/src/adapter.ts | 6 +++--- .../llm/llm-deepseek/tests/adapter.spec.ts | 4 ++-- packages/llm/llm-retry/README.md | 2 +- packages/llm/llm-retry/src/index.ts | 8 +++++--- packages/llm/llm-retry/tests/retry.spec.ts | 4 ++-- packages/llm/llm/src/adapter-failure.ts | 7 ++++--- packages/llm/llm/src/index.ts | 10 +++++----- packages/llm/llm/src/types.ts | 2 +- packages/llm/llm/tests/service.spec.ts | 7 ++++--- scripts/type-equiv.manifest.json | 1 - 17 files changed, 41 insertions(+), 50 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md index ad476b8b35..28de5eb97c 100644 --- a/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md +++ b/.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md @@ -29,7 +29,7 @@ interface LlmFailure { message: string code: string status?: number - retryAfterMs?: number + providerRetryAfterMs?: number requestId?: ProviderRequestId } ``` @@ -64,7 +64,7 @@ interface Config { The defaults are two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the four transient codes above. The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47). Two retries mean at most three provider requests when no other recovery policy applies. `maxTransientRetries` is a non-negative integer, delays are positive finite numbers with `initialDelayMs <= maxDelayMs`, `jitterRatio` is in `[0, 1]`, and codes are non-empty and unique. These are Cordis config fields rather than hidden constants so deployments can choose different cost and latency budgets. -For an eligible failure with budget remaining, the one-based transient retry count uses bounded exponential backoff. A valid provider `retryAfterMs` replaces exponential backoff only when it does not exceed `maxDelayMs`; a longer provider delay causes delegation instead of an earlier retry that violates the provider instruction. Local backoff multiplies by an injected random factor in `[1 - jitterRatio, 1 + jitterRatio]` and clamps the final value to `maxDelayMs`; provider delay is not jittered. +For an eligible failure with budget remaining, the one-based transient retry count uses bounded exponential backoff. A valid `providerRetryAfterMs` replaces exponential backoff only when it does not exceed `maxDelayMs`; a longer provider delay causes delegation instead of an earlier retry that violates the provider instruction. Local backoff multiplies by an injected random factor in `[1 - jitterRatio, 1 + jitterRatio]` and clamps the final value to `maxDelayMs`; provider delay is not jittered. The plugin owns a lifetime `AbortController` and tracks every active backoff callback. Each wait fuses the waterfall's turn signal with that lifetime signal. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; a captured callback whose lifetime signal aborts returns `fail` and can neither retry nor enter the rest of its captured waterfall after disposal. This makes HMR disposal quiescent even though Cordis has already captured the listener. diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index d08297a751..8db120a963 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -224,23 +224,7 @@ interface GenerateOptions { } ``` -Why a model response stopped is a merge-extensible reason: - -```ts type-equiv -/** Serializable provider-boundary facts; policy decides whether they are retryable. */ -interface LlmFailure { - /** Human-readable provider or transport failure. */ - readonly message: string - /** Stable provider-neutral machine-routing code. */ - readonly code: string - /** HTTP status observed at the provider boundary, when available. */ - readonly status?: number - /** Provider-requested delay in milliseconds, when valid and available. */ - readonly retryAfterMs?: number - /** Opaque provider-issued request identifier for diagnostics. */ - readonly requestId?: ProviderRequestId -} -``` +Why a model response stopped is a merge-extensible reason. Terminal provider failures carry the streaming contract's [`LlmFailure`](llm-streaming.md#llmfailure): ```ts type-equiv /** diff --git a/docs/core-data-structures/llm-streaming.md b/docs/core-data-structures/llm-streaming.md index 1ecaa00943..674bff9dc8 100644 --- a/docs/core-data-structures/llm-streaming.md +++ b/docs/core-data-structures/llm-streaming.md @@ -31,7 +31,9 @@ type StreamChunk = } ``` -Every thrown or in-band final-adapter failure normalizes to one serializable provider-neutral payload. `retryAfterMs` is a validated positive delay observed at the provider boundary, not a retry decision; `ProviderRequestId` is an opaque branded string for diagnostics. +## `LlmFailure` + +Every thrown or in-band final-adapter failure normalizes to one serializable provider-neutral payload. `providerRetryAfterMs` is a validated positive delay requested by the provider, not a retry decision; `ProviderRequestId` is an opaque branded string for diagnostics. ```ts type-equiv /** Serializable provider-boundary facts; policy decides whether they are retryable. */ @@ -43,7 +45,7 @@ interface LlmFailure { /** HTTP status observed at the provider boundary, when available. */ readonly status?: number /** Provider-requested delay in milliseconds, when valid and available. */ - readonly retryAfterMs?: number + readonly providerRetryAfterMs?: number /** Opaque provider-issued request identifier for diagnostics. */ readonly requestId?: ProviderRequestId } diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index e6c4cf05cb..c0b54d8444 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -1186,7 +1186,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'LlmFailure', - declaration: 'export interface LlmFailure {\n readonly message: string;\n readonly code: string;\n readonly status?: number;\n readonly retryAfterMs?: number;\n readonly requestId?: ProviderRequestId;\n}', + declaration: 'export interface LlmFailure {\n readonly message: string;\n readonly code: string;\n readonly status?: number;\n readonly providerRetryAfterMs?: number;\n readonly requestId?: ProviderRequestId;\n}', }, { name: 'LlmModelInfo', diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index 926674f866..c237e24fae 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -45,7 +45,9 @@ function finishError(finish: FinishReason): { error: RequestError; failure: LlmF const facts = finish.failure const error = new LlmError(facts.message, facts.code, { ...facts.status === undefined ? {} : { status: facts.status }, - ...facts.retryAfterMs === undefined ? {} : { retryAfterMs: facts.retryAfterMs }, + ...facts.providerRetryAfterMs === undefined + ? {} + : { providerRetryAfterMs: facts.providerRetryAfterMs }, ...facts.requestId === undefined ? {} : { requestId: facts.requestId }, }) return { error, failure: error.failure } diff --git a/packages/core/agent-loop/tests/contract-regressions.spec.ts b/packages/core/agent-loop/tests/contract-regressions.spec.ts index f79f917281..e6d269d215 100644 --- a/packages/core/agent-loop/tests/contract-regressions.spec.ts +++ b/packages/core/agent-loop/tests/contract-regressions.spec.ts @@ -929,7 +929,7 @@ describe('a finish-error stream chunk ends the turn as error, not completed', () message: 'provider 401', code: 'AUTH', status: 401, - retryAfterMs: 2_000, + providerRetryAfterMs: 2_000, requestId: ProviderRequestId('finish-request-1'), } const errorStream: StreamChunk[] = [ diff --git a/packages/core/agent-loop/tests/request-recovery.spec.ts b/packages/core/agent-loop/tests/request-recovery.spec.ts index 72688b96bc..039252a6f4 100644 --- a/packages/core/agent-loop/tests/request-recovery.spec.ts +++ b/packages/core/agent-loop/tests/request-recovery.spec.ts @@ -422,7 +422,7 @@ describe('agent post-step and request-error lifecycle', () => { it('passes structured facts beside the original Error and records them on exhaustion', async () => { const original = new LlmError('provider busy', 'RATE_LIMIT', { status: 429, - retryAfterMs: 2_000, + providerRetryAfterMs: 2_000, requestId: ProviderRequestId('req-9'), }) Object.freeze(original) @@ -448,7 +448,7 @@ describe('agent post-step and request-error lifecycle', () => { message: 'provider busy', code: 'RATE_LIMIT', status: 429, - retryAfterMs: 2_000, + providerRetryAfterMs: 2_000, requestId: ProviderRequestId('req-9'), }) expect(seenHistory).toEqual([]) diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index b34cc49087..45bc229b7d 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -42,7 +42,7 @@ export interface DeepSeekAdapterOptions { export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000 const STREAM_IDLE_TIMEOUT_CODE = 'LLM_STREAM_IDLE_TIMEOUT' -function retryAfterMs(value: string | null): number | undefined { +function providerRetryAfterMs(value: string | null): number | undefined { if (value === null) return undefined if (/^\d+$/.test(value)) { const delay = Number(value) * 1_000 @@ -188,11 +188,11 @@ export class DeepSeekAdapter extends LlmAdapter { // Only swallow error-body parsing: the HTTP status still identifies the // failure, so malformed gateway JSON must not mask it. } - const delay = retryAfterMs(response.headers.get('retry-after')) + const delay = providerRetryAfterMs(response.headers.get('retry-after')) const id = requestId(response.headers) throw new LlmError(message, httpErrorCode(response.status, providerError), { status: response.status, - ...delay === undefined ? {} : { retryAfterMs: delay }, + ...delay === undefined ? {} : { providerRetryAfterMs: delay }, ...id === undefined ? {} : { requestId: id }, }) } diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 2e23fd49f7..8cc02891a6 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -232,7 +232,7 @@ describe('DeepSeekAdapter against a mock server', () => { message: 'slow down', code: 'RATE_LIMIT', status: 429, - retryAfterMs: 2_000, + providerRetryAfterMs: 2_000, requestId: ProviderRequestId('req-429'), }) }) @@ -257,7 +257,7 @@ describe('DeepSeekAdapter against a mock server', () => { message: 'come back later', code: 'SERVER', status: 503, - retryAfterMs: 3_000, + providerRetryAfterMs: 3_000, requestId: ProviderRequestId('deepseek-503'), }, }) diff --git a/packages/llm/llm-retry/README.md b/packages/llm/llm-retry/README.md index baebc2d0b3..f84fdba09a 100644 --- a/packages/llm/llm-retry/README.md +++ b/packages/llm/llm-retry/README.md @@ -2,7 +2,7 @@ Function plugin that retries selected transient model-request failures on the agent loop's closed-step recovery seam. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered step. -The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. Delay bounds must fit Node's supported timer range. A valid provider `retryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead. +The default policy permits two retries for `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. Delay bounds must fit Node's supported timer range. A valid `providerRetryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead. Before waiting, the plugin appends a non-surface `llm/retry` event with the failure and scheduled delay. Cancellation and plugin disposal abort the wait; disposal drains the plugin's active backoffs, and a callback captured before disposal fails closed if invoked afterward. diff --git a/packages/llm/llm-retry/src/index.ts b/packages/llm/llm-retry/src/index.ts index d4c5b47b3d..4edf22d6f2 100644 --- a/packages/llm/llm-retry/src/index.ts +++ b/packages/llm/llm-retry/src/index.ts @@ -190,9 +190,11 @@ export function apply(ctx: Context, config: Config = {}, internals: RetryInterna const retry = priorTransientFailures + 1 let delayMs: number - if (failure.retryAfterMs !== undefined && Number.isFinite(failure.retryAfterMs) && failure.retryAfterMs > 0) { - if (failure.retryAfterMs > resolved.maxDelayMs) return next() - delayMs = failure.retryAfterMs + if (failure.providerRetryAfterMs !== undefined + && Number.isFinite(failure.providerRetryAfterMs) + && failure.providerRetryAfterMs > 0) { + if (failure.providerRetryAfterMs > resolved.maxDelayMs) return next() + delayMs = failure.providerRetryAfterMs } else { delayMs = localDelay(resolved, retry, random) } diff --git a/packages/llm/llm-retry/tests/retry.spec.ts b/packages/llm/llm-retry/tests/retry.spec.ts index bc10e28922..8e2f086e97 100644 --- a/packages/llm/llm-retry/tests/retry.spec.ts +++ b/packages/llm/llm-retry/tests/retry.spec.ts @@ -235,7 +235,7 @@ describe('bounded transient retry policy', () => { it('uses a bounded provider Retry-After verbatim and delegates an over-cap instruction', async () => { vi.useFakeTimers() const accepted = new ScriptedAdapter([ - new LlmError('wait', 'RATE_LIMIT', { retryAfterMs: 2_000 }), + new LlmError('wait', 'RATE_LIMIT', { providerRetryAfterMs: 2_000 }), textResponse('done'), ]) ;({ ctx: context } = await harness(accepted, { jitterRatio: 1 })) @@ -250,7 +250,7 @@ describe('bounded transient retry policy', () => { await context.fiber.dispose() const rejected = new ScriptedAdapter([ - new LlmError('wait too long', 'RATE_LIMIT', { retryAfterMs: 10_001 }), + new LlmError('wait too long', 'RATE_LIMIT', { providerRetryAfterMs: 10_001 }), ]) ;({ ctx: context } = await harness(rejected)) const rejectedAgent = context.agentLoop.create(SessionId('retry-after-rejected'), { provider: 'mock', model: 'mock' }) diff --git a/packages/llm/llm/src/adapter-failure.ts b/packages/llm/llm/src/adapter-failure.ts index b2189fdaf9..390282327d 100644 --- a/packages/llm/llm/src/adapter-failure.ts +++ b/packages/llm/llm/src/adapter-failure.ts @@ -76,18 +76,19 @@ function failureSnapshot(value: unknown): LlmFailure | undefined { const message = candidate.message const code = candidate.code const status = candidate.status - const retryAfterMs = candidate.retryAfterMs + const providerRetryAfterMs = candidate.providerRetryAfterMs const requestId = candidate.requestId if (typeof message !== 'string' || message.length === 0 || typeof code !== 'string' || code.length === 0 || (status !== undefined && (!Number.isInteger(status) || status < 100 || status > 599)) - || (retryAfterMs !== undefined && (!Number.isFinite(retryAfterMs) || retryAfterMs <= 0)) + || (providerRetryAfterMs !== undefined + && (!Number.isFinite(providerRetryAfterMs) || providerRetryAfterMs <= 0)) || (requestId !== undefined && (typeof requestId !== 'string' || requestId.length === 0))) return undefined return Object.freeze({ message, code, ...status === undefined ? {} : { status }, - ...retryAfterMs === undefined ? {} : { retryAfterMs }, + ...providerRetryAfterMs === undefined ? {} : { providerRetryAfterMs }, ...requestId === undefined ? {} : { requestId }, }) } catch (_sdkFailureGetter) { diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index dfebb059be..fb431a308c 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -50,7 +50,7 @@ export interface LlmErrorOptions extends ErrorOptions { /** Valid HTTP status observed at the provider boundary. */ status?: number /** Positive finite provider-requested delay in milliseconds. */ - retryAfterMs?: number + providerRetryAfterMs?: number /** Non-empty opaque provider request id. */ requestId?: ProviderRequestId } @@ -75,9 +75,9 @@ export class LlmError extends HarnessError { && (!Number.isInteger(options.status) || options.status < 100 || options.status > 599)) { throw new Error('LlmError status must be an integer from 100 through 599') } - if (options?.retryAfterMs !== undefined - && (!Number.isFinite(options.retryAfterMs) || options.retryAfterMs <= 0)) { - throw new Error('LlmError retryAfterMs must be a positive finite number') + if (options?.providerRetryAfterMs !== undefined + && (!Number.isFinite(options.providerRetryAfterMs) || options.providerRetryAfterMs <= 0)) { + throw new Error('LlmError providerRetryAfterMs must be a positive finite number') } if (options?.requestId !== undefined && (typeof options.requestId !== 'string' || options.requestId.length === 0)) { @@ -89,7 +89,7 @@ export class LlmError extends HarnessError { message, code, ...options?.status === undefined ? {} : { status: options.status }, - ...options?.retryAfterMs === undefined ? {} : { retryAfterMs: options.retryAfterMs }, + ...options?.providerRetryAfterMs === undefined ? {} : { providerRetryAfterMs: options.providerRetryAfterMs }, ...options?.requestId === undefined ? {} : { requestId: options.requestId }, }) } diff --git a/packages/llm/llm/src/types.ts b/packages/llm/llm/src/types.ts index f8412aee2b..bbe2e09b59 100644 --- a/packages/llm/llm/src/types.ts +++ b/packages/llm/llm/src/types.ts @@ -16,7 +16,7 @@ export interface LlmFailure { /** HTTP status observed at the provider boundary, when available. */ readonly status?: number /** Provider-requested delay in milliseconds, when valid and available. */ - readonly retryAfterMs?: number + readonly providerRetryAfterMs?: number /** Opaque provider-issued request identifier for diagnostics. */ readonly requestId?: ProviderRequestId } diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index e491ad2dc8..6994ed5e7a 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -191,7 +191,7 @@ describe('LlmService', () => { it('keeps structured provider facts beside a frozen third-party Error', async () => { const original = new LlmError('provider busy', 'RATE_LIMIT', { status: 429, - retryAfterMs: 1_500, + providerRetryAfterMs: 1_500, requestId: ProviderRequestId('req-7'), }) Object.freeze(original) @@ -212,7 +212,7 @@ describe('LlmService', () => { message: 'provider busy', code: 'RATE_LIMIT', status: 429, - retryAfterMs: 1_500, + providerRetryAfterMs: 1_500, requestId: ProviderRequestId('req-7'), }) }) @@ -747,7 +747,8 @@ describe('LlmService', () => { it('rejects non-serializable structured failure facts at construction', () => { expect(() => new LlmError('busy', 'RATE_LIMIT', { status: 42 })).toThrow(/status/) - expect(() => new LlmError('busy', 'RATE_LIMIT', { retryAfterMs: Number.NaN })).toThrow(/retryAfterMs/) + expect(() => new LlmError('busy', 'RATE_LIMIT', { providerRetryAfterMs: Number.NaN })) + .toThrow(/providerRetryAfterMs/) expect(() => new LlmError('busy', 'RATE_LIMIT', { requestId: ProviderRequestId('') })).toThrow(/requestId/) expect(() => new LlmError(1 as never, 'RATE_LIMIT')).toThrow(/message/) expect(() => new LlmError('busy', 1 as never)).toThrow(/code/) diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index ccd573dd9f..b9fa1f4874 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -6,7 +6,6 @@ { "doc": "docs/core-data-structures/core.md", "symbol": "AssistantProvenance", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "Message", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "MessageSourceMap", "source": "packages/llm/llm/src/types.ts" }, - { "doc": "docs/core-data-structures/core.md", "symbol": "LlmFailure", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "FinishReasonMap", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "LlmProviderInfo", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "LlmModelInfo", "source": "packages/llm/llm/src/types.ts" }, From 29729f83cf101be3be334a84f93e3a1cf7eaf162 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 20:59:22 +0800 Subject: [PATCH 08/15] Test the TUI through ConPTY on Windows --- examples/package.json | 3 + examples/tui-agent/tests/pty-harness.ts | 135 +++++++++++++----- .../tui-agent/tests/tui-keyless-smoke.e2e.ts | 3 +- pnpm-lock.yaml | 16 +++ pnpm-workspace.yaml | 2 + 5 files changed, 124 insertions(+), 35 deletions(-) diff --git a/examples/package.json b/examples/package.json index 53c392cd4c..f9956c498d 100644 --- a/examples/package.json +++ b/examples/package.json @@ -50,5 +50,8 @@ "@deepseek-ai/dsh-web": "workspace:*", "@deepseek-ai/dsh-web-fetch-local": "workspace:*", "@deepseek-ai/dsh-workflow-workerthread": "workspace:*" + }, + "devDependencies": { + "node-pty": "1.1.0" } } diff --git a/examples/tui-agent/tests/pty-harness.ts b/examples/tui-agent/tests/pty-harness.ts index 21d0b4c9d7..116f7cc9a1 100644 --- a/examples/tui-agent/tests/pty-harness.ts +++ b/examples/tui-agent/tests/pty-harness.ts @@ -2,9 +2,9 @@ import { spawn } from 'node:child_process' import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' -import { resolveExampleLaunch } from '@deepseek-ai/dsh-loader-smoke' +import { resolveExampleLaunch, type ExampleLaunch } from '@deepseek-ai/dsh-loader-smoke' -const PTY_DRIVER = String.raw` +const POSIX_PTY_DRIVER = String.raw` import errno, json, os, pty, select, signal, sys, time node, launch_args_json, launch_env_json, cwd, actions_json, expected_exit, timeout_seconds = sys.argv[1:] env = os.environ.copy() @@ -71,9 +71,103 @@ export interface TuiPtySmokeOptions { readonly timeoutMs?: number } +function definedEnv(env: NodeJS.ProcessEnv): Record { + return Object.fromEntries( + Object.entries(env).filter((entry): entry is [string, string] => entry[1] !== undefined), + ) +} + +async function runPosixPtySmoke( + launch: ExampleLaunch, + cwd: string, + options: TuiPtySmokeOptions, + timeoutMs: number, +): Promise { + return await new Promise((resolve, reject) => { + const child = spawn('python3', [ + '-c', + POSIX_PTY_DRIVER, + launch.command, + JSON.stringify(launch.args), + JSON.stringify(launch.env), + cwd, + JSON.stringify(options.actions ?? []), + String(options.expectedExitCode ?? 0), + String(timeoutMs / 1_000), + ], { stdio: ['ignore', 'pipe', 'pipe'] }) + let stdout = '' + let stderr = '' + child.stdout.setEncoding('utf8') + child.stdout.on('data', (chunk: string) => { stdout += chunk }) + child.stderr.setEncoding('utf8') + child.stderr.on('data', (chunk: string) => { stderr += chunk }) + const timer = setTimeout(() => { + child.kill('SIGKILL') + reject(new Error(`${options.label} PTY driver did not exit. stdout:\n${stdout}\nstderr:\n${stderr}`)) + }, timeoutMs + 5_000) + child.once('error', (error) => { clearTimeout(timer); reject(error) }) + child.once('exit', (code) => { + clearTimeout(timer) + if (code === 0) resolve(stdout) + else reject(new Error(`${options.label} PTY driver exited ${String(code)}. stdout:\n${stdout}\nstderr:\n${stderr}`)) + }) + }) +} + +async function runWindowsPtySmoke( + launch: ExampleLaunch, + cwd: string, + options: TuiPtySmokeOptions, + timeoutMs: number, +): Promise { + const pty = await import('node-pty') + return await new Promise((resolve, reject) => { + const actions = options.actions ?? [] + const expectedExitCode = options.expectedExitCode ?? 0 + let output = '' + let actionIndex = 0 + let timedOut = false + const terminal = pty.spawn(launch.command, launch.args, { + name: 'xterm-256color', + cols: 100, + rows: 30, + cwd, + env: definedEnv({ + ...process.env, + ...launch.env, + COLUMNS: '100', + LINES: '30', + }), + }) + const timer = setTimeout(() => { + timedOut = true + terminal.kill() + }, timeoutMs) + terminal.onData((chunk) => { + output += chunk + while (actionIndex < actions.length && output.includes(actions[actionIndex]!.waitFor)) { + terminal.write(actions[actionIndex]!.send) + actionIndex += 1 + } + }) + terminal.onExit(({ exitCode, signal }) => { + clearTimeout(timer) + if (timedOut) { + reject(new Error(`${options.label} PTY process did not exit before ${String(timeoutMs)}ms. output:\n${output}`)) + } else if (actionIndex !== actions.length) { + reject(new Error(`${options.label} completed ${String(actionIndex)}/${String(actions.length)} PTY actions. output:\n${output}`)) + } else if (exitCode !== expectedExitCode) { + reject(new Error(`${options.label} expected exit ${String(expectedExitCode)}, got ${String(exitCode)} (signal ${String(signal)}). output:\n${output}`)) + } else { + resolve(output) + } + }) + }) +} + /** - * Boot an example in a real pseudo-terminal, drive marker-gated input, and - * return the captured terminal bytes after the expected process exit. + * Boot an example in a real pseudo-terminal (ConPTY on Windows), drive + * marker-gated input, and return captured bytes after the expected process exit. * @param options - launch paths, environment, actions, and expected exit code. * @returns complete pseudo-terminal output. */ @@ -92,35 +186,10 @@ export async function runTuiPtySmoke(options: TuiPtySmokeOptions): Promise { - const child = spawn('python3', [ - '-c', - PTY_DRIVER, - launch.command, - JSON.stringify(launch.args), - JSON.stringify(launch.env), - cwd, - JSON.stringify(options.actions ?? []), - String(options.expectedExitCode ?? 0), - String(timeoutMs / 1_000), - ], { stdio: ['ignore', 'pipe', 'pipe'] }) - let stdout = '' - let stderr = '' - child.stdout.setEncoding('utf8') - child.stdout.on('data', (chunk: string) => { stdout += chunk }) - child.stderr.setEncoding('utf8') - child.stderr.on('data', (chunk: string) => { stderr += chunk }) - const timer = setTimeout(() => { - child.kill('SIGKILL') - reject(new Error(`${options.label} PTY driver did not exit. stdout:\n${stdout}\nstderr:\n${stderr}`)) - }, timeoutMs + 5_000) - child.once('error', (error) => { clearTimeout(timer); reject(error) }) - child.once('exit', (code) => { - clearTimeout(timer) - if (code === 0) resolve(stdout) - else reject(new Error(`${options.label} PTY driver exited ${String(code)}. stdout:\n${stdout}\nstderr:\n${stderr}`)) - }) - }) + if (process.platform === 'win32') { + return await runWindowsPtySmoke(launch, cwd, options, timeoutMs) + } + return await runPosixPtySmoke(launch, cwd, options, timeoutMs) } finally { await rm(cwd, { recursive: true, force: true }) } diff --git a/examples/tui-agent/tests/tui-keyless-smoke.e2e.ts b/examples/tui-agent/tests/tui-keyless-smoke.e2e.ts index 21be35eef5..e2fa7377c6 100644 --- a/examples/tui-agent/tests/tui-keyless-smoke.e2e.ts +++ b/examples/tui-agent/tests/tui-keyless-smoke.e2e.ts @@ -8,8 +8,7 @@ const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url)) const scriptedConfigPath = fileURLToPath(new URL('./fixtures/tui-scripted.cordis.yml', import.meta.url)) const tsconfigPath = fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)) -// The Python PTY driver imports the POSIX-only pty and termios modules. -describe.skipIf(process.platform === 'win32')('tui-agent keyless smoke (real Loader tree in a PTY)', () => { +describe('tui-agent keyless smoke (real Loader tree in a PTY)', () => { it('boots pi-tui, renders the configured banner, accepts /exit, and restores the terminal', async () => { const output = await runTuiPtySmoke({ label: 'tui-agent boot', diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index f1c3c84521..977f5afb28 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -227,6 +227,10 @@ importers: '@deepseek-ai/dsh-workflow-workerthread': specifier: workspace:* version: link:../packages/workflow/workflow-workerthread + devDependencies: + node-pty: + specifier: 1.1.0 + version: 1.1.0 packages/bash/bash: devDependencies: @@ -6179,6 +6183,9 @@ packages: neo-async@2.6.2: resolution: {integrity: sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw==} + node-addon-api@7.1.1: + resolution: {integrity: sha512-5m3bsyrjFWE1xf7nz7YXdN4udnVtXK6/Yfgn5qnahL6bCkf2yKt4k3nuTKAtT4r3IG8JNR2ncsIMdZuAzJjHQQ==} + node-addon-landlock-run-linux-arm64@0.0.0-test.0: resolution: {integrity: sha512-oJsXcC33qKl9mWYx0n9YPJ2pUAoY39PoIX0Gx4lDrSCTEvENFrEaODAsQYNY+eEGpn9YMN7E+FOftvea3/1FqQ==} engines: {node: '>=20'} @@ -6256,6 +6263,9 @@ packages: resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + node-pty@1.1.0: + resolution: {integrity: sha512-20JqtutY6JPXTUnL0ij1uad7Qe1baT46lyolh2sSENDd4sTzKZ4nmAFkeAARDKwmlLjPx6XKRlwRUxwjOy+lUg==} + non-layered-tidy-tree-layout@2.0.2: resolution: {integrity: sha512-gkXMxRzUH+PB0ax9dUN0yYF0S25BqeAYqhgMaLUFmpXLEk7Fcu8f4emJuOAY0V8kjDICxROIKsTAKsV/v355xw==} @@ -10605,6 +10615,8 @@ snapshots: neo-async@2.6.2: {} + node-addon-api@7.1.1: {} + node-addon-landlock-run-linux-arm64@0.0.0-test.0: optional: true @@ -10673,6 +10685,10 @@ snapshots: fetch-blob: 3.2.0 formdata-polyfill: 4.0.10 + node-pty@1.1.0: + dependencies: + node-addon-api: 7.1.1 + non-layered-tidy-tree-layout@2.0.2: optional: true diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 26bfaeeb0b..55947e1131 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -25,6 +25,8 @@ peerDependencyRules: allowBuilds: esbuild: true lefthook: true + # Cross-platform PTY boundary for the TUI process smoke, including ConPTY on Windows. + node-pty: true # Pulled in by @earendil-works/pi-ai (optional LLM API backend). pnpm lists # them only because they ship lifecycle scripts, but those are no-ops we don't # need, so we deny them — install still succeeds. From ad021067e0d8b6e55cbb875cbdcdb71a098492f4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 21:03:31 +0800 Subject: [PATCH 09/15] fix: preserve transport cause diagnostics --- packages/llm/llm-deepseek/src/adapter.ts | 6 +----- packages/llm/llm-deepseek/tests/adapter.spec.ts | 12 +++++++++--- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index f8a8516cae..74adafb225 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -57,10 +57,6 @@ function requestId(headers: Headers): ReturnType | und return value === null || value.length === 0 ? undefined : ProviderRequestId(value) } -function errorMessage(value: unknown): string { - return value instanceof Error ? value.message : String(value) -} - /** * Map an HTTP status to a stable LlmError code. * @param status - status of a non-2xx provider response. @@ -144,7 +140,7 @@ export class DeepSeekAdapter extends LlmAdapter { throw new LlmError('DeepSeek request aborted by caller', 'ABORTED', { cause: error }) } if (error instanceof LlmError) throw error - throw new LlmError(`DeepSeek transport failed: ${errorMessage(error)}`, 'TRANSPORT', { cause: error }) + throw new LlmError(`DeepSeek API stream from ${this.options.baseURL} failed`, 'TRANSPORT', { cause: error }) } finally { consumer.abort('DeepSeek stream consumer stopped') if (!exhausted && iterator.return !== undefined) { diff --git a/packages/llm/llm-deepseek/tests/adapter.spec.ts b/packages/llm/llm-deepseek/tests/adapter.spec.ts index 82ef7506be..0b2dae67bf 100644 --- a/packages/llm/llm-deepseek/tests/adapter.spec.ts +++ b/packages/llm/llm-deepseek/tests/adapter.spec.ts @@ -373,14 +373,20 @@ describe('DeepSeekAdapter against a mock server', () => { } }) - it('rejects with STREAM_CLOSED when the server drops mid-stream', async () => { + it('classifies an abrupt body close as TRANSPORT and retains its cause', async () => { const server = await mockServer([{ kind: 'close-early', events: ['{"choices":[{"delta":{"content":"par"}}]}'], }]) const ctx = await harness(server.url) - await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })) - .rejects.toThrow(/terminated|socket|without \[DONE\]/) + let caught: unknown + try { + await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }) + } catch (error: unknown) { + caught = error + } + expect(caught).toMatchObject({ code: 'TRANSPORT' }) + expect(errorChain(caught)).toMatch(/terminated|socket|without \[DONE\]/) }) it('aborts mid-stream via the request signal', async () => { From 7c55ec9038902fd7a3a2c531927ee93317ea5444 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 21:08:06 +0800 Subject: [PATCH 10/15] docs: condense recovery architecture contract --- docs/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture.md b/docs/architecture.md index 292a15a049..321612c814 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -111,7 +111,7 @@ Each step assembles ordered prompt sections, tool schemas, and `{{name}}` variab Tool-time context—including async `agent.inject()` notices and post-tool `additionalContexts`—settles, then follows recorded results. Steering drains before `agent/post-step`, which observes durable output, results, context, and steering before signal closure. Leftovers become queued input. Terminal `agent/turn-stop` runs after continuation and steering folding, stays authoritative through turn close and flush, and discards later steering but preserves queued prompts. -Optional pruning precedes summaries, and `dsh-compact-basic` retries context overflow only after durable surface progress; `dsh-llm-retry` applies bounded transient backoff. Their independent budgets compose on `agent/request-error`, and cancellation wins ([compaction decision](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md), [transient-recovery decision](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md)). +Pruning precedes summaries; overflow retries require durable progress. Bounded transient retries compose on `agent/request-error`; cancellation wins ([compaction](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md), [retry](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md)). ### Failure Boundaries From e9aef281a1f40fa6ab8fc7f05be0cea2ade8774d Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 21:11:38 +0800 Subject: [PATCH 11/15] Document the Windows TUI contract --- .../2026-07-20-windows-tui-support.i18n.yaml | 6 ++++ .../feature/2026-07-20-windows-tui-support.md | 33 +++++++++++++++++++ .../2026-07-20-windows-tui-support.zh.md | 33 +++++++++++++++++++ packages/ui/tui/README.md | 2 ++ 4 files changed, 74 insertions(+) create mode 100644 .agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml create mode 100644 .agents/notes/implemented/feature/2026-07-20-windows-tui-support.md create mode 100644 .agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md diff --git a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml new file mode 100644 index 0000000000..34b6fe5c07 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write +2026-07-20-windows-tui-support.md: 1771d0c8f71b5273c333b00025f8b58e0f74a868 +2026-07-20-windows-tui-support.zh.md: eb6ada8cb80c232bb92624893ddd19a575fa4bb7 diff --git a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md new file mode 100644 index 0000000000..1771d0c8f7 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md @@ -0,0 +1,33 @@ +# Agent Note: Support the TUI on Windows + +Status: implemented + +English | [中文](2026-07-20-windows-tui-support.zh.md) + +## Problem + +The full-screen TUI delegates raw input, ANSI rendering, resize events, and terminal restoration to pi-tui's `ProcessTerminal`. That dependency contains a native Windows console path, but the repository's real-process smoke used Python's POSIX-only `pty` and `termios` modules. Skipping that smoke on Windows would leave the supported product path without coverage for startup, input, interaction, failure reporting, or restoration. + +The TUI platform contract must follow the runtime shipped to users rather than the portability of one test driver. A platform exclusion is justified only when the product has an unsupported runtime dependency or a demonstrated semantic gap. + +## Decision + +[`@deepseek-ai/dsh-tui`](../../../../packages/ui/tui/README.md) supports interactive terminals on Windows as well as macOS and Linux. The product continues to use pi-tui's `ProcessTerminal`; on Windows it enables virtual-terminal input after raw mode and avoids the Unix-only `SIGWINCH` refresh. DeepSeek Harness adds no platform rejection or reduced Windows mode. + +The real Loader smoke selects a native pseudo-terminal boundary by host. macOS and Linux retain the Python POSIX PTY driver. Windows uses `node-pty` and ConPTY. Both drivers receive the same launch command, environment, terminal dimensions, marker-gated input actions, timeout, expected exit code, and output assertions, and all three smoke scenarios run on every supported platform. + +`node-pty` is a test-only dependency of the examples workspace. Its reviewed native install script is explicitly enabled in `pnpm-workspace.yaml`; production TUI packages do not acquire a new dependency or subprocess layer. + +## Alternatives considered + +- **Declare the TUI unsupported on Windows** — rejected because the pinned terminal runtime implements Windows console input explicitly and the harness has no POSIX-only production dependency. A documentation-only exclusion would discard an existing product path to accommodate a test harness gap. +- **Run the POSIX driver through MSYS, Cygwin, or WSL** — rejected because that would test a compatibility environment rather than the native Windows console path users run. +- **Use `node-pty` on every host** — rejected because the standard POSIX driver already provides the macOS and Linux boundary without another native package path. Platform-specific drivers keep ConPTY limited to the host that requires it while sharing one scenario contract. +- **Rely on renderer unit tests and semantic terminal snapshots** — rejected because fake terminals do not prove Loader boot, real raw input, process exit, or terminal restoration at the operating-system boundary. + +## Consequences + +- The Windows artifact lane executes the startup, scripted interaction, resume-failure, and restoration scenarios, and the suite has no supported-platform skip. +- The Windows process proof depends on ConPTY and a pinned `node-pty` release; changing that dependency or its allowed install script requires native-boundary review. +- The two PTY drivers can differ internally, but shared inputs and assertions keep their observable TUI contract aligned. +- Windows support remains bounded by the Node and pi-tui versions shipped by the repository; unsupported historical Windows console environments do not receive a compatibility layer. diff --git a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md new file mode 100644 index 0000000000..eb6ada8cb8 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md @@ -0,0 +1,33 @@ +# Agent Note: 在 Windows 上支持 TUI + +Status: implemented + +[English](2026-07-20-windows-tui-support.md) | 中文 + +## 问题 + +全屏 TUI 将原始输入、ANSI 渲染、终端尺寸变更事件和终端恢复委托给 pi-tui 的 `ProcessTerminal`。该依赖已实现原生 Windows 控制台路径，但仓库的真实进程冒烟测试此前使用 Python 中仅适用于 POSIX 的 `pty` 和 `termios` 模块。若在 Windows 上跳过该测试，这条受支持的产品路径便会缺少针对启动、输入、交互、失败报告和终端恢复的测试覆盖率。 + +TUI 平台契约必须以交付给用户的运行时为准，而不是取决于某个测试驱动程序的可移植性。只有产品存在不受支持的运行时依赖，或已证实存在语义缺口时，排除某个平台才有依据。 + +## 决策 + +[`@deepseek-ai/dsh-tui`](../../../../packages/ui/tui/README.md) 在 Windows、macOS 和 Linux 上均支持交互式终端。产品继续使用 pi-tui 的 `ProcessTerminal`；在 Windows 上，它会在进入原始模式后启用虚拟终端输入，并避开仅适用于 Unix 的 `SIGWINCH` 刷新。DeepSeek Harness 不增加平台拒绝逻辑，也不采用功能受限的 Windows 模式。 + +真实 Loader 冒烟测试根据宿主选择原生伪终端边界。macOS 和 Linux 继续使用 Python POSIX PTY 驱动，Windows 则使用 `node-pty` 和 ConPTY。两种驱动接收相同的启动命令、环境、终端尺寸、以标记为触发条件的输入动作、超时、预期退出码和输出断言；3 个冒烟场景都会在每个受支持平台上运行。 + +`node-pty` 是 examples 工作区仅供测试使用的依赖。该依赖经评审的原生安装脚本在 `pnpm-workspace.yaml` 中显式启用；生产 TUI 包（package）不会新增依赖或子进程层。 + +## 曾考虑的替代方案 + +- **声明 TUI 不支持 Windows**：不予采纳，因为固定版本的终端运行时已显式实现 Windows 控制台输入，且 harness 没有仅适用于 POSIX 的生产依赖。仅通过文档排除 Windows，等于为迁就测试 harness 的缺口而舍弃现有产品路径。 +- **通过 MSYS、Cygwin 或 WSL 运行 POSIX 驱动**：不予采纳，因为这会测试兼容环境，而不是用户实际运行的原生 Windows 控制台路径。 +- **在所有宿主上使用 `node-pty`**：不予采纳，因为标准 POSIX 驱动已经为 macOS 和 Linux 提供所需边界，无需增加另一条原生包路径。按平台选择驱动可将 ConPTY 限定在需要它的宿主，同时共享同一份场景契约。 +- **依赖渲染器单元测试和语义终端快照**：不予采纳，因为模拟终端无法证明 Loader 启动、真实原始输入、进程退出或操作系统边界上的终端恢复。 + +## 后果 + +- Windows 产物 lane 执行启动、脚本化交互、配置恢复失败和终端恢复场景，这套测试不会在任何受支持平台上跳过。 +- Windows 进程级验证依赖 ConPTY 和固定版本的 `node-pty`；变更该依赖或允许执行的安装脚本时，必须进行原生边界评审。 +- 两种 PTY 驱动的内部实现可以不同，但共享的输入和断言会使其可观测 TUI 契约保持一致。 +- Windows 支持范围以仓库交付的 Node 和 pi-tui 版本为界；不受支持的旧版 Windows 控制台环境不会获得兼容层。 diff --git a/packages/ui/tui/README.md b/packages/ui/tui/README.md index f44644c430..328bc3e55a 100644 --- a/packages/ui/tui/README.md +++ b/packages/ui/tui/README.md @@ -4,6 +4,8 @@ The interactive terminal front door for DeepSeek Harness agents, built on [`@ear The implemented [TUI feature Agent Note](../../../.agents/notes/implemented/feature/2026-07-17-dedicated-full-screen-tui-front-door.md) owns the front-door decision; the [terminal-state snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md) owns its verification strategy. +Interactive terminals on macOS, Linux, and Windows are supported. Windows uses pi-tui's native console VT-input handling, and the [Windows support Agent Note](../../../.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md) owns the platform decision and ConPTY process verification. + This package owns interactive terminal presentation and input only. It injects `agents`, `tools`, and `userInteraction`, then drives an agent created or resumed by app or developer code. Agent lifecycle, persistence, and the model-facing [`ask_user_question`](../tool-ask-user/README.md) tool remain separate composition entries. The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions as keyboard-driven overlays. Surface replacement events rebuild the transcript so compacted history does not reappear. From 85c658b0285015e84d8764f6fc51eefb7f6d6fe6 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 21:11:38 +0800 Subject: [PATCH 12/15] fix: drop retry dependency from pruner --- docs/module-graph.md | 21 +++++++++---------- .../compact-tool-result-prune/package.json | 2 -- pnpm-lock.yaml | 3 --- 3 files changed, 10 insertions(+), 16 deletions(-) diff --git a/docs/module-graph.md b/docs/module-graph.md index b4c7a6217d..fe774feedc 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -183,6 +183,8 @@ flowchart TD pkg_fs --> pkg_sandbox pkg_compact --> pkg_llm pkg_compact --> pkg_session + pkg_compact_tool_result_prune --> pkg_llm + pkg_compact_tool_result_prune --> pkg_session pkg_web_fetch_local --> pkg_timeout pkg_web_fetch_local --> pkg_web pkg_web_search_deepseek --> pkg_web @@ -209,6 +211,12 @@ flowchart TD pkg_skill_local --> pkg_fs pkg_skill_local --> pkg_home pkg_skill_local --> pkg_skill + pkg_compact_basic --> pkg_agent + pkg_compact_basic --> pkg_compact + pkg_compact_basic --> pkg_compact_tool_result_prune + pkg_compact_basic --> pkg_llm + pkg_compact_basic --> pkg_session + pkg_compact_basic --> pkg_token_meter pkg_spill_local --> pkg_spill pkg_hook_protocol --> pkg_bash pkg_hook_protocol --> pkg_session @@ -255,9 +263,6 @@ flowchart TD pkg_fs_sandbox --> pkg_fs_local pkg_fs_sandbox --> pkg_sandbox pkg_fs_sandbox --> pkg_sandbox_policy - pkg_compact_tool_result_prune --> pkg_llm - pkg_compact_tool_result_prune --> pkg_llm_retry - pkg_compact_tool_result_prune --> pkg_session pkg_permission --> pkg_bash pkg_permission --> pkg_sandbox pkg_permission --> pkg_sandbox_policy @@ -300,12 +305,6 @@ flowchart TD pkg_tool_skill --> pkg_llm pkg_tool_skill --> pkg_skill pkg_tool_skill --> pkg_tools - pkg_compact_basic --> pkg_agent - pkg_compact_basic --> pkg_compact - pkg_compact_basic --> pkg_compact_tool_result_prune - pkg_compact_basic --> pkg_llm - pkg_compact_basic --> pkg_session - pkg_compact_basic --> pkg_token_meter pkg_subagent --> pkg_agent pkg_subagent --> pkg_brand pkg_subagent --> pkg_llm @@ -499,6 +498,7 @@ flowchart TD | [`bash`](../packages/bash/bash) | `bash` | [`sandbox`](../packages/sandbox/sandbox) | | [`fs`](../packages/fs/fs) | `fs` | [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox) | | [`compact`](../packages/compact/compact) | `compact` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | +| [`compact-tool-result-prune`](../packages/compact/compact-tool-result-prune) | `compact` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | | [`web-fetch-local`](../packages/web/web-fetch-local) | `web` | [`timeout`](../packages/util/timeout), [`web`](../packages/web/web) | | [`web-search-deepseek`](../packages/web/web-search-deepseek) | `web` | [`web`](../packages/web/web) | | [`web-search-exa`](../packages/web/web-search-exa) | `web` | [`web`](../packages/web/web) | @@ -513,6 +513,7 @@ flowchart TD | [`fs-local`](../packages/fs/fs-local) | `fs` | [`fs`](../packages/fs/fs) | | [`fs-policy`](../packages/fs/fs-policy) | `fs` | [`fs`](../packages/fs/fs) | | [`skill-local`](../packages/skill/skill-local) | `skill` | [`fs`](../packages/fs/fs), [`home`](../packages/util/home), [`skill`](../packages/skill/skill) | +| [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`compact-tool-result-prune`](../packages/compact/compact-tool-result-prune), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`token-meter`](../packages/llm/token-meter) | | [`spill-local`](../packages/spill/spill-local) | `spill` | [`spill`](../packages/spill/spill) | | [`hook-protocol`](../packages/hooks/hook-protocol) | `hooks` | [`bash`](../packages/bash/bash), [`session`](../packages/core/session) | | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) | @@ -527,14 +528,12 @@ flowchart TD | [`tools`](../packages/core/tools) | `core` | [`agent`](../packages/core/agent), [`code-runtime`](../packages/code-runtime/code-runtime), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`user-approval`](../packages/ui/user-approval) | | [`bash-sandbox`](../packages/bash/bash-sandbox) | `bash` | [`bash`](../packages/bash/bash), [`bash-local`](../packages/bash/bash-local), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy) | | [`fs-sandbox`](../packages/fs/fs-sandbox) | `fs` | [`fs`](../packages/fs/fs), [`fs-local`](../packages/fs/fs-local), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy) | -| [`compact-tool-result-prune`](../packages/compact/compact-tool-result-prune) | `compact` | [`llm`](../packages/llm/llm), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session) | | [`permission`](../packages/ui/permission) | `ui` | [`bash`](../packages/bash/bash), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`session`](../packages/core/session), [`user-approval`](../packages/ui/user-approval) | | [`agent-loop`](../packages/core/agent-loop) | `core` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`tool-bash`](../packages/bash/tool-bash) | `bash` | [`agent`](../packages/core/agent), [`bash`](../packages/bash/bash), [`home`](../packages/util/home), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`session-persistence`](../packages/session-persistence/session-persistence), [`system-prompt`](../packages/core/system-prompt), [`tasks`](../packages/tasks/tasks), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval) | | [`tool-fs`](../packages/fs/tool-fs) | `fs` | [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm), [`sandbox`](../packages/sandbox/sandbox), [`sandbox-policy`](../packages/sandbox/sandbox-policy), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`user-approval`](../packages/ui/user-approval) | | [`tool-fs-search`](../packages/fs/tool-fs-search) | `fs` | [`bash`](../packages/bash/bash), [`llm`](../packages/llm/llm), [`retention`](../packages/util/retention), [`session`](../packages/core/session), [`spill`](../packages/spill/spill), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools) | | [`tool-skill`](../packages/skill/tool-skill) | `skill` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`skill`](../packages/skill/skill), [`tools`](../packages/core/tools) | -| [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`compact-tool-result-prune`](../packages/compact/compact-tool-result-prune), [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`token-meter`](../packages/llm/token-meter) | | [`subagent`](../packages/subagent/subagent) | `subagent` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`tools`](../packages/core/tools) | | [`tool-web`](../packages/web/tool-web) | `web` | [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt), [`tools`](../packages/core/tools), [`web`](../packages/web/web) | | [`spill-policy`](../packages/spill/spill-policy) | `spill` | [`llm`](../packages/llm/llm), [`retention`](../packages/util/retention), [`session`](../packages/core/session), [`spill`](../packages/spill/spill), [`tools`](../packages/core/tools) | diff --git a/packages/compact/compact-tool-result-prune/package.json b/packages/compact/compact-tool-result-prune/package.json index a2b27b894e..81c81eb894 100644 --- a/packages/compact/compact-tool-result-prune/package.json +++ b/packages/compact/compact-tool-result-prune/package.json @@ -23,7 +23,6 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-llm": "^0.0.1", - "@deepseek-ai/dsh-llm-retry": "^0.0.1", "@deepseek-ai/dsh-session": "^0.0.1", "cordis": "^4.0.0-rc.7" }, @@ -35,7 +34,6 @@ "@cordisjs/plugin-loader": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", - "@deepseek-ai/dsh-llm-retry": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "cordis": "^4.0.0-rc.7" } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 659dcb125b..67dccbb360 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -436,9 +436,6 @@ importers: '@deepseek-ai/dsh-llm': specifier: workspace:^ version: link:../../llm/llm - '@deepseek-ai/dsh-llm-retry': - specifier: workspace:^ - version: link:../../llm/llm-retry '@deepseek-ai/dsh-session': specifier: workspace:^ version: link:../../core/session From 02a61d3a25c32425f30a94f4fc3e6674dd79843b Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 21:16:40 +0800 Subject: [PATCH 13/15] Clarify the platform-specific PTY path --- .../feature/2026-07-20-windows-tui-support.i18n.yaml | 4 ++-- .../implemented/feature/2026-07-20-windows-tui-support.md | 2 +- .../implemented/feature/2026-07-20-windows-tui-support.zh.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml index 34b6fe5c07..4edd7b7223 100644 --- a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-20-windows-tui-support.md: 1771d0c8f71b5273c333b00025f8b58e0f74a868 -2026-07-20-windows-tui-support.zh.md: eb6ada8cb80c232bb92624893ddd19a575fa4bb7 +2026-07-20-windows-tui-support.md: 6b728486dd50faac067933ce06f883447aae821f +2026-07-20-windows-tui-support.zh.md: 2b53b05ff6231361d79b4304181dc0e6d8e24e68 diff --git a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md index 1771d0c8f7..6b728486dd 100644 --- a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md +++ b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.md @@ -22,7 +22,7 @@ The real Loader smoke selects a native pseudo-terminal boundary by host. macOS a - **Declare the TUI unsupported on Windows** — rejected because the pinned terminal runtime implements Windows console input explicitly and the harness has no POSIX-only production dependency. A documentation-only exclusion would discard an existing product path to accommodate a test harness gap. - **Run the POSIX driver through MSYS, Cygwin, or WSL** — rejected because that would test a compatibility environment rather than the native Windows console path users run. -- **Use `node-pty` on every host** — rejected because the standard POSIX driver already provides the macOS and Linux boundary without another native package path. Platform-specific drivers keep ConPTY limited to the host that requires it while sharing one scenario contract. +- **Use `node-pty` on every host** — rejected because the established POSIX driver already provides the macOS and Linux boundary; replacing it would widen the runtime change without improving those hosts. Platform-specific drivers reserve the `node-pty` runtime path for Windows while sharing one scenario contract. - **Rely on renderer unit tests and semantic terminal snapshots** — rejected because fake terminals do not prove Loader boot, real raw input, process exit, or terminal restoration at the operating-system boundary. ## Consequences diff --git a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md index eb6ada8cb8..2b53b05ff6 100644 --- a/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md +++ b/.agents/notes/implemented/feature/2026-07-20-windows-tui-support.zh.md @@ -22,7 +22,7 @@ TUI 平台契约必须以交付给用户的运行时为准，而不是取决于 - **声明 TUI 不支持 Windows**：不予采纳，因为固定版本的终端运行时已显式实现 Windows 控制台输入，且 harness 没有仅适用于 POSIX 的生产依赖。仅通过文档排除 Windows，等于为迁就测试 harness 的缺口而舍弃现有产品路径。 - **通过 MSYS、Cygwin 或 WSL 运行 POSIX 驱动**：不予采纳，因为这会测试兼容环境，而不是用户实际运行的原生 Windows 控制台路径。 -- **在所有宿主上使用 `node-pty`**：不予采纳，因为标准 POSIX 驱动已经为 macOS 和 Linux 提供所需边界，无需增加另一条原生包路径。按平台选择驱动可将 ConPTY 限定在需要它的宿主，同时共享同一份场景契约。 +- **在所有宿主上使用 `node-pty`**：不予采纳，因为现有 POSIX 驱动已经为 macOS 和 Linux 提供所需边界；替换该驱动会扩大运行时变更范围，却不会给这两个宿主带来改进。按平台选择驱动，仅在 Windows 上启用 `node-pty` 运行时路径，同时共享同一份场景契约。 - **依赖渲染器单元测试和语义终端快照**：不予采纳，因为模拟终端无法证明 Loader 启动、真实原始输入、进程退出或操作系统边界上的终端恢复。 ## 后果 From cc2e14f76e1c3919e4bb0be27c7785ca978c4ff4 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 22:11:26 +0800 Subject: [PATCH 14/15] fix: close recovery review gaps --- docs/core-data-structures/session.md | 2 +- packages/core/agent-loop/src/loop.ts | 10 +++++-- .../agent-loop/tests/request-recovery.spec.ts | 26 +++++++++++++++++++ packages/core/session/README.md | 2 +- packages/llm/llm-pi-ai/src/stream.ts | 5 +++- packages/llm/llm-pi-ai/tests/convert.spec.ts | 13 ++++++++++ packages/llm/llm/src/error.ts | 1 + packages/llm/llm/tests/service.spec.ts | 1 + packages/ui/acp/src/index.ts | 7 ++--- packages/ui/acp/tests/stream-update.spec.ts | 6 ++++- 10 files changed, 62 insertions(+), 11 deletions(-) diff --git a/docs/core-data-structures/session.md b/docs/core-data-structures/session.md index c73b750b4b..8650688222 100644 --- a/docs/core-data-structures/session.md +++ b/docs/core-data-structures/session.md @@ -430,7 +430,7 @@ declare class Session { - `context/message` → a user-role message carrying its `content` verbatim at its chronological position. Optional JSON `meta` remains in the event log and is never rendered. - `steering/message` → a user-role message carrying its content verbatim at its chronological position. -Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token usage is observed on `assistant/message.usage` (the step that produced it); an operational error's step number is on `turn/end.reason` for `kind: 'error'`, with normalized `LlmFailure` facts for a final model-request failure and message/code for other live errors. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data. +Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token accounting reads per-step `assistant/chunk { type: 'usage' }` records and treats `assistant/message.usage` as the committed-step fallback when no usage chunk exists; failed model-request attempts have no assistant message, so their usage chunk is the durable accounting record. An operational error's step number is on `turn/end.reason` for `kind: 'error'`, with normalized `LlmFailure` facts for a final model-request failure and message/code for other live errors. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers without provider+model and assistant messages without provider/model provenance instead of guessing a route for historical data. ## Live-session fork API diff --git a/packages/core/agent-loop/src/loop.ts b/packages/core/agent-loop/src/loop.ts index b67a8f9712..97a32e2f38 100644 --- a/packages/core/agent-loop/src/loop.ts +++ b/packages/core/agent-loop/src/loop.ts @@ -32,7 +32,7 @@ class TerminalModelRequestFailure extends Error { readonly requestError: RequestError, readonly failure: LlmFailure, ) { - super(requestError.message, { cause: requestError }) + super(failure.message, { cause: requestError }) this.name = 'TerminalModelRequestFailure' } } @@ -69,6 +69,12 @@ function errorData(err: RequestError): { message: string; code?: string } { return { message: errorChain(err), ...typeof err.code === 'string' ? { code: err.code } : {} } } +/** Preserve cause diagnostics, falling back to adapter-normalized prose for a hostile Error. */ +function durableFailure(err: RequestError, failure: LlmFailure): LlmFailure { + const message = errorChain(err) + return { ...failure, message: message === '' ? failure.message : message } +} + /** Map a successful max-token finish onto the turn reason; other successful finishes add nothing. */ function stepFinishReason(finish: FinishReason): TurnEndReason | undefined { switch (finish.kind) { @@ -231,7 +237,7 @@ async function runTurn( errorReported = true reason = failure === undefined ? { kind: 'error', step, ...errorData(err) } - : { kind: 'error', step, failure: { ...failure, message: errorChain(err) } } + : { kind: 'error', step, failure: durableFailure(err, failure) } try { events.emit('agent/error', turn, step, err) } catch { diff --git a/packages/core/agent-loop/tests/request-recovery.spec.ts b/packages/core/agent-loop/tests/request-recovery.spec.ts index ab82cf182e..cf87d376ef 100644 --- a/packages/core/agent-loop/tests/request-recovery.spec.ts +++ b/packages/core/agent-loop/tests/request-recovery.spec.ts @@ -3,6 +3,7 @@ import { Context } from 'cordis' import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, + HarnessError, LlmAdapter, LlmError, ProviderRequestId, @@ -419,6 +420,31 @@ describe('agent post-step and request-error lifecycle', () => { expect(seen).toBe(original) }) + it('keeps an adapter error with a hostile message accessor on the recovery path', async () => { + const original = Object.defineProperty(new HarnessError('provider failed', 'SERVER'), 'message', { + get() { throw new Error('SDK message accessor trap') }, + }) + const ctx = await harness(new SynchronousDispatchFailureAdapter(original)) + const agent = ctx.agentLoop.create(SessionId('hostile-message-recovery'), { provider: 'mock', model: 'mock' }) + let seenError: Error | undefined + let seenFailure: LlmFailure | undefined + ctx.on('agent/request-error', async (_agent, _turn, _step, error, failure, _history, _signal, next) => { + seenError = error + seenFailure = failure + return next() + }) + + send(agent) + await waitForIdle(ctx, agent) + + expect(seenError).toBe(original) + expect(seenFailure).toEqual({ message: 'LLM adapter failed', code: 'SERVER' }) + expect(agent.session.events.at(-1)).toMatchObject({ + type: 'turn/end', + data: { reason: { kind: 'error', failure: { message: 'LLM adapter failed', code: 'SERVER' } } }, + }) + }) + it('passes structured facts beside the original Error and records its cause chain on exhaustion', async () => { const original = new LlmError('provider busy', 'RATE_LIMIT', { cause: new Error('upstream connection reset'), diff --git a/packages/core/session/README.md b/packages/core/session/README.md index 79e657e3ca..28210e8f6c 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -60,7 +60,7 @@ Durable values need one accepted representation, not a check followed by a secon ### Session event vocabulary (`types.ts`) -The append-only log's event types, enumerated member by member — payloads, surface badges, provenance — in the generated [persistence log event catalog](../../../docs/persistence-catalog.md). Token usage and provider/model/replay provenance ride on `assistant/message`; an operational error's step is on `turn/end.reason` for `kind: 'error'`, with structured provider facts for a final model-request failure. +The append-only log's event types, enumerated member by member — payloads, surface badges, provenance — in the generated [persistence log event catalog](../../../docs/persistence-catalog.md). Token accounting reads per-step `assistant/chunk { type: 'usage' }` records and treats `assistant/message.usage` as the committed-step fallback when no usage chunk exists; failed model-request attempts have no assistant message. Provider/model/replay provenance rides on `assistant/message`; an operational error's step is on `turn/end.reason` for `kind: 'error'`, with structured provider facts for a final model-request failure. Merge-extensible via `SessionEventMap` — a plugin declaration-merges its own types (the compaction seam's `compact/*`, bounded recovery's non-surface `llm/retry`, the hook bridges' `hook/*`); merged members appear in the same catalog. diff --git a/packages/llm/llm-pi-ai/src/stream.ts b/packages/llm/llm-pi-ai/src/stream.ts index 2c89d1e224..37736af716 100644 --- a/packages/llm/llm-pi-ai/src/stream.ts +++ b/packages/llm/llm-pi-ai/src/stream.ts @@ -35,7 +35,10 @@ function classifyPiAiError(message: string): string { if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST' if (/\b5\d\d\b/.test(message)) return 'SERVER' if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT' - if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message)) return 'TRANSPORT' + if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message) + || /\b(?:other side closed|HTTP2 request did not get a response|WebSocket closed unexpectedly)\b/i.test(message)) { + return 'TRANSPORT' + } return 'PI_AI_ERROR' } diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index e33f0bf09a..15471875d2 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -535,6 +535,10 @@ describe('mapStopReason / mapUsage', () => { .toMatchObject({ kind: 'error', failure: { code: 'RATE_LIMIT' } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 429: insufficient_quota' }))) .toMatchObject({ kind: 'error', failure: { code: 'QUOTA' } }) + expect(mapStopReason(assistant({ + stopReason: 'error', + errorMessage: 'OpenAI API error (429): You exceeded your current quota, please check your plan and billing details.', + }))).toMatchObject({ kind: 'error', failure: { code: 'QUOTA' } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 500: backend down' }))) .toMatchObject({ kind: 'error', failure: { code: 'SERVER' } }) expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'provider timed out' }))) @@ -555,6 +559,15 @@ describe('mapStopReason / mapUsage', () => { }))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } }) }) + it.each([ + 'other side closed', + 'HTTP2 request did not get a response', + 'WebSocket closed unexpectedly', + ])('maps pi-ai transport wording %j', (errorMessage) => { + expect(mapStopReason(assistant({ stopReason: 'error', errorMessage }))) + .toMatchObject({ kind: 'error', failure: { code: 'TRANSPORT' } }) + }) + it('uses pi-ai provider-specific overflow classification without losing rate-limit exclusions', () => { expect(mapStopReason(assistant({ stopReason: 'error', diff --git a/packages/llm/llm/src/error.ts b/packages/llm/llm/src/error.ts index 8752f30b2f..758e062895 100644 --- a/packages/llm/llm/src/error.ts +++ b/packages/llm/llm/src/error.ts @@ -74,6 +74,7 @@ export function isContextWindowExceededError(detail: string): boolean { export function isQuotaExceededError(detail: string): boolean { return /\binsufficient[\s_-]+(?:quota|balance|credits?)\b/i.test(detail) || /\b(?:quota|usage[\s_-]+limit)[\s_-]+(?:exceeded|exhausted|reached)\b/i.test(detail) + || /\bexceed(?:ed|s)?[\s_-]+(?:(?:your|the)[\s_-]+)?(?:current[\s_-]+)?quota\b/i.test(detail) || /\b(?:balance|credits?)[\s_-]+(?:exhausted|depleted)\b/i.test(detail) || /\bout[\s_-]+of[\s_-]+(?:credits?|budget)\b/i.test(detail) } diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index e50a5ce615..9f90a2b7cd 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -90,6 +90,7 @@ describe('LlmService', () => { 'account balance depleted', 'usage-limit-exceeded', 'out of credits', + 'OpenAI API error (429): You exceeded your current quota, please check your plan and billing details.', ]) expect(isQuotaExceededError(detail)).toBe(true) expect(isQuotaExceededError('HTTP 429: rate limit reached')).toBe(false) expect(isQuotaExceededError('quota resets in one minute')).toBe(false) diff --git a/packages/ui/acp/src/index.ts b/packages/ui/acp/src/index.ts index 60294ddacb..38eeb3f914 100644 --- a/packages/ui/acp/src/index.ts +++ b/packages/ui/acp/src/index.ts @@ -1124,11 +1124,8 @@ export function streamSessionEventUpdate( return } case 'turn/end': { - if (event.data.reason.kind !== 'error') return - const message = 'failure' in event.data.reason - ? event.data.reason.failure.message - : event.data.reason.message - const text = `\n\n[Model attempt failed; any partial output above is discarded: ${message}]\n\n` + if (event.data.reason.kind !== 'error' || !('failure' in event.data.reason)) return + const text = `\n\n[Model attempt failed; any partial output above is discarded: ${event.data.reason.failure.message}]\n\n` notify({ sessionId, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text } } }) return } diff --git a/packages/ui/acp/tests/stream-update.spec.ts b/packages/ui/acp/tests/stream-update.spec.ts index ad71b1135e..feda449054 100644 --- a/packages/ui/acp/tests/stream-update.spec.ts +++ b/packages/ui/acp/tests/stream-update.spec.ts @@ -65,7 +65,7 @@ describe('streamSessionEventUpdate', () => { .toEqual([]) }) - it('marks retry and terminal failure boundaries in the append-only update stream', () => { + it('marks retry and terminal model failure boundaries but not ordinary turn errors', () => { expect(updatesFor(evt('llm/retry', { turn: 1, step: 1, @@ -90,6 +90,10 @@ describe('streamSessionEventUpdate', () => { text: '\n\n[Model attempt failed; any partial output above is discarded: still busy]\n\n', }, }]) + expect(updatesFor(evt('turn/end', { + turn: 1, + reason: { kind: 'error', step: 2, message: 'post-step failed' }, + }))).toEqual([]) }) it('maps tool/call to an in_progress tool_call with kind other and parsed rawInput (generic fallback, no presenter)', () => { From de72f972b71c58629fff7844986bbdedd971e860 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 22:58:01 +0800 Subject: [PATCH 15/15] Exercise normalized goal-round rate limits --- packages/goal/goal-session/tests/goal-session.spec.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/goal/goal-session/tests/goal-session.spec.ts b/packages/goal/goal-session/tests/goal-session.spec.ts index 1d41514a3c..8495b73ecc 100644 --- a/packages/goal/goal-session/tests/goal-session.spec.ts +++ b/packages/goal/goal-session/tests/goal-session.spec.ts @@ -6,7 +6,7 @@ import AgentLoop from '@deepseek-ai/dsh-agent-loop' import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit' import GoalService, { GoalId } from '@deepseek-ai/dsh-goal' import type { GoalView } from '@deepseek-ai/dsh-goal' -import { LlmAdapter } from '@deepseek-ai/dsh-llm' +import { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm' import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm' import { SessionId } from '@deepseek-ai/dsh-session' import type { TurnEndReason } from '@deepseek-ai/dsh-session' @@ -232,7 +232,7 @@ describe('same-session goal driving', () => { }) it.each([ - ['rate limit', Object.assign(new Error('slow down'), { code: 'RATE_LIMIT' }), 'usage-limited'], + ['rate limit', new LlmError('slow down', 'RATE_LIMIT'), 'usage-limited'], ['request error', new Error('provider broke'), 'turn-error'], ['max tokens', maxTokensResponse('unfinished'), 'max-tokens'], ] as const)('stops after a %s without an automatic retry', async (_label, response, code) => {