Merge branch 'master' into codex/bump-pi-ai-0.82.1
This commit is contained in:
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md
|
||||
2026-06-21-bounded-llm-request-recovery.md: 9c9d8a02595b988535158c9aa0ec43d6f1fa0c89
|
||||
2026-06-21-bounded-llm-request-recovery.zh.md: bb9e430eaf87789452fd4cc89085d7d635f54ed1
|
||||
2026-06-21-bounded-llm-request-recovery.md: 83d47e3a7d91bbcd2ceaf7b11cf13316142eb3ed
|
||||
2026-06-21-bounded-llm-request-recovery.zh.md: 00dcbad3d1023ad33a22297bfe938b94bce839d4
|
||||
@@ -4,6 +4,8 @@ Status: implemented
|
||||
|
||||
English | [中文](2026-06-21-bounded-llm-request-recovery.zh.md)
|
||||
|
||||
The [per-provider request retry policy](../feature/2026-07-24-provider-retry-policies.md) extends this foundation with exact-provider configuration and an explicit unbounded mode. This note continues to own structured failure facts, the closed-step recovery boundary, normal mode's transient defaults, visible single attempts, and durable retry status.
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-llm` can report provider failures either by throwing during adapter dispatch or iteration or by ending with `finish { kind: 'error' | 'aborted' }`. The final adapter boundary tags thrown failures so `dsh-agent-loop` can distinguish them from middleware and result-processing defects, and the loop normalizes both delivery forms into `agent/request-error` after closing the failed step. An unhandled failure is terminal; a handling listener repairs policy-owned state, returns `{ kind: 'retry' }`, and stops waterfall delegation. The [retry-action decision](../simplification/2026-07-27-request-error-retry-action.md) owns this return contract.
|
||||
@@ -16,7 +18,7 @@ The prior boundary left three narrower gaps.
|
||||
- Retry ownership differs by adapter. The hand-written DeepSeek adapter makes one attempt, while pi-ai profiles can enable opaque library retries. Combining hidden transport retries with an `agent/request-error` listener would multiply attempts and omit intermediate failures from the session log.
|
||||
- A recovered failure has no durable status fact. The failed step and chunks remain reconstructable, but an observer cannot tell whether the agent is deliberately backing off, for how long, or why. A long silent wait looks like a stalled loop.
|
||||
|
||||
The goal is bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer.
|
||||
The default policy provides bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -50,31 +52,19 @@ The shared transient-code set is intentionally small: adapter mappings for `RATE
|
||||
|
||||
`@deepseek-ai/dsh-llm-retry` is a function plugin that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow.
|
||||
|
||||
The `agent/request-error` seam carries only the current `LlmFailure`; the loop owns no retry policy or attempt history. Each recovery plugin keeps a private per-agent counter for its own handled failures and clears it at terminal `agent/settled`. Alternating transient and context-overflow failures therefore consume the `dsh-llm-retry` and compact-basic budgets independently; the maximum request count is one plus the sum of the finite budgets of the loaded recovery policies.
|
||||
The `agent/request-error` seam carries the current `LlmFailure`, an immutable list of prior failures that authorized retry turns in the consecutive recovery sequence, and the serving registration's immutable retry policy. The loop transports but does not interpret that policy, owns the consecutive failure history, and clears it after a successful model request. Normal `dsh-llm-retry` policy counts durable retry records scheduled by the same exact-provider policy, while `dsh-compact-basic` keeps its own context-overflow budget. Alternating transient and context-overflow failures therefore consume their owning finite budgets independently; the maximum request count is one plus the sum of the loaded finite budgets.
|
||||
|
||||
The plugin resolves and validates this deployment configuration at load:
|
||||
|
||||
```ts ignore-check
|
||||
interface Config {
|
||||
maxTransientRetries?: number
|
||||
initialDelayMs?: number
|
||||
maxDelayMs?: number
|
||||
jitterRatio?: number
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
The defaults are two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the five transient codes above (`RATE_LIMIT`, `SERVER`, `TIMEOUT`, `TRANSPORT`, and `EMPTY_RESPONSE`). The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47). Two retries mean at most three provider requests when no other recovery policy applies. `maxTransientRetries` is a non-negative integer, delays are positive finite numbers with `initialDelayMs <= maxDelayMs`, `jitterRatio` is in `[0, 1]`, and codes are non-empty and unique. These are Cordis config fields rather than hidden constants so deployments can choose different cost and latency budgets.
|
||||
The [provider-policy decision](../feature/2026-07-24-provider-retry-policies.md) owns the current configuration shape. Provider adapters register their nested `retryPolicy`; omission uses normal defaults: two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the five transient codes above. The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47).
|
||||
|
||||
For an eligible failure with budget remaining, the one-based transient retry count uses bounded exponential backoff. A valid `providerRetryAfterMs` replaces exponential backoff only when it does not exceed `maxDelayMs`; a longer provider delay causes delegation instead of an earlier retry that violates the provider instruction. Local backoff multiplies by an injected random factor in `[1 - jitterRatio, 1 + jitterRatio]` and clamps the final value to `maxDelayMs`; provider delay is not jittered.
|
||||
|
||||
The plugin owns a lifetime `AbortController` and tracks every active backoff callback. Each wait fuses the waterfall's turn signal with that lifetime signal. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; a captured callback whose lifetime signal aborts returns without retrying or entering the rest of its captured waterfall. This makes HMR disposal quiescent even though Cordis has already captured the listener.
|
||||
The plugin owns a lifetime `AbortController` and tracks every active recovery callback, including delegated waterfall work and backoff. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; abort wins over a late delegated retry decision, and a captured callback can neither retry nor enter the rest of its waterfall after disposal. This makes HMR disposal quiescent even though Cordis has already captured the listener.
|
||||
|
||||
Before sleeping, `dsh-llm-retry` appends one non-surface `llm/retry` session event containing the turn, failed step, one-based transient retry number, configured maximum, scheduled delay, and `LlmFailure`. The plugin owns the `SessionEventMap` augmentation; `dsh-session` remains generic persistence and does not absorb the optional policy's vocabulary. The event says what was scheduled, not that the next request completed; cancellation during the delay is subsequently visible on `turn/end`. The event ships only with a production renderer and replay/snapshot coverage, because its purpose is operational state rather than trace collection.
|
||||
Before sleeping, `dsh-llm-retry` appends one non-surface `llm/retry` session event containing the turn, failed step, provider, policy mode, complete resolved-policy key, provider-policy retry number, mode-specific finite maximum when present, scheduled delay, and `LlmFailure`. The key sorts the code set and separates retry histories when a provider route is replaced by a behaviorally different same-mode policy. The plugin owns the `SessionEventMap` augmentation; `dsh-session` remains generic persistence and does not absorb the optional policy's vocabulary. The event says what was scheduled, not that the next request completed; cancellation during the delay is subsequently visible on `turn/end`. The event ships only with a production renderer and replay/snapshot coverage, because its purpose is operational state rather than trace collection.
|
||||
|
||||
The listener calls `next()` for a non-transient code, an exhausted policy budget, or an over-cap provider delay. This preserves composition with context-overflow recovery and later policy plugins. For an owned failure it records and awaits the delay, then returns `{ kind: 'retry' }` without delegating. Turn cancellation and plugin disposal end the wait without returning a retry; the loop's cancellation/disposal checks remain authoritative.
|
||||
|
||||
The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot CLI, and ACP example compositions use the same bounded policy. Library consumers retain explicit plugin composition: omitting the plugin leaves request failures terminal.
|
||||
The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot CLI, and ACP example compositions use the same provider-routed policy. Library consumers retain explicit plugin composition: omitting the plugin leaves request failures terminal.
|
||||
|
||||
### Make one layer own visible attempts
|
||||
|
||||
@@ -101,7 +91,7 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
|
||||
- Automatic provider or model failover. Requests already select one explicit provider and model, and the provider registry deliberately has one adapter owner per provider.
|
||||
- Retrying or continuing after a successful terminal finish, or splicing chunks from two attempts into one assistant message.
|
||||
- Repairing malformed tool arguments, refusals, content filters, or other semantic model output.
|
||||
- Unbounded retries, unattended retry-until-cancelled behavior, circuit breakers, shared provider health, or cross-agent retry budgets.
|
||||
- Circuit breakers, shared provider health, or cross-agent retry budgets.
|
||||
- Changing `llm/stream` into a response lifecycle or adding convenience generation APIs without a production consumer.
|
||||
|
||||
## Alternatives considered
|
||||
@@ -110,7 +100,7 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
|
||||
- **Add response start, interrupted, discarded, failed, and committed events to `dsh-llm`** — rejected because the agent log already separates raw chunks, successful messages, and numbered attempts. A second state machine would duplicate ownership without enabling the bounded same-route retry.
|
||||
- **Add logical routes, capability matrices, and failover selection** — rejected because current requests already name provider and model explicitly, one adapter owns each provider, and no current consumer requires automatic fallback or can prove semantic compatibility.
|
||||
- **Put `retryable` or `failover` on `LlmFailure`** — rejected because adapters report facts while deployment policy decides action. The same 429 may be retried in an interactive bundle and rejected in a cost-capped batch.
|
||||
- **Retry forever while the caller remains active** — rejected because it gives one request unbounded cost and latency. Visible status makes bounded waiting understandable; it does not make an unlimited budget safe.
|
||||
- **Retry forever while the caller remains active** — the [per-provider policy](../feature/2026-07-24-provider-retry-policies.md) supersedes this rejection for explicit `always` entries while retaining bounded normal mode as the default.
|
||||
- **Log retry status only through the process logger** — rejected because process logs do not reconstruct session behavior and cannot drive replayed UI state.
|
||||
- **Keep only flat codes** — rejected because retry delay and provider request id are structured provider facts, and HTTP status is necessary for diagnosis when different wire failures share one stable code.
|
||||
|
||||
@@ -120,11 +110,11 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
|
||||
- An adapter-thrown `Error` reaches `agent/request-error` as the exact same object while its sidecar `LlmFailure` reaches the adjacent argument; tests retain the existing identity assertion for extensible and frozen third-party errors.
|
||||
- DeepSeek and pi-ai adapter tests cover representative 400, 401/403, 429, 5xx, connection, malformed/truncated stream, timeout, abort, retry-after seconds/date, request-id, and unknown-SDK-error paths without recovery policy parsing message text.
|
||||
- Pi-ai pins the SDK option to zero retries and performs one observed wire attempt for a retryable provider response; separate tests make removing either boundary fail.
|
||||
- `agent/request-error` carries only current failure facts; each plugin clears its private per-agent counter at terminal idle, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets.
|
||||
- `dsh-llm-retry` validates every config field at Loader startup, delegates all ineligible paths with `next()`, and makes at most `maxTransientRetries + 1` provider requests when no other policy applies.
|
||||
- HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, makes no retry request after disposal, and leaves no timer or promise alive.
|
||||
- `agent/request-error` carries current failure facts, immutable prior-retried failure facts, and the serving registration's immutable retry policy; a success clears the history, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets.
|
||||
- Each provider adapter validates its nested retry policy at Loader startup, and `ctx.llm` captures it with the route; normal mode delegates ineligible paths and makes at most `maxRetries + 1` provider requests when no other policy applies.
|
||||
- HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, emits no retry decision after disposal, and leaves no timer or promise alive.
|
||||
- Pure unit tests cover transient-code selection, exponential backoff and jitter bounds, valid and over-cap `Retry-After`, exhausted budgets, deterministic timer/random seams, and abort during backoff.
|
||||
- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new step, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery.
|
||||
- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new turn, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery.
|
||||
- The partial-chunk integration test proves failed chunks remain attributed to the failed step, no assistant message or tool side effect is committed for that step, and the successful retry has distinct provenance.
|
||||
- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and drives TUI retraction plus scheduled-retry rendering. Keyless snapshots cover scheduling, cancellation, success, and exhaustion; ACP automation snapshots confirm that a discarded attempt stays off the wire while the recovered reply is emitted.
|
||||
- Idle-watchdog tests prove the stable signal is rearmed only while `next()` is outstanding, disarmed during consumer think time and in `finally`, and classified separately from a total-call deadline and an earlier caller abort; adapter tests prove the signal stops the underlying request rather than merely detaching it.
|
||||
@@ -132,12 +122,12 @@ If recovery is exhausted, the final failure is stored once on `turn/end.reason`
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every transient recovery attempt is visible as a closed failed turn plus `llm/retry`, and the bounded policy prevents hidden SDK retries from multiplying cost. A retry can still duplicate provider billing even when no chunk arrived; the finite attempt budget limits but cannot remove that risk.
|
||||
- Every retry attempt is visible as a closed failed turn plus `llm/retry`, and adapter-level single-attempt behavior prevents hidden SDK retries from multiplying policy decisions. A retry can still duplicate provider billing even when no chunk arrived; normal mode limits that risk, while explicit always mode accepts it until cancellation or success.
|
||||
- Provider SDKs may hide status or retry headers. Those adapters retain the stable facts they expose and otherwise use a coarse code rather than letting recovery policy parse fragile text.
|
||||
- Durable retry events expand the session protocol and UI state machine. Shipping the event and its consumer together prevents an unused telemetry vocabulary, but later schema changes still require persistence and replay work.
|
||||
- Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots pin the transition.
|
||||
- Adapter-local idle enforcement stops stalled transports without counting consumer think time. Contract tests at each transport boundary guard against SDK drift.
|
||||
- Multiple recovery plugins add their finite budgets. Their classifiers remain disjoint here; an overlapping classifier would be registration-order policy and must be documented and tested by the plugins that introduce it.
|
||||
- Multiple normal recovery plugins add their finite budgets. Always mode delegates first and then supplies an unbounded fallback; overlapping classifiers remain registration-order policy and must be documented and tested by the plugins that introduce them.
|
||||
|
||||
## Related
|
||||
|
||||
|
||||
+16
-26
@@ -4,6 +4,8 @@ Status: implemented
|
||||
|
||||
[English](2026-06-21-bounded-llm-request-recovery.md) | 中文
|
||||
|
||||
[按提供方配置的请求重试策略](../feature/2026-07-24-provider-retry-policies.md)在此基础上增加了确切提供方配置与显式无界 mode。本说明继续负责结构化失败事实、已关闭步骤的恢复边界、normal mode 的暂时性默认值、可见的单次尝试和持久重试状态。
|
||||
|
||||
## 问题
|
||||
|
||||
`dsh-llm` 可能在适配器分发或迭代时抛出异常,也可能以 `finish { kind: 'error' | 'aborted' }` 结束,以这两种形式报告提供方失败。最终适配器边界会标记抛出的失败,使 `dsh-agent-loop` 能将其与中间件和结果处理缺陷区分开。循环关闭失败步骤后,会把两种交付形式统一规范化为 `agent/request-error`。未被处理的失败是终态;处理失败的监听器修复策略自有状态,返回 `{ kind: 'retry' }`,并停止 waterfall 委托。[重试动作决策](../simplification/2026-07-27-request-error-retry-action.md)规定这一返回契约。
|
||||
@@ -16,7 +18,7 @@ Status: implemented
|
||||
- 重试的归属因适配器而异。手写 DeepSeek 适配器只尝试一次,pi-ai profile 则可以启用库内部的不透明重试。如果把隐藏的传输重试与 `agent/request-error` 监听器结合,尝试次数会成倍增加,中间失败也不会记入会话日志。
|
||||
- 恢复后的失败没有持久状态事实。失败的步骤和分片仍可重建,但观察者无法得知 agent(智能体)是否在有意退避、将等待多久,以及等待原因。长时间的静默等待看起来与循环停滞无异。
|
||||
|
||||
本决策的目标是从同一个显式提供方/模型请求的暂时性失败中进行有界恢复。提供方或模型故障转移、响应拼接和语义输出修复都属于其他问题,目前没有消费方。
|
||||
默认策略的目标是从同一个显式提供方/模型请求的暂时性失败中进行有界恢复。提供方或模型故障转移、响应拼接和语义输出修复都属于其他问题,目前没有消费方。
|
||||
|
||||
## 决策
|
||||
|
||||
@@ -50,31 +52,19 @@ agent loop(智能体循环)会保留 `RequestError` 作为该精确的错误
|
||||
|
||||
`@deepseek-ai/dsh-llm-retry` 是监听 `agent/request-error` 的函数插件。它不引入服务或新的循环分支;agent-loop 包仅会更改通过现有失败步骤恢复控制流携带的数据。
|
||||
|
||||
`agent/request-error` seam 只携带当前 `LlmFailure`;循环不拥有重试策略或尝试历史。每个恢复插件为自身处理的失败维护一个逐 agent 的私有计数器,并在终态 `agent/settled` 时清零。因此,暂时性失败与上下文溢出交替出现时,`dsh-llm-retry` 与 compact-basic 的预算独立消耗;最大请求数等于 1 加上已加载恢复策略的有限预算总和。
|
||||
`agent/request-error` seam 携带当前 `LlmFailure`、在连续恢复序列中授权重试轮次的不可变先前失败列表,以及实际服务注册所对应的不可变重试策略。循环只传递而不解释该策略;它拥有连续失败历史,并在模型请求成功后清除。`dsh-llm-retry` 的 normal 策略统计由同一项确切提供方策略安排的持久重试记录,`dsh-compact-basic` 则维护自己的上下文溢出预算。因此,暂时性失败与上下文溢出交替出现时,会各自独立消耗其有限预算;最大请求数等于 1 加上所有已加载有限预算之和。
|
||||
|
||||
该插件在加载时解析并验证以下部署配置:
|
||||
|
||||
```ts ignore-check
|
||||
interface Config {
|
||||
maxTransientRetries?: number
|
||||
initialDelayMs?: number
|
||||
maxDelayMs?: number
|
||||
jitterRatio?: number
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
默认值为两次暂时性重试、500 毫秒初始延迟、10 秒延迟上限、10% 抖动,以及上述五个暂时性 code(`RATE_LIMIT`、`SERVER`、`TIMEOUT`、`TRANSPORT` 和 `EMPTY_RESPONSE`)。计数与延迟边界参考了所调查实现中较保守的一端:[OpenCode 使用两次请求重试,延迟边界为 500 毫秒/10 秒](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39);[Pi 将三次 agent 级重试与提供方重试分开,且提供方重试默认为零](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147);[Codex 使用有限请求/流预算以及五分钟空闲超时](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33)。10% 抖动参考 [Codex 的有界抖动](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47)。在没有其他恢复策略时,两次重试表示最多发起三次提供方请求。`maxTransientRetries` 是非负整数,延迟是正的有限数且满足 `initialDelayMs <= maxDelayMs`,`jitterRatio` 位于 `[0, 1]`,code 非空且不重复。这些都是 Cordis 配置字段,而不是隐藏常量,使部署能够选择不同的成本与延迟预算。
|
||||
当前配置形状由[提供方策略决策](../feature/2026-07-24-provider-retry-policies.md)规定。提供方适配器会注册嵌套的 `retryPolicy`;省略时使用 normal 默认值:两次暂时性重试、500 毫秒初始延迟、10 秒延迟上限、10% 抖动,以及上述五个暂时性 code。计数与延迟边界参考了所调查实现中较保守的一端:[OpenCode 使用两次请求重试,延迟边界为 500 毫秒/10 秒](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39);[Pi 将三次 agent 级重试与提供方重试分开,且提供方重试默认为零](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147);[Codex 使用有限请求/流预算以及五分钟空闲超时](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33)。10% 抖动参考 [Codex 的有界抖动](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47)。
|
||||
|
||||
对于预算未耗尽的合格失败,从 1 开始的暂时性重试计数使用有界指数退避。有效的 `providerRetryAfterMs` 只有在不超过 `maxDelayMs` 时才会取代指数退避;提供方延迟更长时,系统会委托给下一监听器,而不会违反提供方指令提前重试。本地退避乘以 `[1 - jitterRatio, 1 + jitterRatio]` 内的注入随机因子,并将最终值限制到 `maxDelayMs`;提供方延迟不加抖动。
|
||||
|
||||
插件拥有一个全生命期 `AbortController`,并跟踪每个活跃的退避回调。每次等待都会融合 waterfall(瀑布式事件)的轮次信号与该生命期信号。effect 清理会先注销监听器,再中止并等待活跃回调;被捕获回调的生命期信号中止时,回调会直接返回,不重试,也不进入其捕获 waterfall 的剩余部分。尽管 Cordis 已捕获该监听器,此设计仍能使 HMR(热模块替换)释放达到完全停稳。
|
||||
插件拥有一个全生命期 `AbortController`,并跟踪每个活跃的恢复回调,包括委托的 waterfall(瀑布式事件)工作与退避。effect 清理会先注销监听器,再中止并等待活跃回调;中止会胜过较晚到达的委托重试决策,被捕获的回调在插件释放后既不能重试,也不能进入其 waterfall 的剩余部分。尽管 Cordis 已捕获该监听器,此设计仍能使 HMR(热模块替换)释放达到完全停稳。
|
||||
|
||||
休眠前,`dsh-llm-retry` 会追加一条不进入表层的 `llm/retry` 会话事件,其中包含轮次、失败步骤、从 1 开始的暂时性重试编号、已配置上限、计划延迟和 `LlmFailure`。该插件拥有 `SessionEventMap` 声明合并;`dsh-session` 继续负责通用持久化,不会吸收可选策略的词汇。事件记录已安排的内容,而不是下一个请求已完成;延迟期间取消随后会在 `turn/end` 中可见。因为该事件的目的是表示运行状态,而不是收集跟踪数据,所以它仅与生产渲染器及回放/快照覆盖一起交付。
|
||||
休眠前,`dsh-llm-retry` 会追加一条不进入表层的 `llm/retry` 会话事件,其中包含轮次、失败步骤、提供方、策略 mode、完整的解析策略 key、提供方策略重试编号、该 mode 存在时的有限上限、计划延迟和 `LlmFailure`。该 key 会对 code 集排序,并在提供方路由被行为不同但 mode 相同的策略替换时分隔重试历史。该插件拥有 `SessionEventMap` 声明合并;`dsh-session` 继续负责通用持久化,不会吸收可选策略的词汇。事件记录已安排的内容,而不是下一个请求已完成;延迟期间取消随后会在 `turn/end` 中可见。因为该事件的目的是表示运行状态,而不是收集跟踪数据,所以它仅与生产渲染器及回放/快照覆盖一起交付。
|
||||
|
||||
对非暂时性 code、耗尽的策略预算或超出上限的提供方延迟,监听器会调用 `next()`。这保留了与上下文溢出恢复及后续策略插件的组合能力。对自身处理的失败,它会记录并等待延迟,然后在不委托的情况下返回 `{ kind: 'retry' }`。轮次取消和插件释放会结束等待且不返回重试动作,此后仍以循环的取消/释放检查为准。
|
||||
|
||||
agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次性 CLI(命令行界面)和 ACP(Agent Client Protocol)示例组合使用同一有界策略。库消费方仍需显式组合插件:省略该插件时,请求失败保持终态。
|
||||
agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次性 CLI(命令行界面)和 ACP(Agent Client Protocol)示例组合使用同一套按提供方路由的策略。库消费方仍需显式组合插件:省略该插件时,请求失败保持终态。
|
||||
|
||||
### 由单一层负责可见的尝试
|
||||
|
||||
@@ -101,7 +91,7 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
|
||||
- 自动提供方或模型故障转移。请求已显式选择一个提供方和模型,提供方注册表也有意规定每个提供方只由一个适配器负责。
|
||||
- 在成功的终止性 finish 后重试或继续,或将两次尝试的分片拼接成一条 assistant 消息。
|
||||
- 修复格式错误的工具参数、拒答、内容过滤或其他语义模型输出。
|
||||
- 无界重试、无人值守地持续重试直至取消、熔断器、共享提供方健康状态或跨 agent 重试预算。
|
||||
- 熔断器、共享提供方健康状态或跨 agent 重试预算。
|
||||
- 在没有生产消费方的情况下,把 `llm/stream` 改造成响应生命周期或增加便利的生成 API。
|
||||
|
||||
## 考虑过的替代方案
|
||||
@@ -110,7 +100,7 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
|
||||
- **向 `dsh-llm` 增加响应开始、中断、丢弃、失败和提交事件**:拒绝采用,因为 agent 日志已经分隔原始分片、成功消息和编号尝试。第二套状态机会重复归属关系,又不能支持有界的同路由重试。
|
||||
- **增加逻辑路由、能力矩阵和故障转移选择**:拒绝采用,因为当前请求已经显式指定提供方和模型,每个提供方由一个适配器负责,而且没有当前消费方要求自动回退或能够证明语义兼容性。
|
||||
- **把 `retryable` 或 `failover` 放在 `LlmFailure` 上**:拒绝采用,因为适配器报告事实,部署策略决定动作。同一个 429 可以在交互式组合包中重试,也可以在成本受限的批处理中被拒绝。
|
||||
- **只要调用方仍处于活跃状态就无限重试**:拒绝采用,因为这会让一次请求产生无界成本和延迟。可见状态能使有界等待易于理解,却不能让无限预算变得安全。
|
||||
- **只要调用方仍处于活跃状态就无限重试**:[按提供方配置的策略](../feature/2026-07-24-provider-retry-policies.md)对显式 `always` 配置项推翻了这项拒绝,同时保留有界的 normal mode 作为默认值。
|
||||
- **只通过进程 logger 记录重试状态**:拒绝采用,因为进程日志无法重建会话行为,也不能驱动回放后的 UI 状态。
|
||||
- **只保留扁平 code**:拒绝采用,因为重试延迟和提供方请求 id 是结构化的提供方事实,而当不同协议失败共用一个稳定 code 时,诊断还需要 HTTP 状态。
|
||||
|
||||
@@ -120,11 +110,11 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
|
||||
- 适配器抛出的 `Error` 会以完全相同的对象抵达 `agent/request-error`,其伴随的 `LlmFailure` 则抵达相邻参数;测试保留针对可扩展及冻结第三方错误的现有对象标识断言。
|
||||
- DeepSeek 和 pi-ai 适配器测试覆盖具有代表性的 400、401/403、429、5xx、连接、格式错误/截断流、超时、中止、Retry-After 秒数/日期、请求 id 和未知 SDK 错误路径,恢复策略无需解析消息文本。
|
||||
- Pi 将 SDK 选项固定为零次重试,并针对可重试的提供方响应执行一次可观测的实际网络请求;独立测试确保移除任一边界都会失败。
|
||||
- `agent/request-error` 只携带当前失败事实;每个插件在终态空闲时清零其逐 agent 私有计数器,暂时性失败/上下文溢出交替发生的集成测试证明两种策略只消耗各自的有限预算。
|
||||
- `dsh-llm-retry` 在 Loader 启动时验证每个配置字段,使用 `next()` 委托所有不合格路径,而且在没有其他策略时最多发起 `maxTransientRetries + 1` 次提供方请求。
|
||||
- 退避期间执行 HMR 的测试证明:释放过程会注销监听器、中止并等待其捕获的回调,释放后不发起重试请求,也不留下存活的定时器或 promise。
|
||||
- `agent/request-error` 携带当前失败事实、不可变的先前已重试失败事实,以及实际服务注册所对应的不可变重试策略;成功会清除历史,暂时性失败/上下文溢出交替发生的集成测试证明两种策略只消耗各自的有限预算。
|
||||
- 每个提供方适配器都在 Loader 启动时验证其嵌套重试策略,`ctx.llm` 则将该策略与路由一同捕获;normal mode 会委托不合格路径,而且在没有其他策略时最多发起 `maxRetries + 1` 次提供方请求。
|
||||
- 退避期间执行 HMR 的测试证明:释放过程会注销监听器、中止并等待其捕获的回调,释放后不发出重试决策,也不留下存活的定时器或 promise。
|
||||
- 纯单元测试覆盖暂时性 code 选择、指数退避和抖动边界、有效及超出上限的 `Retry-After`、耗尽的预算、确定性定时器/随机数 seam,以及退避期间中止。
|
||||
- 真实 agent-loop 测试覆盖分片前失败、部分分片后失败、抛出及带内失败、在新步骤中重试至成功、耗尽后写入结构化 `turn/end.reason`,以及与 `dsh-compact-basic` 上下文溢出恢复的组合。
|
||||
- 真实 agent-loop 测试覆盖分片前失败、部分分片后失败、抛出及带内失败、在新轮次中重试至成功、耗尽后写入结构化 `turn/end.reason`,以及与 `dsh-compact-basic` 上下文溢出恢复的组合。
|
||||
- 部分分片集成测试证明:失败分片仍归属于失败步骤,该步骤不会提交 assistant 消息或工具副作用,成功的重试具有不同的来源信息。
|
||||
- 插件拥有的不进入表层的 `llm/retry` 事件可在 JSONL 和 SQLite 往返后保留,被消息派生忽略,并驱动 TUI 撤回和计划重试渲染。无密钥快照覆盖调度、取消、成功和耗尽;ACP 自动化快照确认,被丢弃的尝试不会通过协议发出,而恢复后的回复会正常发出。
|
||||
- 空闲看门狗测试证明:只有 `next()` 尚未完成时才会重新布防稳定信号;在消费方思考期间及 `finally` 中会解除布防;它与总调用 deadline 以及更早发生的调用方中止分开分类。适配器测试证明该信号会终止底层请求,而不只是与其脱离。
|
||||
@@ -132,12 +122,12 @@ agent-spine 演示组合包加载该插件,因此共享的 stdio/TUI、一次
|
||||
|
||||
## 后果
|
||||
|
||||
- 每次暂时性恢复尝试都以一个已关闭失败轮次加 `llm/retry` 的形式可见,有界策略还会防止隐藏的 SDK 重试成倍增加成本。即使没有分片到达,重试仍可能造成提供方重复计费;有限的尝试预算只能限制而无法消除此风险。
|
||||
- 每次重试尝试都以一个已关闭失败轮次加 `llm/retry` 的形式可见,适配器级的单次尝试行为会防止隐藏的 SDK 重试成倍增加策略决策。即使没有分片到达,重试仍可能造成提供方重复计费;normal mode 会限制此风险,而显式 always mode 会接受它,直至取消或成功。
|
||||
- 提供方 SDK 可能隐藏状态或重试标头。适配器会保留 SDK 公开的稳定事实,否则使用粗粒度 code,而不会让恢复策略解析脆弱的文本。
|
||||
- 持久重试事件扩展了会话协议和 UI 状态机。事件与其消费方一同交付,可避免产生无人使用的遥测词汇;但以后更改 schema 仍需要同步完成持久化和回放工作。
|
||||
- 清除失败步骤的实时分片可能会明显撤回输出。与把丢弃的文本或不完整工具 JSON 呈现为已提交历史相比,这是更好的选择;快照固定这一转换。
|
||||
- 适配器局部的空闲强制机制可以终止停滞的传输,而不会计入消费方思考时间。每个传输边界的契约测试会防止 SDK 漂移。
|
||||
- 多个恢复插件会叠加各自的有限预算。此处它们的分类器互不重叠;重叠的分类器会形成依赖注册顺序的策略,必须由引入它们的插件记录并测试。
|
||||
- 多个 normal 恢复插件会叠加各自的有限预算。always mode 会先委托,再提供无界回退;重叠的分类器仍会形成依赖注册顺序的策略,必须由引入它们的插件记录并测试。
|
||||
|
||||
## 相关资料
|
||||
|
||||
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-14-provider-routed-llm-adapters.md: 98205d18d07752e0cdba86d7cba80368d45fd816
|
||||
2026-07-14-provider-routed-llm-adapters.zh.md: c35225a86baf4c2d09732b5940abbc8046d365fb
|
||||
2026-07-14-provider-routed-llm-adapters.md: 1bd9197667f6e49c5025c98b4a77500f78595c2b
|
||||
2026-07-14-provider-routed-llm-adapters.zh.md: 4d57f2cb33ac296500a4a19771ea493621ff93f6
|
||||
@@ -28,7 +28,7 @@ A provider has exactly one adapter owner in a Cordis context. `dsh-llm-deepseek`
|
||||
|
||||
### Explicit pi-ai provider profiles
|
||||
|
||||
`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, SDK timeouts, and a Harness stream-idle timeout. Provider retry fields are deliberately absent: the adapter forces pi-ai's `maxRetries` to zero so one `stream()` call makes one visible provider attempt, while `dsh-llm-retry` owns bounded agent-level recovery. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback.
|
||||
`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, SDK timeouts, a Harness stream-idle timeout, and a provider-owned `retryPolicy`. The adapter forces pi-ai's `maxRetries` to zero so one `stream()` call makes one visible provider attempt, while `dsh-llm-retry` executes the resolved policy at the agent failed-step seam. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback.
|
||||
|
||||
The plugin registers all configured provider names against one `PiAiAdapter` in one all-or-nothing call. A request uses its provider to select the matching profile and finds its model in `getModels(provider)` to obtain the catalog descriptor. An unknown provider fails at plugin load; an unknown model fails before network I/O with `UNKNOWN_MODEL`. The catalog object is never mutated. When a profile supplies `baseURL`, the adapter clones the selected descriptor and overrides only `baseUrl`, so a private endpoint can retain pi-ai's API, capabilities, compatibility flags, context limits, and reasoning map. The private endpoint must implement the selected provider's protocol, and the model id must still exist in the installed pi-ai catalog.
|
||||
|
||||
|
||||
+1
-1
@@ -28,7 +28,7 @@ Status: implemented
|
||||
|
||||
### 显式 pi-ai 提供方配置
|
||||
|
||||
`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一,并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称,以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、SDK 超时和 Harness 流空闲超时。配置中有意不提供重试字段:适配器强制将 pi-ai 的 `maxRetries` 设为零,使一次 `stream()` 调用只发起一次可见的提供方请求;有界的 agent 层恢复由 `dsh-llm-retry` 负责。凭据不设全局值:显式密钥仅对所属配置生效;未提供密钥时,pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置,不会回退到环境认证。
|
||||
`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一,并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称,以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、SDK 超时、Harness 流空闲超时,以及由提供方拥有的 `retryPolicy`。适配器强制将 pi-ai 的 `maxRetries` 设为零,使一次 `stream()` 调用只发起一次可见的提供方请求;`dsh-llm-retry` 则在 agent 失败步骤 seam 上执行解析后的策略。凭据不设全局值:显式密钥仅对所属配置生效;未提供密钥时,pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置,不会回退到环境认证。
|
||||
|
||||
插件通过一次全有或全无调用,将所有已配置的提供方名称注册到同一个 `PiAiAdapter`。请求按 provider 选择对应配置,并在 `getModels(provider)` 中查找模型以取得目录描述符。未知提供方会在插件加载时失败;未知模型会在网络 I/O 前以 `UNKNOWN_MODEL` 失败。适配器不会修改目录对象。当配置提供 `baseURL` 时,适配器复制选中的描述符,仅覆盖 `baseUrl`,使私有端点保留 pi-ai 的 API、能力、兼容标志、上下文限制与推理映射。私有端点必须实现所选提供方的协议,模型 ID 也仍须存在于已安装的 pi-ai 目录中。
|
||||
|
||||
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-24-empty-model-response-is-retryable.md: f4a6373178efd5ca1ba5882fb2aaf97dffb2526b
|
||||
2026-07-24-empty-model-response-is-retryable.zh.md: 4c3afe44140c029d274f34ade97803b958c6d669
|
||||
2026-07-24-empty-model-response-is-retryable.md: 3ecb106fc3a53070f66d1de351120aaf99d4d0de
|
||||
2026-07-24-empty-model-response-is-retryable.zh.md: 91ce4105ebe60b71f12667ccf28ea905566353d5
|
||||
@@ -15,7 +15,7 @@ An adapter classifies a completed empty response as a provider-boundary failure,
|
||||
- `dsh-llm` exports the canonical code `EMPTY_RESPONSE_CODE` (`'EMPTY_RESPONSE'`) beside `CONTEXT_WINDOW_EXCEEDED_CODE`/`QUOTA_EXCEEDED_CODE`.
|
||||
- `dsh-llm-pi-ai` (`mapStopReason`): a terminal `stop` whose assistant message has no content blocks becomes a `finish {kind: 'error'}` with that code. Context-overflow detection still wins where it applies (it is checked first and is the more actionable classification).
|
||||
- `dsh-llm-deepseek` (`translate`): at `[DONE]`, a `stop` (or absent) finish with no opened blocks becomes the same error finish. Reasoning-only streams count as content and stay successful.
|
||||
- `dsh-llm-retry` adds `EMPTY_RESPONSE` to `DEFAULT_RETRYABLE_CODES`: the attempt produced nothing durable, so repeating it is safe; deployments can still remove it via `retryableCodes`.
|
||||
- The provider-owned normal retry default includes `EMPTY_RESPONSE`: the attempt produced nothing durable, so repeating it is safe; deployments can still remove it via `retryableCodes`, and `dsh-llm-retry` executes the resolved policy.
|
||||
|
||||
Detection is scoped to `stop` finishes only. `max-tokens` with empty content keeps its existing meaning (pi-ai already normalizes the zero-output overflow case), `tool-calls` cannot be block-empty in practice, and error/aborted finishes already fail.
|
||||
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ Status: implemented
|
||||
- `dsh-llm` 在 `CONTEXT_WINDOW_EXCEEDED_CODE`/`QUOTA_EXCEEDED_CODE` 之外,导出规范代码 `EMPTY_RESPONSE_CODE`(`'EMPTY_RESPONSE'`)。
|
||||
- `dsh-llm-pi-ai`(`mapStopReason`):当终止性 `stop` 所对应的 assistant 消息没有内容块时,它会变成一个携带该代码的 `finish {kind: 'error'}`。上下文溢出检测在其适用场景中仍然优先(它先被检查,也是更具可操作性的归类)。
|
||||
- `dsh-llm-deepseek`(`translate`):在 `[DONE]` 处,若 `stop`(或缺失)结束且没有打开过任何块,则同样变成该错误结束。仅含 reasoning 的流算作有内容,仍视为成功。
|
||||
- `dsh-llm-retry` 把 `EMPTY_RESPONSE` 加入 `DEFAULT_RETRYABLE_CODES`:这次尝试没有产生任何持久内容,因此重复它是安全的;部署方仍可通过 `retryableCodes` 将其移除。
|
||||
- 提供方拥有的 normal 重试默认策略包含 `EMPTY_RESPONSE`:这次尝试没有产生任何持久内容,因此重复它是安全的;部署方仍可通过 `retryableCodes` 将其移除,而 `dsh-llm-retry` 会执行解析后的策略。
|
||||
|
||||
检测仅限于 `stop` 结束。内容为空的 `max-tokens` 保持其既有含义(pi-ai 已经把零输出的溢出场景归一化处理),`tool-calls` 在实践中不可能是空块,而 error/aborted 结束本身已经算失败。
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-24-provider-retry-policies.md: 1831ce6b96178d11e7c9927ceccbe07ea578cd2c
|
||||
2026-07-24-provider-retry-policies.zh.md: 788f1f1963861e1b46ff0d8798e53e01bd9892b4
|
||||
@@ -0,0 +1,65 @@
|
||||
# Agent Note: Per-provider request retry policies
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-24-provider-retry-policies.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
One process may route model requests to providers with different reliability and cost constraints. A single transient classifier and finite retry budget cannot express a deployment that wants bounded recovery for most providers but requires one provider to keep retrying every model-request failure until the request succeeds or the caller cancels it.
|
||||
|
||||
Provider policy must follow the request that actually failed, including a route selected by `agent/request`, rather than the agent's initial options. Unbounded policy also cannot store JavaScript `Infinity` in the durable session event, and neither provider error text nor discarded partial output may enter the next model request.
|
||||
|
||||
## Decision
|
||||
|
||||
Each concrete adapter accepts an optional `retryPolicy` inside its provider configuration. The adapter validates and resolves the policy, and `ctx.llm` captures it when that exact provider route registers. When a call enters its final adapter boundary, `ctx.llm` binds the serving registration's immutable policy to that call; the agent loop passes it to closed-step recovery even if the route is disposed or replaced while the request is in flight. `@deepseek-ai/dsh-llm-retry` combines that call-local policy with the failed step's durable provider identity. A call that never reaches a final adapter has no serving policy and delegates. A provider without `retryPolicy` uses the normal defaults.
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- provider: deepseek
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 2
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
- provider: internal
|
||||
retryPolicy:
|
||||
mode: always
|
||||
backoff:
|
||||
initialDelayMs: 1000
|
||||
maxDelayMs: 30000
|
||||
jitterRatio: 0.2
|
||||
```
|
||||
|
||||
The listener reads the provider from the durable `request/header` in force when the failed step closed, excluding later recovery mutations, but never re-resolves policy from the mutable provider registry. It derives a canonical key from every field of the resolved serving policy, sorting `retryableCodes` because eligibility uses set membership, and continues retry history only for the same provider and key. Replacing a route with different limits, code membership, or backoff therefore starts a new count and initial delay even when the mode is unchanged. Normal mode retains the bounded transient behavior: it retries configured codes up to `maxRetries` and otherwise delegates.
|
||||
|
||||
Always mode asks downstream recovery first so a specialized policy such as context-overflow compaction can make progress. A downstream retry wins. A downstream failure decision or thrown recovery error falls back to an unbounded retry of the same provider request; the thrown error is logged. The retry listener owns and drains delegated recovery before cancellation or plugin disposal can finish, then applies the abort instead of a late downstream decision. Success, turn cancellation, and plugin disposal are the only termination paths.
|
||||
|
||||
Both modes use exponential local delays from `initialDelayMs` to `maxDelayMs`. `jitterRatio` multiplies each target by a uniform sample in `[1 - jitterRatio, 1 + jitterRatio]`, then applies the cap. A positive provider `Retry-After` within the cap remains exact and unjittered. An over-cap provider delay makes normal mode delegate; always mode retains its guarantee by using the configured local backoff.
|
||||
|
||||
Each scheduled retry appends a non-surface `llm/retry` event with the failed provider, policy mode, canonical resolved-policy key, provider-policy retry number, delay, and failure facts. Normal events carry finite `maxRetries`; always events omit it, and UIs render the limit as `∞`. The event and failed `assistant/chunk` records do not contribute surface messages, so the next request contains the same derived context as the failed request unless another recovery policy deliberately changes the surface.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**One global `always` switch** — rejected because it cannot isolate the unbounded cost and latency risk to the provider that needs it and can silently apply after runtime rerouting.
|
||||
|
||||
**A separate exact-provider list on `dsh-llm-retry`** — rejected because it duplicates provider route names outside their owning adapter configuration and lets provider registration drift from recovery policy.
|
||||
|
||||
**A very large finite retry count** — rejected because it eventually violates the requested keep-retrying contract and serializes an arbitrary operational limit as if it were meaningful.
|
||||
|
||||
**Provider-SDK retries** — rejected because hidden attempts multiply agent-level budgets, cannot use the closed-step durability boundary, and may splice or discard streamed output without a reconstructable retry record.
|
||||
|
||||
**Put the error into model context** — rejected because a transport or provider diagnostic is operational state, not conversation content. It can expose sensitive provider details and changes the retried request instead of repeating the failed request.
|
||||
|
||||
## Verification
|
||||
|
||||
Adapter tests validate nested policies at provider load, prove registration captures configured and default policies, and retain the serving policy across in-flight route replacement. Unit tests select policies from the failed request's serving registration, separate provider and changed-policy histories, exercise always mode beyond the normal budget, pin jitter and delay caps, prove downstream recovery ordering, prove cancellation and disposal drain delegated recovery before reaching quiescence, and prove both abort active backoff waits. Request-level coverage compares the complete messages of failed and retried attempts and rejects both provider error text and discarded partial output. A keyless headless `stream-json` snapshot runs failure, retry, and success through the assembled app, pins the complete `llm/retry` record, and rejects any model-message change between attempts. JSONL and SQLite tests round-trip an always event without `Infinity`; invariant tests bind provider identity to the request header, validate failure and mode-specific timer bounds, and bind retry numbers to provider-policy keys; TUI tests render finite and infinite limits.
|
||||
|
||||
## Consequences
|
||||
|
||||
Normal mode remains a finite default, while an explicit always policy can spend unbounded requests and time on permanent authentication, quota, invalid-request, protocol, or context failures. Operators must pair always mode with a cancellable caller and provider-specific cost controls. Retry state stays observable and durable without becoming model-visible, and serving-registration capture prevents adapter lifecycle changes from retroactively changing an in-flight request's recovery contract.
|
||||
|
||||
This decision extends the closed-step recovery, single visible adapter attempt, structured failure, and durable status design in [bounded recovery for transient LLM request failures](../architecture/2026-06-21-bounded-llm-request-recovery.md).
|
||||
@@ -0,0 +1,65 @@
|
||||
# Agent Note: 逐提供方请求重试策略
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-24-provider-retry-policies.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
同一进程可能把模型请求路由到可靠性和成本约束各不相同的提供方。单一的瞬态错误分类器与有限重试预算无法表达这种部署需求:大多数提供方只需有界恢复,但其中一个提供方必须持续重试每次模型请求失败,直到请求成功或调用方取消。
|
||||
|
||||
提供方策略必须跟随实际失败的请求,包括 `agent/request` 选择的路由,而不能跟随 agent(智能体)的初始选项。无界策略也不能把 JavaScript `Infinity` 存入持久会话事件;提供方错误文本与丢弃的部分输出都不得进入下一次模型请求。
|
||||
|
||||
## 决策
|
||||
|
||||
每个具体适配器都在其提供方配置中接受可选的 `retryPolicy`。适配器负责校验并解析策略,`ctx.llm` 则在该精确提供方路由注册时捕获策略。当调用进入最终适配器边界时,`ctx.llm` 会把实际提供服务的注册项所持不可变策略绑定到该调用;即使路由在请求进行期间被 dispose 或替换,agent loop 仍会把该策略传给已关闭步骤恢复。`@deepseek-ai/dsh-llm-retry` 会把绑定到该调用的策略与失败步骤的持久提供方标识结合起来。未到达最终适配器的调用没有实际提供服务的策略,因而会委托后续处理。未配置 `retryPolicy` 的提供方使用 normal 默认值。
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- provider: deepseek
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 2
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
- provider: internal
|
||||
retryPolicy:
|
||||
mode: always
|
||||
backoff:
|
||||
initialDelayMs: 1000
|
||||
maxDelayMs: 30000
|
||||
jitterRatio: 0.2
|
||||
```
|
||||
|
||||
监听器从失败步骤关闭时生效的持久 `request/header` 读取提供方,后续恢复产生的改动不参与选择,但绝不会从可变的提供方注册表重新解析策略。它会根据已解析实际服务策略的所有字段生成规范键;由于错误资格按集合成员判断,生成时会对 `retryableCodes` 排序。重试历史只会对同一提供方和同一规范键延续。因此,即使模式未变,只要路由替换后的次数上限、错误代码成员或退避不同,重试计数与初始延迟都会重新开始。normal 模式保留有界瞬态错误处理行为:它重试配置的错误代码,次数不超过 `maxRetries`;其他情况委托后续处理。
|
||||
|
||||
always 模式先请求下游恢复,使上下文溢出压缩(compaction)之类的专用策略有机会取得进展。下游若决定重试,则以该决定为准。下游若决定失败或恢复过程抛出错误,则回退为无界重试同一提供方请求;抛出的错误会写入日志。重试监听器会持有并排空已委托的恢复,轮次取消或插件 dispose(资源释放)只能在其结束后完成;随后监听器会应用取消,而不会采用迟到的下游决定。成功、轮次取消和插件 dispose 是仅有的终止路径。
|
||||
|
||||
两种模式的本地延迟都按指数增长,从 `initialDelayMs` 增至 `maxDelayMs`。`jitterRatio` 用 `[1 - jitterRatio, 1 + jitterRatio]` 区间内的均匀随机样本乘以每次目标值,再应用上限。提供方给出的正数 `Retry-After` 若未超过上限,则保持精确且不加抖动。若提供方延迟超过上限,normal 模式会委托后续处理;always 模式则改用配置的本地退避,以维持无限重试保证。
|
||||
|
||||
每次安排重试都会追加一条不进入表层的 `llm/retry` 事件,其中包含失败的提供方、策略模式、已解析策略的规范键、提供方策略内的重试编号、延迟和失败事实。normal 事件包含有限的 `maxRetries`;always 事件省略该字段,UI 将上限渲染为 `∞`。该事件与失败的 `assistant/chunk` 记录都不会生成表层消息,因此除非其他恢复策略有意改变表层,否则下一次请求包含的派生上下文与失败请求相同。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**单一全局 `always` 开关**:不予采纳,因为它无法把无界成本与延迟风险限制在确有需要的提供方,还可能在运行时重新路由后悄然生效。
|
||||
|
||||
**在 `dsh-llm-retry` 上维护单独的精确提供方列表**:不予采纳,因为它会在所属适配器配置之外重复提供方路由名称,并让提供方注册与恢复策略发生偏差。
|
||||
|
||||
**设置很大的有限重试次数**:不予采纳,因为它最终仍会违反持续重试的契约,并把任意选取的运维上限序列化成看似有意义的数值。
|
||||
|
||||
**使用提供方 SDK 重试**:不予采纳,因为隐藏尝试会叠加 agent 层预算,无法利用已关闭 step 的持久性边界,还可能在没有可重建重试记录的情况下拼接或丢弃流式输出。
|
||||
|
||||
**把错误放入模型上下文**:不予采纳,因为传输或提供方诊断信息属于运维状态,而非对话内容。它可能暴露敏感的提供方细节,并会改变重试请求,无法重复原本失败的请求。
|
||||
|
||||
## 验证
|
||||
|
||||
适配器测试会在提供方加载时校验嵌套策略,证明注册流程会捕获已配置策略和默认策略,并证明请求进行期间替换路由后仍会保留实际提供服务的策略。单元测试根据失败请求实际使用的注册项选择策略、分离不同提供方和策略变更后的重试历史、验证 always 模式可越过 normal 预算、固定抖动和延迟上限、证明下游恢复顺序、证明取消与 dispose 会先排空已委托的恢复再达到完全停稳,并证明二者都会停止正在进行的退避等待。请求级覆盖会比较失败尝试与重试尝试的完整消息,并排除提供方错误文本和丢弃的部分输出。一个无密钥 headless `stream-json` 快照会通过组装后的应用执行失败、重试与成功流程,固定完整的 `llm/retry` 记录,并拒绝各次尝试之间出现任何模型消息变化。JSONL 与 SQLite 测试会往返读写不含 `Infinity` 的 always 事件;不变式测试会将提供方标识绑定到请求头、校验失败事实和各模式的计时器边界,并将重试编号绑定到提供方策略键;TUI 测试会渲染有限和无限上限。
|
||||
|
||||
## 后果
|
||||
|
||||
normal 模式仍是有限的默认策略;显式的 always 策略可能在永久性的身份验证、配额、无效请求、协议或上下文错误上耗费无限次请求和无限时间。运维方必须为 always 模式配备可取消的调用方和针对提供方的成本控制。重试状态保持可观察且持久,但不会对模型可见;捕获实际提供服务的注册项,也能防止适配器生命周期变化反过来改变进行中请求的恢复契约。
|
||||
|
||||
本决策扩展了[瞬态 LLM(大语言模型)请求失败的有界恢复](../architecture/2026-06-21-bounded-llm-request-recovery.md)中确定的已关闭 step 恢复、单次可见适配器尝试、结构化失败与持久状态设计。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/architecture.md
|
||||
architecture.md: 3b0f72400ab1e6a9157aed2966b4a17c36e0c3ac
|
||||
architecture.zh.md: fa21c82a686d88a9bbec02180feae1dd0dbf4e47
|
||||
architecture.md: 56d391101aaa44908fc65201bd364452a2eaed27
|
||||
architecture.zh.md: ab76b0a256c4f23de360bbf08c042725816278bd
|
||||
@@ -117,11 +117,11 @@ Each step assembles ordered prompt sections, tool schemas, and variables; unknow
|
||||
|
||||
Admission-time and active-turn `inject()` stage for the next step; post-tool `additionalContexts` settles after results. Steering shares that staging boundary and requests another step. Idle `inject()` appends immediately without changing turn numbers; persistence drains eagerly.
|
||||
|
||||
Pruning precedes summaries; overflow retries require durable progress. Recovery uses `agent/request-error` after the failed step and before turn close. Its policy returns a retry action to schedule one retry turn; cancellation wins ([compaction](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md), [retry](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md)).
|
||||
Pruning precedes summaries; overflow retries require durable progress. `agent/request-error` may authorize one retry turn between failed-step and turn close; cancellation wins. Adapter-owned `retryPolicy` makes normal mode bounded; always mode delegates specialized recovery before retrying until success or cancellation ([compaction](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md), [retry foundation](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md), [provider policy](../.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md)).
|
||||
|
||||
### Failure Boundaries
|
||||
|
||||
Adapter failures close the step before `agent/request-error` receives the exact `Error`, normalized `LlmFailure`, and turn signal. A handling listener returns `{ kind: 'retry' }`; the loop closes the failed turn and opens another from durable history without idle notification. Exhaustion leaves the failed `turn/end` terminal; failed chunks commit neither message nor tool call.
|
||||
Adapter failures close their step before `agent/request-error` receives the exact `Error`, normalized `LlmFailure`, and signal. A handled failure closes its turn and opens a retry turn from durable history without an idle notification; exhaustion leaves terminal `turn/end`. Failed chunks commit neither messages nor tool calls.
|
||||
|
||||
Other failures use `agent/error`. Cancellation and disposal beat recovery. Before request-header commit, the turn signal cancels asynchronous model-capability preparation; undispatched tools get synthetic `tool/call`/`ABORTED_BEFORE_DISPATCH` pairs. Effective `cancel(cause)` emits its cause before queue clearing and abort; observers cannot veto; idle calls emit nothing. Durability records user or parent cancellation as `aborted`, teardown as `disposed`; teardown awaits quiescence. The cause affects reporting, not late result-context handling ([decision](../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)).
|
||||
|
||||
|
||||
@@ -117,11 +117,11 @@ idle inject:
|
||||
|
||||
接纳期间和活跃轮次内的 `inject()` 会为下一步骤暂存;工具执行后的 `additionalContexts` 会在结果记录完毕后落定。steering 与其共用这一暂存边界,并请求再执行一个步骤。空闲状态下的 `inject()` 会立即追加,且不改变轮次编号;持久化层会尽快排空。
|
||||
|
||||
裁剪先于摘要;溢出重试必须取得持久进展。恢复会在失败步骤关闭后、轮次关闭前使用 `agent/request-error`。其策略返回重试动作以安排一个重试轮次;取消优先([压缩](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)、[重试](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md))。
|
||||
裁剪先于摘要;溢出重试必须取得持久进展。`agent/request-error` 可以在失败步骤与轮次关闭之间授权一个重试轮次;取消优先。适配器拥有的 `retryPolicy` 使 normal mode 保持有界;always mode 先委托专门恢复,再持续重试直至成功或取消([压缩](../.agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md)、[重试基础](../.agents/notes/implemented/architecture/2026-06-21-bounded-llm-request-recovery.md)、[提供方策略](../.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md))。
|
||||
|
||||
### 失败边界
|
||||
|
||||
适配器故障会先关闭步骤,再由 `agent/request-error` 接收准确的 `Error`、标准化的 `LlmFailure` 和轮次信号。负责处理的监听器返回 `{ kind: 'retry' }`;循环关闭失败轮次,并从持久历史开启另一个轮次,不发出空闲通知。重试耗尽后,失败的 `turn/end` 即为终态记录;失败分片不会提交消息或工具调用。
|
||||
适配器故障会先关闭自身步骤,再由 `agent/request-error` 接收准确的 `Error`、标准化的 `LlmFailure` 和信号。已处理的失败会关闭所在轮次,并从持久历史开启重试轮次,不发出空闲通知;重试耗尽则留下终态 `turn/end`。失败分片既不提交消息,也不提交工具调用。
|
||||
|
||||
其他故障使用 `agent/error`。取消和资源释放优先于恢复。在提交请求头之前,轮次信号会取消异步模型能力准备;尚未分派的工具会得到合成的 `tool/call`/`ABORTED_BEFORE_DISPATCH` 对。实际生效的 `cancel(cause)` 在清空队列和中止前发出原因;观察方不能否决;空闲调用不发事件。持久化层将用户或父级取消记录为 `aborted`,拆卸记录为 `disposed`;拆卸会等待完全停稳。原因只影响报告方式,不影响延迟完成的结果上下文处理([决策](../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md))。
|
||||
|
||||
|
||||
+22
-28
@@ -73,8 +73,6 @@ export interface Config {
|
||||
toolTasks?: NonNullable<agentCore.Config['toolTasks']>
|
||||
/** Persisted same-session goals; owner defaults enable them, or false disables the stack and tools. */
|
||||
goals?: agentCore.GoalConfig | false
|
||||
/** Bounded transient model-request retry policy forwarded through agent-core. */
|
||||
llmRetry?: NonNullable<agentCore.Config['llmRetry']>
|
||||
}
|
||||
```
|
||||
|
||||
@@ -124,8 +122,9 @@ Source: [`packages/core/agent-loop/src/index.ts:147`](../packages/core/agent-loo
|
||||
* `dshHome` to bash environment and local skill discovery, `sessionTitle` to
|
||||
* the fallback title service, `skills` to the
|
||||
* skill registry/local provider/tool consumer, `workspaceContext` to the
|
||||
* workspace-context loader, `llmRetry` to the bounded request-recovery policy,
|
||||
* and `toolBash`/`toolTasks` to the model-facing tool plugins this bundle owns.
|
||||
* workspace-context loader, and `toolBash`/`toolTasks` to the model-facing tool
|
||||
* plugins this bundle owns. Provider adapters own their `retryPolicy`; this
|
||||
* bundle always mounts its executor.
|
||||
* `goals` opts into and configures the persisted goal domain plus its model tool
|
||||
* and same-session driver; `invariants` configures global and package-filtered
|
||||
* relational checks. Owner schemas supply defaults for optional input;
|
||||
@@ -161,8 +160,6 @@ export interface Config {
|
||||
invariants?: InvariantConfig
|
||||
/** Opt-in persisted same-session goal stack; set false or omit to leave it unmounted. */
|
||||
goals?: GoalConfig | false
|
||||
/** Bounded transient model-request retry policy. */
|
||||
llmRetry?: llmRetry.Config
|
||||
}
|
||||
|
||||
/** Skill bundle config forwarded to the registry, local provider, and model-facing consumer. */
|
||||
@@ -186,9 +183,9 @@ export interface GoalConfig {
|
||||
}
|
||||
```
|
||||
|
||||
Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`GoalDomainConfig`](#deepseek-aidsh-goal) · [`InvariantConfig`](#deepseek-aidsh-invariants) · [`llmRetry`](../packages/llm/llm-retry/src/index.ts) · [`SessionTitleConfig`](#deepseek-aidsh-session-title) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`toolGoal`](../packages/goal/tool-goal/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts)
|
||||
Depends on: [`AgentLoopConfig`](#deepseek-aidsh-agent-loop) · [`GoalDomainConfig`](#deepseek-aidsh-goal) · [`InvariantConfig`](#deepseek-aidsh-invariants) · [`SessionTitleConfig`](#deepseek-aidsh-session-title) · [`SkillLocal`](../packages/skill/skill-local/src/index.ts) · [`SkillRegistryConfig`](#deepseek-aidsh-skill) · [`SystemPromptConfig`](#deepseek-aidsh-system-prompt) · [`toolBash`](../packages/bash/tool-bash/src/index.ts) · [`toolGoal`](../packages/goal/tool-goal/src/index.ts) · [`ToolsConfig`](#deepseek-aidsh-tools) · [`toolSkill`](../packages/skill/tool-skill/src/index.ts) · [`toolTasks`](../packages/tasks/tool-tasks/src/index.ts) · [`workspaceContext`](../packages/context/workspace-context/src/index.ts)
|
||||
|
||||
Source: [`packages/examples/agent-spine-demo/src/index.ts:87`](../packages/examples/agent-spine-demo/src/index.ts)
|
||||
Source: [`packages/examples/agent-spine-demo/src/index.ts:88`](../packages/examples/agent-spine-demo/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-bash-local`
|
||||
|
||||
@@ -264,8 +261,6 @@ export interface Config {
|
||||
toolBash?: NonNullable<agentCore.Config['toolBash']>
|
||||
/** Generic background-task control-tool config forwarded through agent-spine-demo. */
|
||||
toolTasks?: NonNullable<agentCore.Config['toolTasks']>
|
||||
/** Bounded transient model-request retry policy forwarded through agent-spine-demo. */
|
||||
llmRetry?: NonNullable<agentCore.Config['llmRetry']>
|
||||
/** Controls automatic AGENTS.md/CLAUDE.md loading; configure a byte budget or set `false`. */
|
||||
workspaceContext: agentCore.Config['workspaceContext']
|
||||
}
|
||||
@@ -586,6 +581,8 @@ export interface Config {
|
||||
models?: DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** One optional model entry advertised by the direct-fetch adapter. */
|
||||
@@ -601,7 +598,9 @@ export interface DeepSeekCatalogModel {
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/llm/llm-deepseek/src/index.ts:35`](../packages/llm/llm-deepseek/src/index.ts)
|
||||
Depends on: [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
|
||||
|
||||
Source: [`packages/llm/llm-deepseek/src/index.ts:36`](../packages/llm/llm-deepseek/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-pi-ai`
|
||||
|
||||
@@ -638,12 +637,14 @@ export interface PiAiProviderProfile {
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
```
|
||||
|
||||
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:48`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:54`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-replay`
|
||||
|
||||
@@ -676,6 +677,8 @@ export interface ReplayProviderConfig {
|
||||
name?: string
|
||||
/** Advisory models exposed to replay scenarios that exercise discovery. */
|
||||
models?: ReplayModelConfig[]
|
||||
/** Optional provider-owned retry policy used by assembled recovery snapshots. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** One model exposed by a replay-only provider catalog. */
|
||||
@@ -691,29 +694,20 @@ export interface ReplayModelConfig {
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/support/llm-replay/src/index.ts:598`](../packages/support/llm-replay/src/index.ts)
|
||||
Depends on: [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
|
||||
|
||||
Source: [`packages/support/llm-replay/src/index.ts:617`](../packages/support/llm-replay/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-llm-retry`
|
||||
|
||||
Requires: `agents`
|
||||
|
||||
```ts config-catalog
|
||||
/** Deployment-owned limits and classification for transient request recovery. */
|
||||
export interface Config {
|
||||
/** Maximum transient retries after the first request (default 2). */
|
||||
maxTransientRetries?: number
|
||||
/** Initial local exponential-backoff delay in milliseconds (default 500). */
|
||||
initialDelayMs?: number
|
||||
/** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */
|
||||
maxDelayMs?: number
|
||||
/** Symmetric random multiplier range around one (default 0.1). */
|
||||
jitterRatio?: number
|
||||
/** Stable failure codes eligible for this policy. */
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
/** This policy executor has no config; providers own `retryPolicy`. */
|
||||
export type Config = Readonly<Record<string, never>>
|
||||
```
|
||||
|
||||
Source: [`packages/llm/llm-retry/src/index.ts:39`](../packages/llm/llm-retry/src/index.ts)
|
||||
Source: [`packages/llm/llm-retry/src/index.ts:45`](../packages/llm/llm-retry/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-lsp-local`
|
||||
|
||||
|
||||
@@ -96,7 +96,7 @@ A step or turn errored. The machine reports a failure here (plus the logger) eve
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:419`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:423`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
### `agent/inbox/dequeue` — emit
|
||||
|
||||
@@ -227,16 +227,20 @@ Handle a model-request failure after its failed step has closed but before the f
|
||||
* @param step - the failed step number.
|
||||
* @param error - the original model-request failure.
|
||||
* @param failure - serializable facts normalized at the final adapter boundary.
|
||||
* @param priorFailures - immutable failures that already authorized another
|
||||
* retry turn in this consecutive sequence.
|
||||
* @param retryPolicy - immutable policy of the adapter registration that served
|
||||
* the failed request, or `undefined` if no final adapter served it.
|
||||
* @param signal - the turn abort signal.
|
||||
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
||||
* @mode waterfall
|
||||
*/
|
||||
'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, signal: AbortSignal, next: () => Promise<RequestErrorAction>): Promise<RequestErrorAction>
|
||||
'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], retryPolicy: ResolvedRetryPolicy | undefined, signal: AbortSignal, next: () => Promise<RequestErrorAction>): Promise<RequestErrorAction>
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [LlmFailure](../core-data-structures/llm-streaming.md) · [RequestError](../core-data-structures/core.md) · [RequestErrorAction](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)
|
||||
Types: [Agent](../core-data-structures/core.md) · [LlmFailure](../core-data-structures/llm-streaming.md) · [RequestError](../core-data-structures/core.md) · [RequestErrorAction](../core-data-structures/core.md) · [ResolvedRetryPolicy](../core-data-structures/llm-streaming.md) · [Scoped](../core-data-structures/scope.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:377`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:381`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
### `agent/session-start` — emit
|
||||
|
||||
@@ -283,7 +287,7 @@ One drain chain reached its terminal turn: that turn's `turn/end` is already com
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md) · [SettleReason](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:406`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:410`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
### `agent/status` — emit
|
||||
|
||||
@@ -353,7 +357,7 @@ The turn is about to close: the model owes no response (no live tool calls, no f
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [Scoped](../core-data-structures/scope.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:392`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:396`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
## `agent-loop/*`
|
||||
|
||||
@@ -544,7 +548,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmService](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:53`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:56`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `session/*`
|
||||
|
||||
|
||||
@@ -721,6 +721,13 @@ registerAdapter(providers: string[], adapter: LlmAdapter): () => void
|
||||
*/
|
||||
listProviders(): LlmProviderInfo[]
|
||||
|
||||
/**
|
||||
* Resolve the retry policy captured when one provider route was registered.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @returns the provider-owned policy, with normal defaults already resolved.
|
||||
*/
|
||||
providerRetryPolicy(provider: string): ResolvedRetryPolicy
|
||||
|
||||
/**
|
||||
* Discover models advertised by one registered provider. Catalog membership
|
||||
* is advisory and never changes routing or request validation.
|
||||
@@ -778,9 +785,9 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<Prepared
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
```
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [ResolvedRetryPolicy](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:177`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:189`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `ctx.permission` — `PermissionService`
|
||||
|
||||
@@ -840,7 +847,7 @@ Source: [`packages/ui/permission/src/index.ts:97`](../../packages/ui/permission/
|
||||
get(agent: Agent): { active: boolean; pending?: boolean }
|
||||
|
||||
/**
|
||||
* Select whether plan mode should be active from the next turn boundary.
|
||||
* Select whether plan mode should be active from the next request boundary.
|
||||
* Repeated selection of the current or already-pending state is a no-op.
|
||||
*
|
||||
* @param agent The agent to switch.
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/core-data-structures/llm-streaming.md
|
||||
llm-streaming.md: a6aaaf3fed2ab25821efdf308c7297526ee7f3b5
|
||||
llm-streaming.zh.md: 521df7d1dad620cea322865fbefa8e4e2615fa78
|
||||
llm-streaming.md: db46deee28cd053d034f889eb7625c9f222b418b
|
||||
llm-streaming.zh.md: fbff50bf1f3b86afd313c2d5020c15dd88e0b449
|
||||
@@ -59,8 +59,8 @@ Every adapter MUST obey these, and every consumer may rely on them:
|
||||
|
||||
- **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering.
|
||||
- **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`.
|
||||
- **Two sanctioned error paths, one fact shape.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted', failure}` (provider in-band errors, for adapters that can't throw mid-stream). `LlmError.failure` carries the same `LlmFailure`. The final adapter boundary preserves the exact thrown `Error` object and associates immutable facts with that call; the agent loop closes the failed step and offers the error and facts to `agent/request-error`. A handling listener returns `{ kind: 'retry' }` after its awaited repair; absent recovery the structured failure becomes the turn error, and no normal assistant message or tool side effect is committed for that attempt.
|
||||
- **One adapter call is one provider attempt.** Adapters disable library retries. Agent-level recovery opens another durable numbered step; direct `ctx.llm.stream()` callers remain single-attempt.
|
||||
- **Two sanctioned error paths, one fact shape.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted', failure}` (provider in-band errors, for adapters that can't throw mid-stream). `LlmError.failure` carries the same `LlmFailure`. The final adapter boundary preserves the exact thrown `Error` object and associates immutable facts plus the serving registration's immutable retry policy with that call; the agent loop closes the failed step and offers the error, facts, immutable prior-retried facts, serving policy, and turn signal to `agent/request-error`. A handling listener returns `{ kind: 'retry' }` after its awaited repair; absent recovery the structured failure becomes the turn error, and no normal assistant message or tool side effect is committed for that attempt.
|
||||
- **One adapter call is one provider attempt.** Adapters disable library retries. Agent-level recovery opens another durable numbered turn; direct `ctx.llm.stream()` callers remain single-attempt.
|
||||
- **Provider stalls are bounded at the transport.** Both shipping remote adapters expose positive finite `streamIdleTimeoutMs` with a five-minute default. The watchdog arms only while iterator `next()` is outstanding, uses one stable signal for the whole request, maps its own expiry to `TIMEOUT`, and keeps an earlier caller abort as `ABORTED`.
|
||||
- **Context overflow has one canonical code.** Both DeepSeek adapters classify explicit provider detail through `isContextWindowExceededError()` and surface `CONTEXT_WINDOW_EXCEEDED`, whether the failure arrives as a thrown HTTP `LlmError` or an in-band finish error. Consumers route on the code, never provider text.
|
||||
- **An empty completion is a retryable error, not a silent success.** Both adapters map a terminal `stop` finish that carried no content blocks to `finish {kind:'error'}` with the canonical `EMPTY_RESPONSE` code, and `dsh-llm-retry` retries it by default; see [empty model responses are retryable](../../.agents/notes/implemented/bug-fix/2026-07-24-empty-model-response-is-retryable.md).
|
||||
@@ -69,6 +69,10 @@ Every adapter MUST obey these, and every consumer may rely on them:
|
||||
|
||||
This contract is pinned down by two deliberately independent implementations: `dsh-llm-deepseek` (direct fetch, SSE framing via `eventsource-parser`) and `dsh-llm-pi-ai` (a generic multi-provider adapter through `@earendil-works/pi-ai`). The library-backed adapter exercises the finish-chunk error path, while transport-boundary tests prove each idle watchdog stops its actual request.
|
||||
|
||||
## `ResolvedRetryPolicy`
|
||||
|
||||
Provider configuration resolves before route registration into an immutable discriminated union. Normal mode carries `mode: 'normal'`, finite `maxRetries`, `retryableCodes`, and required `initialDelayMs`, `maxDelayMs`, and `jitterRatio`; always mode carries `mode: 'always'` and the same required backoff fields without a finite maximum. `LlmService.providerRetryPolicy(provider)` returns the currently registered value and supplies normal defaults when the adapter omits one; `llmRetryPolicyOf(stream)` returns the exact serving registration's captured value after that call enters its final adapter boundary, so later route disposal or replacement cannot change an in-flight failure's recovery policy. The [generated config catalog](../config-catalog.md) owns the optional input shapes.
|
||||
|
||||
## `AppIdentity` — app attribution
|
||||
|
||||
The static public application identity every adapter sends to providers ([`packages/llm/llm/src/attribution.ts`](../../packages/llm/llm/src/attribution.ts)). `attributionHeaders(identity?)` maps it to the standard `User-Agent` header only; OpenRouter-specific app attribution headers are intentionally not supported by this contract. The default `APP_IDENTITY` sources its version from the package manifest; every field is a public product fact - no secrets, paths, session ids, or per-user identifiers, and nothing per-request may influence the values. Rationale: [Mandatory `User-Agent` attribution](../../.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md).
|
||||
@@ -157,7 +161,7 @@ declare class BlockAssembler {
|
||||
|
||||
## The seam
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerInfo()` and asynchronous `listModels()` methods feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
@@ -189,6 +193,12 @@ declare abstract class LlmAdapter {
|
||||
* @returns detached display metadata whose id must equal `provider`.
|
||||
*/
|
||||
providerInfo(provider: string): LlmProviderInfo;
|
||||
/**
|
||||
* Return the provider-owned retry policy captured with this route.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @returns a resolved policy, or `undefined` to use the normal defaults.
|
||||
*/
|
||||
providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
|
||||
@@ -59,8 +59,8 @@ interface LlmFailure {
|
||||
|
||||
- **`usage` 在 `finish` 之前,`finish` 之后不再有任何分片。** 将两者都推迟到提供方的流结束标记,这样尾部的 usage-only 分片就不会违反顺序。
|
||||
- **工具调用的 `arguments` 全程保持原始 JSON 字符串。** 部分片段通过 `argumentsDelta` 流式传输;如果提供方返回的是已解析的对象,适配器在 `block-end` 时重新序列化为字符串。
|
||||
- **两条受支持的错误路径,一种事实形状。** 失败可以从 `stream()` 抛出(传输/协议错误),**或者**以 `finish {kind:'error'|'aborted', failure}` 结束流(无法在流中途抛异常的适配器用它表示提供方带内错误)。`LlmError.failure` 携带同一个 `LlmFailure`。最终适配器边界保留被抛出的确切 `Error` 对象,并将不可变事实关联到该调用;agent loop(智能体循环)关闭失败的步骤,再把错误与事实提供给 `agent/request-error`。处理该错误的 listener 在其 await 的修复完成后返回 `{ kind: 'retry' }`;若未恢复,结构化失败会成为轮次错误,并且该次尝试不会提交正常 assistant 消息或工具副作用。
|
||||
- **一次适配器调用就是一次提供方尝试。** 适配器禁用库重试。agent 层恢复会打开另一个持久、带编号的步骤;直接调用 `ctx.llm.stream()` 的调用方仍然只尝试一次。
|
||||
- **两条受支持的错误路径,一种事实形状。** 失败可以从 `stream()` 抛出(传输/协议错误),**或者**以 `finish {kind:'error'|'aborted', failure}` 结束流(无法在流中途抛异常的适配器用它表示提供方带内错误)。`LlmError.failure` 携带同一个 `LlmFailure`。最终适配器边界保留被抛出的确切 `Error` 对象,并将不可变事实以及实际服务注册所对应的不可变重试策略关联到该调用;agent loop(智能体循环)关闭失败步骤,再把错误、事实、不可变的先前已重试失败事实、实际服务策略和轮次信号提供给 `agent/request-error`。处理该错误的 listener 在其 await 的修复完成后返回 `{ kind: 'retry' }`;若未恢复,结构化失败会成为轮次错误,并且该次尝试不会提交正常 assistant 消息或工具副作用。
|
||||
- **一次适配器调用就是一次提供方尝试。** 适配器禁用库重试。agent 层恢复会打开另一个持久、带编号的轮次;直接调用 `ctx.llm.stream()` 的调用方仍然只尝试一次。
|
||||
- **提供方停顿在传输层受到时限约束。** 两个已交付的远程适配器都暴露正数且有限的 `streamIdleTimeoutMs`,默认五分钟。watchdog 只在 iterator `next()` 尚未完成时启动,整个请求使用同一个稳定 signal,把自身到期映射为 `TIMEOUT`,并把更早发生的调用方中止保留为 `ABORTED`。
|
||||
- **上下文溢出只有一个规范 code。** 两个 DeepSeek 适配器都通过 `isContextWindowExceededError()` 对提供方的显式细节分类并暴露 `CONTEXT_WINDOW_EXCEEDED`,无论失败以抛出的 HTTP `LlmError` 还是带内 finish error 到达。消费方按 code 路由,绝不依赖提供方文本。
|
||||
- **空 completion 是可重试错误,而不是静默的成功结果。** 两个适配器都把没有携带任何内容块的终止性 `stop` 结束映射为携带规范 `EMPTY_RESPONSE` code 的 `finish {kind:'error'}`,`dsh-llm-retry` 默认会重试它;详见[空模型响应可重试](../../.agents/notes/implemented/bug-fix/2026-07-24-empty-model-response-is-retryable.md)。
|
||||
@@ -69,6 +69,10 @@ interface LlmFailure {
|
||||
|
||||
该契约由两个有意保持独立的实现锁定:`dsh-llm-deepseek`(直接 fetch,SSE(Server-Sent Events)分帧经由 `eventsource-parser`)和 `dsh-llm-pi-ai`(通过 `@earendil-works/pi-ai` 实现的通用多提供方适配器)。基于库的适配器覆盖 finish 分片错误路径,而传输边界测试证明每个空闲 watchdog 都会停止其实际请求。
|
||||
|
||||
## `ResolvedRetryPolicy`
|
||||
|
||||
提供方配置会在路由注册前解析为不可变的可辨识联合。normal mode 携带 `mode: 'normal'`、有限的 `maxRetries`、`retryableCodes`,以及必填的 `initialDelayMs`、`maxDelayMs` 与 `jitterRatio`;always mode 携带 `mode: 'always'` 和相同的必填退避字段,但没有有限上限。`LlmService.providerRetryPolicy(provider)` 返回当前注册的值,并在适配器省略策略时提供 normal 默认值;调用进入最终适配器边界后,`llmRetryPolicyOf(stream)` 返回为其提供服务的确切注册所捕获的值,因此之后释放或替换路由都无法改变进行中失败的恢复策略。可选输入形状由[生成的配置目录](../config-catalog.md)规定。
|
||||
|
||||
## `AppIdentity`:应用归属
|
||||
|
||||
每个适配器都会向提供方发送的静态公开应用标识([`packages/llm/llm/src/attribution.ts`](../../packages/llm/llm/src/attribution.ts))。`attributionHeaders(identity?)` 只把它映射到标准 `User-Agent` header;该契约有意不支持 OpenRouter 特有的应用归属 header。默认 `APP_IDENTITY` 从包(package) manifest(元数据清单)获取版本;每个字段都是公开产品事实——不含 secret、路径、会话 id 或逐用户标识,且任何逐请求信息都不得影响这些值。设计理由见[强制 `User-Agent` 归属](../../.agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md)。
|
||||
@@ -157,7 +161,7 @@ declare class BlockAssembler {
|
||||
|
||||
## seam
|
||||
|
||||
`LlmAdapter` 是提供方 seam:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerInfo()` 与异步 `listModels()` 方法为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值,因此直接调用也无法绕过对不支持推理强度的拒绝;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册。适配器查找发生在 `llm/stream` waterfall(瀑布式事件)的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface(`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
`LlmAdapter` 是提供方 seam:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值,`providerInfo()` 与异步 `listModels()` 方法则为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值,因此直接调用也无法绕过对不支持推理强度的拒绝;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册。适配器查找发生在 `llm/stream` waterfall(瀑布式事件)的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface(`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
|
||||
|
||||
```ts type-equiv
|
||||
/** One model call whose config and adapter registration were resolved together. */
|
||||
@@ -189,6 +193,12 @@ declare abstract class LlmAdapter {
|
||||
* @returns detached display metadata whose id must equal `provider`.
|
||||
*/
|
||||
providerInfo(provider: string): LlmProviderInfo;
|
||||
/**
|
||||
* Return the provider-owned retry policy captured with this route.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @returns a resolved policy, or `undefined` to use the normal defaults.
|
||||
*/
|
||||
providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
|
||||
@@ -11,18 +11,18 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| `agent/cancel-requested` | `emit` | [`packages/core/agent/src/types.ts:308`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`goal-session`](../packages/goal/goal-session) |
|
||||
| `agent/created` | `emit` | [`packages/core/agent/src/types.ts:247`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`goal-session`](../packages/goal/goal-session), [`tui`](../packages/ui/tui) |
|
||||
| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:256`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`goal-session`](../packages/goal/goal-session), [`tui`](../packages/ui/tui) |
|
||||
| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:419`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | `apiproxy`, [`goal-session`](../packages/goal/goal-session), [`session-telemetry`](../packages/telemetry/session-telemetry), [`tui`](../packages/ui/tui) |
|
||||
| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:423`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | `apiproxy`, [`goal-session`](../packages/goal/goal-session), [`session-telemetry`](../packages/telemetry/session-telemetry), [`tui`](../packages/ui/tui) |
|
||||
| `agent/inbox/dequeue` | `emit` | [`packages/core/agent/src/types.ts:286`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`agent`](../packages/core/agent), `apiproxy`, [`tui`](../packages/ui/tui) |
|
||||
| `agent/inbox/discard` | `emit` | [`packages/core/agent/src/types.ts:298`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`agent`](../packages/core/agent), `apiproxy`, [`tui`](../packages/ui/tui) |
|
||||
| `agent/inbox/enqueue` | `emit` | [`packages/core/agent/src/types.ts:276`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`agent`](../packages/core/agent), `apiproxy`, [`goal-session`](../packages/goal/goal-session), [`tui`](../packages/ui/tui) |
|
||||
| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:336`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`goal-session`](../packages/goal/goal-session), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`repeat-tool-guard`](../packages/guard/repeat-tool-guard), [`tui`](../packages/ui/tui) |
|
||||
| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:362`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`agent`](../packages/core/agent) |
|
||||
| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:377`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic), [`llm-retry`](../packages/llm/llm-retry) |
|
||||
| `agent/request-error` | `waterfall` | [`packages/core/agent/src/types.ts:381`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`compact-basic`](../packages/compact/compact-basic), [`llm-retry`](../packages/llm/llm-retry) |
|
||||
| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:321`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`goal`](../packages/goal/goal), [`goal-session`](../packages/goal/goal-session), [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex), [`workspace-context`](../packages/context/workspace-context) |
|
||||
| `agent/settled` | `emit` | [`packages/core/agent/src/types.ts:406`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`compact-basic`](../packages/compact/compact-basic), [`llm-retry`](../packages/llm/llm-retry) |
|
||||
| `agent/settled` | `emit` | [`packages/core/agent/src/types.ts:410`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`compact-basic`](../packages/compact/compact-basic) |
|
||||
| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:265`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emitAgentEvent`) | [`agent`](../packages/core/agent), `apiproxy`, [`goal-session`](../packages/goal/goal-session), [`tui`](../packages/ui/tui) |
|
||||
| `agent/step` | `serial` | [`packages/core/agent/src/types.ts:349`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic), [`plan-mode`](../packages/plan/plan-mode), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`time-context`](../packages/context/time-context), [`tool-skill`](../packages/skill/tool-skill), [`user-approval`](../packages/ui/user-approval), [`workspace-context`](../packages/context/workspace-context) |
|
||||
| `agent/turn-stopping` | `serial` | [`packages/core/agent/src/types.ts:392`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `agent/turn-stopping` | `serial` | [`packages/core/agent/src/types.ts:396`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `approval/request` | `waterfall` | [`packages/ui/user-approval/src/index.ts:30`](../packages/ui/user-approval/src/index.ts) | [`user-approval`](../packages/ui/user-approval) (`waterfall`) | [`acp`](../packages/acp/acp) |
|
||||
| `commands/change` | `emit` | [`packages/ui/commands/src/index.ts:103`](../packages/ui/commands/src/index.ts) | [`commands`](../packages/ui/commands) (`events.dispatch`) | `apiproxy`, [`tui`](../packages/ui/tui) |
|
||||
| `domain/changed` | `emit` | [`packages/storage/storage-domain/src/events.ts:46`](../packages/storage/storage-domain/src/events.ts) | [`storage-domain`](../packages/storage/storage-domain) (`emit`) | `apiproxy`, [`storage-domain`](../packages/storage/storage-domain), [`workspace`](../packages/workspace/workspace) |
|
||||
@@ -30,10 +30,10 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:71`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:54`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `goal/changed` | `emit` | [`packages/goal/goal/src/types.ts:169`](../packages/goal/goal/src/types.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-session`](../packages/goal/goal-session) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:53`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/support/llm-replay), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`session-title`](../packages/session-title/session-title) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:56`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/support/llm-replay), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`session-title`](../packages/session-title/session-title) |
|
||||
| `session/created` | `emit` | [`packages/core/session/src/index.ts:70`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | `apiproxy`, [`compact`](../packages/compact/compact), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`jsonrpc`](../packages/ui/jsonrpc), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-telemetry`](../packages/telemetry/session-telemetry), [`user-approval`](../packages/ui/user-approval) |
|
||||
| `session/disposed` | `emit` | [`packages/core/session/src/index.ts:80`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `apiproxy`, [`session-persistence`](../packages/session-persistence/session-persistence), [`session-telemetry`](../packages/telemetry/session-telemetry), [`session-title`](../packages/session-title/session-title) |
|
||||
| `session/event` | `emit` | [`packages/core/session/src/index.ts:92`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/acp/acp), `apiproxy`, [`cli-demo`](../packages/examples/cli-demo), [`compact`](../packages/compact/compact), [`compact-basic`](../packages/compact/compact-basic), [`goal`](../packages/goal/goal), [`goal-session`](../packages/goal/goal-session), [`hook-protocol`](../packages/hooks/hook-protocol), [`jsonrpc`](../packages/ui/jsonrpc), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-telemetry`](../packages/telemetry/session-telemetry), [`session-title`](../packages/session-title/session-title), [`token-meter`](../packages/llm/token-meter), [`tui`](../packages/ui/tui), [`user-approval`](../packages/ui/user-approval), [`workspace-context`](../packages/context/workspace-context) |
|
||||
| `session/event` | `emit` | [`packages/core/session/src/index.ts:92`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/acp/acp), `apiproxy`, [`cli-demo`](../packages/examples/cli-demo), [`compact`](../packages/compact/compact), [`compact-basic`](../packages/compact/compact-basic), [`goal`](../packages/goal/goal), [`goal-session`](../packages/goal/goal-session), [`hook-protocol`](../packages/hooks/hook-protocol), [`jsonrpc`](../packages/ui/jsonrpc), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-telemetry`](../packages/telemetry/session-telemetry), [`session-title`](../packages/session-title/session-title), [`token-meter`](../packages/llm/token-meter), [`tui`](../packages/ui/tui), [`user-approval`](../packages/ui/user-approval), [`workspace-context`](../packages/context/workspace-context) |
|
||||
| `session/flush` | `parallel` | [`packages/core/session/src/index.ts:102`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`session-persistence`](../packages/session-persistence/session-persistence), [`session-telemetry`](../packages/telemetry/session-telemetry) |
|
||||
| `slash/input-begin-command` | `bail` | [`packages/client/ui-slash/src/types.ts:220`](../packages/client/ui-slash/src/types.ts) | - | `ui-conversation` |
|
||||
| `slash/input-consume-token` | `bail` | [`packages/client/ui-slash/src/types.ts:234`](../packages/client/ui-slash/src/types.ts) | - | `ui-conversation` |
|
||||
|
||||
@@ -258,6 +258,7 @@ flowchart TD
|
||||
pkg_subprocess --> pkg_invariants
|
||||
pkg_llm --> pkg_brand
|
||||
pkg_llm --> pkg_invariants
|
||||
pkg_llm --> pkg_timeout
|
||||
pkg_client_connection --> pkg_host_webserver
|
||||
pkg_client_connection --> pkg_invariants
|
||||
pkg_client_hmr --> pkg_client_modules
|
||||
@@ -897,7 +898,7 @@ flowchart TD
|
||||
| [`host-webserver`](../packages/host/webserver) | `host` | [`invariants`](../packages/support/invariants) |
|
||||
| [`storage`](../packages/storage/storage) | `storage` | [`invariants`](../packages/support/invariants) |
|
||||
| [`subprocess`](../packages/subprocess/subprocess) | `subprocess` | [`invariants`](../packages/support/invariants) |
|
||||
| [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) |
|
||||
| [`llm`](../packages/llm/llm) | `llm` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`timeout`](../packages/util/timeout) |
|
||||
| [`client-connection`](../packages/client/connection) | `client` | [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) |
|
||||
| [`client-hmr`](../packages/client/hmr) | `client` | [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) |
|
||||
| [`client-locale`](../packages/client/locale) | `client` | [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`invariants`](../packages/support/invariants) |
|
||||
|
||||
@@ -271,14 +271,26 @@ Source: [`packages/hooks/hook-protocol/src/types.ts:31`](../packages/hooks/hook-
|
||||
#### `llm/retry` — log-only
|
||||
|
||||
```ts persistence-catalog
|
||||
/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */
|
||||
/** Durable, non-surface record of one provider-routed retry scheduled after a closed failed step. */
|
||||
'llm/retry': {
|
||||
turn: number
|
||||
step: number
|
||||
provider: string
|
||||
mode: 'normal'
|
||||
policyKey: string
|
||||
retry: number
|
||||
maxRetries: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
} | {
|
||||
turn: number
|
||||
step: number
|
||||
provider: string
|
||||
mode: 'always'
|
||||
policyKey: string
|
||||
retry: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
# Keyless replay for the retry overlay: disable the key-requiring DeepSeek
|
||||
# adapter, insert `llm-replay`, and restate the app config with the same
|
||||
# deterministic 1 ms zero-jitter retry policy as the live sibling. A config
|
||||
# patch replaces the whole app config, so the base fields are restated
|
||||
# verbatim (raw JSONL persistence so the harness can harvest the log).
|
||||
# adapter, insert `llm-replay`, and give its provider the same deterministic
|
||||
# 1 ms zero-jitter retry policy as the live sibling. The app patch still
|
||||
# restates its whole config for raw JSONL persistence and the recorded model.
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
@@ -20,11 +19,6 @@
|
||||
persistenceCompression: none
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
llmRetry:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 1
|
||||
maxDelayMs: 1
|
||||
jitterRatio: 0
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug.
|
||||
|
||||
@@ -36,6 +30,13 @@
|
||||
providers:
|
||||
- id: deepseek
|
||||
name: DeepSeek
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 2
|
||||
backoff:
|
||||
initialDelayMs: 1
|
||||
maxDelayMs: 1
|
||||
jitterRatio: 0
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
- id: deepseek-v4-pro
|
||||
@@ -2,14 +2,32 @@
|
||||
# deterministic 1 ms zero-jitter delay so the durable `llm/retry` event
|
||||
# (`delayMs`) and replay wall time stay reproducible. The overlay changes no
|
||||
# tool or prompt composition, so its scenarios share the default header class.
|
||||
# A config patch replaces the whole app config, so the base fields are restated
|
||||
# verbatim; the model is re-pinned to `deepseek-v4-flash` like the other
|
||||
# snapshot overlays because the recorded corpus was captured on flash.
|
||||
# Config patches replace whole plugin configs: the provider patch restates its
|
||||
# adapter fields around `retryPolicy`, while the app patch re-pins the recorded
|
||||
# flash model and restates its base fields.
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
patches:
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL
|
||||
thinking: enabled
|
||||
reasoningEffort: max
|
||||
defaultContextWindow: 256000
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 2
|
||||
backoff:
|
||||
initialDelayMs: 1
|
||||
maxDelayMs: 1
|
||||
jitterRatio: 0
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
- id: deepseek-v4-pro
|
||||
- id: acp-agent
|
||||
name: '@deepseek-ai/dsh-acp-demo'
|
||||
config:
|
||||
@@ -19,11 +37,6 @@
|
||||
persistenceCompression: !!js "process.env.DSH_SNAPSHOT === undefined ? 'zstd' : 'none'"
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
llmRetry:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 1
|
||||
maxDelayMs: 1
|
||||
jitterRatio: 0
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug.
|
||||
|
||||
|
||||
@@ -131,10 +131,10 @@ const SCENARIOS: Scenario[] = [
|
||||
{ name: 'error-finish', hasModelTurn: true, recorded: false, overridden: true },
|
||||
// Keyless, authored (like error-finish): a live provider cannot be coaxed
|
||||
// into a degenerate empty completion, so the fixture scripts the adapters'
|
||||
// EMPTY_RESPONSE error finish (step 1) followed by the recovered reply
|
||||
// (step 2), proving the default retry policy end to end: the durable
|
||||
// EMPTY_RESPONSE error finish in turn 1 followed by the recovered reply
|
||||
// in retry turn 2, proving the default retry policy end to end: the durable
|
||||
// llm/retry event, no ACP output for the discarded attempt, the recovered
|
||||
// reply, and a clean completed turn. Its overlay only pins a deterministic
|
||||
// reply, and a clean completed retry turn. Its overlay only pins a deterministic
|
||||
// 1 ms zero-jitter delay, so it shares the default header class.
|
||||
{ name: 'empty-response-retry', hasModelTurn: true, recorded: false, configPath: RETRY_CONFIG },
|
||||
// Keyless, authored (like error-finish/cancel): deterministically forcing a
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
{"type":"assistant/chunk","seq":5,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":0,"outputTokens":0}}}}
|
||||
{"type":"assistant/chunk","seq":6,"time":0,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"error","failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}}}
|
||||
{"type":"step/end","seq":7,"time":0,"data":{"turn":1,"step":1}}
|
||||
{"type":"llm/retry","seq":8,"time":0,"data":{"turn":1,"step":1,"retry":1,"maxRetries":2,"delayMs":1,"failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}
|
||||
{"type":"llm/retry","seq":8,"time":0,"data":{"turn":1,"step":1,"provider":"deepseek","mode":"normal","policyKey":"[\"normal\",2,[\"EMPTY_RESPONSE\",\"RATE_LIMIT\",\"SERVER\",\"TIMEOUT\",\"TRANSPORT\"],1,1,0]","retry":1,"maxRetries":2,"delayMs":1,"failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}
|
||||
{"type":"turn/end","seq":9,"time":1785047244285,"data":{"turn":1,"reason":{"kind":"error","step":1,"failure":{"message":"model returned a completed response with no content","code":"EMPTY_RESPONSE"}}}}
|
||||
{"type":"turn/start","seq":10,"time":1785047244285,"data":{"turn":2,"trigger":{"kind":"retry"}}}
|
||||
{"type":"step/start","seq":11,"time":1785047244289,"data":{"turn":2,"step":1}}
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Keyless provider-retry composition for the headless stream-json snapshot.
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
patches:
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
disabled: true
|
||||
- insert:
|
||||
- id: retry-snapshot-backend
|
||||
name: './tests/fixtures/retry-snapshot-backend.mjs'
|
||||
@@ -0,0 +1,53 @@
|
||||
/** Deterministic provider adapter for the headless retry-policy snapshot. */
|
||||
|
||||
import {
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
resolveRetryPolicy,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
|
||||
class RetrySnapshotAdapter extends LlmAdapter {
|
||||
requests = 0
|
||||
firstMessages
|
||||
policy = resolveRetryPolicy({
|
||||
mode: 'normal',
|
||||
maxRetries: 1,
|
||||
retryableCodes: ['RATE_LIMIT'],
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
}, 'retry-snapshot-backend.retryPolicy')
|
||||
|
||||
providerRetryPolicy() {
|
||||
return this.policy
|
||||
}
|
||||
|
||||
async * stream(options) {
|
||||
const messages = JSON.stringify(options.messages)
|
||||
this.requests++
|
||||
if (this.requests === 1) {
|
||||
this.firstMessages = messages
|
||||
throw new LlmError('snapshot transient failure', 'RATE_LIMIT', { status: 429 })
|
||||
}
|
||||
if (this.requests === 2 && messages !== this.firstMessages) {
|
||||
throw new Error('retry snapshot changed the model-visible messages')
|
||||
}
|
||||
const text = 'RETRY_OK'
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'text-delta', index: 0, text }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text } }
|
||||
yield { type: 'usage', usage: { inputTokens: 4, outputTokens: 2 } }
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
}
|
||||
}
|
||||
|
||||
/** Cordis plugin name. */
|
||||
export const name = 'retry-snapshot-backend'
|
||||
/** Required LLM registry service. */
|
||||
export const inject = ['llm']
|
||||
|
||||
/**
|
||||
* Register the deterministic provider adapter.
|
||||
* @param {import('cordis').Context} ctx - plugin context carrying the LLM service.
|
||||
*/
|
||||
export function apply(ctx) {
|
||||
ctx.llm.registerAdapter(['deepseek'], new RetrySnapshotAdapter())
|
||||
}
|
||||
@@ -24,6 +24,8 @@ const ptyStreamExpected = join(ptyScenarioDir, 'stream-json.expected.jsonl')
|
||||
const ptyConfigPath = fileURLToPath(new URL('../pty.cordis.snapshot.yml', import.meta.url))
|
||||
const goalScenarioDir = join(snapshotsDir, 'goal-tools')
|
||||
const goalConfigPath = fileURLToPath(new URL('../goal.cordis.snapshot.yml', import.meta.url))
|
||||
const retryScenarioDir = join(snapshotsDir, 'provider-retry')
|
||||
const retryConfigPath = fileURLToPath(new URL('../retry.cordis.snapshot.yml', import.meta.url))
|
||||
const ralphScenarioDir = join(snapshotsDir, 'ralph-loop')
|
||||
const ralphConfigPath = fileURLToPath(new URL('../ralph.cordis.snapshot.yml', import.meta.url))
|
||||
const binScript = fileURLToPath(new URL('../../../packages/examples/cli-demo/src/bin.ts', import.meta.url))
|
||||
@@ -125,6 +127,46 @@ async function persistedLogs(cwd: string): Promise<PersistedLog[]> {
|
||||
}
|
||||
|
||||
describe('headless stream-json snapshots', () => {
|
||||
it('retries a transient provider failure through the one-shot app', async () => {
|
||||
const prompt = await scenarioPrompt(retryScenarioDir, 'provider-retry')
|
||||
const streamExpected = join(retryScenarioDir, 'stream-json.expected.jsonl')
|
||||
let runCwd = ''
|
||||
const result = await runLoaderSmoke({
|
||||
label: 'provider retry headless stream-json snapshot',
|
||||
tempDirPrefix: 'headless-snapshot-provider-retry-',
|
||||
binScript,
|
||||
configPath: retryConfigPath,
|
||||
binArgs: ['--config', retryConfigPath, '--output-format', 'stream-json', prompt],
|
||||
tsconfigPath,
|
||||
env: {
|
||||
DSH_SNAPSHOT: 'replay',
|
||||
NODE_OPTIONS: [process.env.NODE_OPTIONS, '--disable-warning=ExperimentalWarning'].filter(Boolean).join(' '),
|
||||
},
|
||||
prepare: (cwd) => { runCwd = cwd },
|
||||
inspect: async (cwd) => {
|
||||
const logs = await persistedLogs(cwd)
|
||||
expect(logs).toHaveLength(1)
|
||||
const records = parseJsonl(logs[0]?.content ?? '')
|
||||
const retries = records.filter(record => record.type === 'llm/retry')
|
||||
expect(retries).toHaveLength(1)
|
||||
expect(retries[0]?.data).toMatchObject({
|
||||
provider: 'deepseek',
|
||||
mode: 'normal',
|
||||
policyKey: '["normal",1,["RATE_LIMIT"],1,1,0]',
|
||||
retry: 1,
|
||||
maxRetries: 1,
|
||||
delayMs: 1,
|
||||
failure: { message: 'snapshot transient failure', code: 'RATE_LIMIT', status: 429 },
|
||||
})
|
||||
},
|
||||
})
|
||||
|
||||
expect(result.stderr).toBe('')
|
||||
const normalized = normalizeHeadlessStream(result.stdout, runCwd)
|
||||
if (refreshing) await writeFile(streamExpected, normalized)
|
||||
expect(normalized).toBe(await readFile(streamExpected, 'utf8'))
|
||||
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
|
||||
|
||||
it('logs the model default and a dynamic next-step reasoning effort', async () => {
|
||||
const result = await runLoaderSmoke({
|
||||
label: 'reasoning effort headless stream-json snapshot',
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"steps": [
|
||||
{
|
||||
"op": "prompt",
|
||||
"text": "retry the transient provider failure"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/start","seq":0,"time":0,"data":{"turn":1,"trigger":{"kind":"message","source":{"kind":"user"}}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"user/message","seq":1,"time":0,"data":{"content":[{"type":"text","text":"retry the transient provider failure"}],"source":{"kind":"user"}},"surfaceOp":"append"}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"session/title","seq":2,"time":0,"data":{"title":"retry the transient provider failure","messageSeqs":[1],"source":{"kind":"fallback"}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/start","seq":3,"time":0,"data":{"turn":1,"step":1}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"request/header","seq":4,"time":0,"data":{"header":{"config":{"provider":"deepseek","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/end","seq":5,"time":0,"data":{"turn":1,"step":1}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"llm/retry","seq":6,"time":0,"data":{"turn":1,"step":1,"provider":"deepseek","mode":"normal","policyKey":"[\"normal\",1,[\"RATE_LIMIT\"],1,1,0]","retry":1,"maxRetries":1,"delayMs":1,"failure":{"message":"snapshot transient failure","code":"RATE_LIMIT","status":429}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/end","seq":7,"time":0,"data":{"turn":1,"reason":{"kind":"error","step":1,"failure":{"message":"snapshot transient failure","code":"RATE_LIMIT","status":429}}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/start","seq":8,"time":0,"data":{"turn":2,"trigger":{"kind":"retry"}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/start","seq":9,"time":0,"data":{"turn":2,"step":1}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":10,"time":0,"data":{"turn":2,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":11,"time":0,"data":{"turn":2,"step":1,"chunk":{"type":"text-delta","index":0,"text":"RETRY_OK"}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":12,"time":0,"data":{"turn":2,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"RETRY_OK"}}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":13,"time":0,"data":{"turn":2,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":4,"outputTokens":2}}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/chunk","seq":14,"time":0,"data":{"turn":2,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"assistant/message","seq":15,"time":0,"data":{"turn":2,"step":1,"content":[{"type":"text","text":"RETRY_OK"}],"provenance":{"provider":"deepseek","model":"deepseek-v4-flash"},"usage":{"inputTokens":4,"outputTokens":2}},"sourceEventSeqs":[10,11,12,13,14],"surfaceOp":"append"}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"step/end","seq":16,"time":0,"data":{"turn":2,"step":1}}}
|
||||
{"type":"session_event","sessionId":"{{sessionId}}","event":{"type":"turn/end","seq":17,"time":0,"data":{"turn":2,"reason":{"kind":"completed"}}}}
|
||||
{"type":"result","success":true,"sessionId":"{{sessionId}}","turn":2,"result":"RETRY_OK","reason":{"kind":"completed"},"usage":{"inputTokens":4,"outputTokens":2}}
|
||||
@@ -173,6 +173,8 @@ export class BasicCompactService extends CompactService {
|
||||
_step,
|
||||
_error,
|
||||
failure,
|
||||
_priorFailures,
|
||||
_retryPolicy,
|
||||
signal,
|
||||
next,
|
||||
) => {
|
||||
|
||||
@@ -1296,7 +1296,7 @@ describe('automatic listener and loader composition', () => {
|
||||
const failure: LlmFailure = { message: error.message, code: error.code ?? 'UNKNOWN' }
|
||||
const turn = owner.session.events.findLast(event => event.type === 'turn/start')?.data.turn ?? 1
|
||||
return agentEvents(ctx, owner).waterfall(
|
||||
'agent/request-error', turn, 1, error, failure, signal, next,
|
||||
'agent/request-error', turn, 1, error, failure, [], undefined, signal, next,
|
||||
).then(action => action?.kind === 'retry')
|
||||
}
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
|
||||
import { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, LlmResolvedModelInfo, ResolvedRetryPolicy, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
@@ -73,6 +73,11 @@ class StepwiseToolAdapter extends LlmAdapter {
|
||||
class OverflowRecoveryAdapter extends LlmAdapter {
|
||||
readonly conversationRequests: GenerateOptions[] = []
|
||||
readonly summaryRequests: GenerateOptions[] = []
|
||||
private readonly retryPolicy = resolveRetryPolicy({
|
||||
mode: 'normal',
|
||||
maxRetries: 1,
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
}, 'compaction test provider retryPolicy')
|
||||
|
||||
constructor(
|
||||
private readonly delivery: 'thrown' | 'in-band',
|
||||
@@ -90,6 +95,10 @@ class OverflowRecoveryAdapter extends LlmAdapter {
|
||||
})
|
||||
}
|
||||
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
override async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
// The cache-reusing summarizer replays the conversation prefix and marks
|
||||
// its call only by the compaction instruction in the trailing user message.
|
||||
@@ -375,12 +384,7 @@ describe('context-overflow recovery across the real loop and compact-basic', ()
|
||||
const adapter = new OverflowRecoveryAdapter('thrown', true)
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await mountInvariants(ctx)
|
||||
await ctx.plugin(LlmRetry, {
|
||||
maxTransientRetries: 1,
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
jitterRatio: 0,
|
||||
})
|
||||
await ctx.plugin(LlmRetry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(TokenMeterService)
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
|
||||
@@ -376,6 +376,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
signature: 'listProviders(): LlmProviderInfo[]',
|
||||
jsDoc: '/**\n * Describe provider routes with a registered adapter.\n * @returns detached provider metadata in registration order.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'providerRetryPolicy(provider: string): ResolvedRetryPolicy',
|
||||
jsDoc: '/**\n * Resolve the retry policy captured when one provider route was registered.\n * @param provider - registered provider route to inspect.\n * @returns the provider-owned policy, with normal defaults already resolved.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'async listModels(provider: string): Promise<LlmModelInfo[]>',
|
||||
jsDoc: '/**\n * Discover models advertised by one registered provider. Catalog membership\n * is advisory and never changes routing or request validation.\n * @param provider - registered provider route to inspect.\n * @returns detached model metadata in adapter-preferred order.\n */',
|
||||
@@ -430,7 +434,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
},
|
||||
{
|
||||
signature: 'set(agent: Agent, active: boolean): void',
|
||||
jsDoc: '/**\n * Select whether plan mode should be active from the next turn boundary.\n * Repeated selection of the current or already-pending state is a no-op.\n *\n * @param agent The agent to switch.\n * @param active Whether plan mode should be active.\n */',
|
||||
jsDoc: '/**\n * Select whether plan mode should be active from the next request boundary.\n * Repeated selection of the current or already-pending state is a no-op.\n *\n * @param agent The agent to switch.\n * @param active Whether plan mode should be active.\n */',
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -1065,8 +1069,8 @@ export const EVENT_API: readonly EventApiEntry[] = [
|
||||
{
|
||||
name: 'agent/request-error',
|
||||
mode: 'waterfall',
|
||||
signature: '\'agent/request-error\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, signal: AbortSignal, next: () => Promise<RequestErrorAction>): Promise<RequestErrorAction>',
|
||||
jsDoc: '/**\n * Handle a model-request failure after its failed step has closed but\n * before the failed turn closes. A listener returns `{ kind: \'retry\' }`\n * without calling `next()` when it owns the error, or calls `next()` to\n * delegate. The default `undefined` leaves the failure terminal.\n * @param agent - the agent whose request failed.\n * @param turn - the open turn number.\n * @param step - the failed step number.\n * @param error - the original model-request failure.\n * @param failure - serializable facts normalized at the final adapter boundary.\n * @param signal - the turn abort signal.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */',
|
||||
signature: '\'agent/request-error\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], retryPolicy: ResolvedRetryPolicy | undefined, signal: AbortSignal, next: () => Promise<RequestErrorAction>): Promise<RequestErrorAction>',
|
||||
jsDoc: '/**\n * Handle a model-request failure after its failed step has closed but\n * before the failed turn closes. A listener returns `{ kind: \'retry\' }`\n * without calling `next()` when it owns the error, or calls `next()` to\n * delegate. The default `undefined` leaves the failure terminal.\n * @param agent - the agent whose request failed.\n * @param turn - the open turn number.\n * @param step - the failed step number.\n * @param error - the original model-request failure.\n * @param failure - serializable facts normalized at the final adapter boundary.\n * @param priorFailures - immutable failures that already authorized another\n * retry turn in this consecutive sequence.\n * @param retryPolicy - immutable policy of the adapter registration that served\n * the failed request, or `undefined` if no final adapter served it.\n * @param signal - the turn abort signal.\n * Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.\n * @mode waterfall\n */',
|
||||
summary: 'Handle a model-request failure after its failed step has closed but before the failed turn closes.',
|
||||
},
|
||||
{
|
||||
@@ -1751,7 +1755,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'LlmAdapter',
|
||||
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmCallConfig',
|
||||
@@ -1929,6 +1933,22 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
name: 'RequestHeaderReason',
|
||||
declaration: 'export type RequestHeaderReason = \'initial\' | \'resume\' | \'change\';',
|
||||
},
|
||||
{
|
||||
name: 'ResolvedAlwaysRetryPolicy',
|
||||
declaration: 'export interface ResolvedAlwaysRetryPolicy extends ResolvedRetryBackoff {\n readonly mode: \'always\';\n}',
|
||||
},
|
||||
{
|
||||
name: 'ResolvedNormalRetryPolicy',
|
||||
declaration: 'export interface ResolvedNormalRetryPolicy extends ResolvedRetryBackoff {\n readonly mode: \'normal\';\n readonly maxRetries: number;\n readonly retryableCodes: readonly string[];\n}',
|
||||
},
|
||||
{
|
||||
name: 'ResolvedRetryBackoff',
|
||||
declaration: 'export interface ResolvedRetryBackoff {\n readonly initialDelayMs: number;\n readonly maxDelayMs: number;\n readonly jitterRatio: number;\n}',
|
||||
},
|
||||
{
|
||||
name: 'ResolvedRetryPolicy',
|
||||
declaration: 'export type ResolvedRetryPolicy = ResolvedNormalRetryPolicy | ResolvedAlwaysRetryPolicy;',
|
||||
},
|
||||
{
|
||||
name: 'ResumeAgentOptions',
|
||||
declaration: 'export interface ResumeAgentOptions {\n readonly resumeSessionId: SessionId;\n readonly agentOptions?: AgentOptions;\n readonly signal?: AbortSignal;\n readonly setup?: (agentCtx: Context) => Promise<void> | void;\n}',
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/core/agent-loop/README.md
|
||||
README.md: ab6df4d49f05ff00b16a210830e7fd21504a0a9f
|
||||
README.zh.md: 5adb2a11ba3de2bb6fc7ff57b9d6dd07ac7f650e
|
||||
README.md: c12140f27aed400b0f7b4246700473e877d37632
|
||||
README.zh.md: 6394cd86f5f3241be07ef711c76079624bce1bfe
|
||||
@@ -64,7 +64,7 @@ Every provider call that reaches a successful finish appends exactly one `assist
|
||||
|
||||
After `agent/request` returns a provider/model call config, the loop asks `ctx.llm.prepareCall()` to validate any adapter-owned reasoning effort and materialize its configured default under the active turn signal. The prepared call retains the exact adapter registration across this asynchronous resolution, `request/header` logging, and terminal dispatch, so HMR cannot mix one adapter's capability result with another adapter's request. The effective config is logged before dispatch, so a listener can change effort between steps without hidden request drift. A route with no registered adapter preserves the proposed config so an `llm/stream` listener can own and short-circuit it; unhandled terminal dispatch still fails with `NO_ADAPTER`. A new loop instance restores the last effort only when its initial provider/model route exactly matches the logged route; a route change discards that opaque model-owned ID and resolves the new model independently.
|
||||
|
||||
Plugin failure ends the current turn, not the loop. A model-request failure first closes its step and enters `agent/request-error` with the exact live error, normalized provider facts, and the turn signal. A handling listener returns `{ kind: 'retry' }`; the loop closes the failed turn with its error and opens one numbered retry turn without an intervening idle notification. An unhandled failure is terminal. Other failures close directly. AgentLoop owns one cancellation signal for the current admission or turn. An effective `cancel(cause)` clears pending work unless `keepInbox` is set and cooperatively aborts that signal; idle cancellation is a no-op. Durable `turn/end` records `aborted` for `user` and `parent`, while disposal records `disposed`; undispatched model tool calls receive synthetic `tool/call` and `ABORTED_BEFORE_DISPATCH` result pairs. The cancellation cause changes reporting, not how result context finalized after cancellation is handled. Disposal waits for signal-ignoring work before registry removal. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns the lifecycle and race contract.
|
||||
Plugin failure ends the current turn, not the loop. Only final adapter dispatch/iteration failures and terminal in-band error or aborted finishes enter `agent/request-error`; middleware, result processing, tools, and other extension failures close directly. Recovery receives the exact live error, immutable provider facts, immutable prior failures, the immutable retry policy of the adapter registration that served the request, and the turn signal after the failed step closes; the policy is absent if no final adapter served it. A handling listener returns `{ kind: 'retry' }`; the loop closes the failed turn with its error and opens one numbered retry turn without an intervening idle notification. Success clears the consecutive history, and an unhandled failure is terminal. AgentLoop owns one cancellation signal for the current admission or turn. An effective `cancel(cause)` clears pending work unless `keepInbox` is set and cooperatively aborts that signal; idle cancellation is a no-op. Durable `turn/end` records `aborted` for `user` and `parent`, while disposal records `disposed`; undispatched model tool calls receive synthetic `tool/call` and `ABORTED_BEFORE_DISPATCH` result pairs. The cancellation cause changes reporting, not how result context finalized after cancellation is handled. Disposal waits for signal-ignoring work before registry removal. The [explicit-cancellation decision](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md) owns the lifecycle and race contract.
|
||||
|
||||
Within a step, exclusive calls form barriers; parallel-safe calls use a bounded rolling pool and are reclassified before start. Only dispatch/body overlaps. Policy, durable results, and result context remain model-ordered. Abort stops new calls, drains started results, and retains their finalized result context without distinguishing the cancellation cause.
|
||||
|
||||
@@ -73,7 +73,7 @@ Within a step, exclusive calls form barriers; parallel-safe calls use a bounded
|
||||
Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy:
|
||||
- Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute` → `tools/execute` → `tools/post-execute` → definition-owned `finalizeContent` → `tools/result` pipeline; exact event signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md)
|
||||
- Compaction: pressure on `agent/step`; canonical overflow repair on `agent/request-error`
|
||||
- Transient model recovery: `dsh-llm-retry` records and waits its finite backoff on `agent/request-error`, then returns a retry action
|
||||
- Model-request recovery: `dsh-llm-retry` records and waits exact-provider normal or unbounded backoff on `agent/request-error`, emits non-surface `llm/retry` status, then returns a retry action
|
||||
- Sandbox, permission, plan mode: `tools/pre-execute` for extensible deny/ask, `tools.guard()` for monotonic owner policy, `tools/post-execute` for result decisions, and `tools/result` for final observation
|
||||
- Sub-agents: implemented outside the loop as `ctx.subagents` providers; in-process providers use `ctx.agents.create()` and owned `AgentHandle` teardown, while generic [`ctx.tasks`](../../tasks/tasks/) plus [`dsh-tool-subagent`](../../subagent/tool-subagent/) own background collection.
|
||||
- Persistence: eager write-behind from `session/event`; `session/flush` is an explicit observation barrier
|
||||
|
||||
@@ -64,7 +64,7 @@ interface Config {
|
||||
|
||||
在 `agent/request` 返回提供方/模型调用配置后,循环会调用 `ctx.llm.prepareCall()`,在活跃轮次信号的控制下校验由适配器持有的推理(reasoning)强度,并填入其配置默认值。准备完成的调用会在这次异步解析、`request/header` 日志记录和最终分派期间保留同一项确切的适配器注册,因此 HMR(热模块替换)不会把某个适配器的能力解析结果与另一适配器的请求混用。生效配置会在分派前写入日志,因此监听器可以在步骤之间更改推理强度,而不会产生未记录的请求变化。没有已注册适配器的路由会保留原定配置,使 `llm/stream` 监听器可以接管并短路该请求;最终分派仍会以 `NO_ADAPTER` 拒绝未得到处理的路由。新循环实例仅在初始提供方/模型路由与日志路由完全一致时恢复上次的推理强度;路由变化会丢弃由前一模型持有的不透明 ID,并单独解析新模型。
|
||||
|
||||
插件失败会结束当前轮次,而不是结束循环。模型请求失败会先关闭其步骤,再带着确切的实时错误、规范化的提供方事实和轮次信号进入 `agent/request-error`。处理失败的监听器返回 `{ kind: 'retry' }`;循环用其错误关闭失败轮次,并在不插入空闲通知的情况下开启一个编号重试轮次。未被处理的失败是终态。其他失败直接关闭轮次。AgentLoop 为当前接纳或轮次拥有一个取消信号。有效的 `cancel(cause)` 在未设置 `keepInbox` 时清除待处理工作,并以协作方式中止该信号;空闲取消是空操作。持久 `turn/end` 为 `user` 和 `parent` 记录 `aborted`,dispose 则记录 `disposed`;未分发的模型工具调用会收到合成的 `tool/call` 与 `ABORTED_BEFORE_DISPATCH` 结果对。取消原因只改变报告方式,不改变对取消后已定案结果上下文的处理。Dispose 会等待忽略信号的工作完成,然后才从注册表移除。[显式取消决策](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)规定生命周期与竞态契约。
|
||||
插件失败会结束当前轮次,而不是结束循环。只有最终适配器分发/迭代失败以及带内的终止错误或中止结束才进入 `agent/request-error`;中间件、结果处理、工具及其他扩展失败会直接关闭轮次。失败步骤关闭后,恢复逻辑会接收确切的实时错误、不可变的提供方事实、不可变的先前失败、为请求提供服务的适配器注册所对应的不可变重试策略,以及轮次信号;如果没有最终适配器为其提供服务,则该策略缺失。处理失败的监听器返回 `{ kind: 'retry' }`;循环用其错误关闭失败轮次,并在不插入空闲通知的情况下开启一个编号重试轮次。成功会清除连续失败历史;未被处理的失败是终态。AgentLoop 为当前接纳或轮次拥有一个取消信号。有效的 `cancel(cause)` 在未设置 `keepInbox` 时清除待处理工作,并以协作方式中止该信号;空闲取消是空操作。持久 `turn/end` 为 `user` 和 `parent` 记录 `aborted`,dispose(资源释放)则记录 `disposed`;未分发的模型工具调用会收到合成的 `tool/call` 与 `ABORTED_BEFORE_DISPATCH` 结果对。取消原因只改变报告方式,不改变对取消后已定案结果上下文的处理。dispose 会等待忽略信号的工作完成,然后才从注册表移除。[显式取消决策](../../../.agents/notes/implemented/architecture/2026-07-16-explicit-turn-cancellation.md)规定生命周期与竞态契约。
|
||||
|
||||
在步骤内,独占调用形成屏障;并行安全调用使用有界滚动池,并在启动前重新分类。只有分发/主体会重叠。策略、持久结果和结果上下文仍保持模型顺序。中止会停止新调用,drain 已启动的结果,并保留其已定案的结果上下文,不区分取消原因。
|
||||
|
||||
@@ -73,7 +73,7 @@ interface Config {
|
||||
超出「调用模型、运行工具、重复」的所有内容,都属于监听事件分类体系的插件:
|
||||
- 钩子与策略:相关的 `agent/*` 检查点,加上受守卫保护的 `tools/pre-execute` → `tools/execute` → `tools/post-execute` → 定义拥有的 `finalizeContent` → `tools/result` 流水线;确切事件签名与 mode 位于生成的[事件目录](../../../docs/cordis-catalog/events.md)
|
||||
- 压缩(compaction):在 `agent/step` 上观测压力;在 `agent/request-error` 上修复规范溢出
|
||||
- 瞬时模型恢复:`dsh-llm-retry` 在 `agent/request-error` 上记录并等待其有限退避,然后返回重试动作
|
||||
- 模型请求恢复:`dsh-llm-retry` 在 `agent/request-error` 上记录并等待按确切提供方配置的 normal 或无界退避,发出不进入表层的 `llm/retry` 状态,然后返回重试动作
|
||||
- 沙箱、权限、计划模式:使用 `tools/pre-execute` 提供可扩展的拒绝/询问,使用 `tools.guard()` 提供单调拥有方策略,使用 `tools/post-execute` 处理结果决定,并使用 `tools/result` 进行最终观测
|
||||
- subagent:在循环外部实现为 `ctx.subagents` 提供方;进程内提供方使用 `ctx.agents.create()` 和拥有的 `AgentHandle` 进行 teardown,而通用的 [`ctx.tasks`](../../tasks/tasks/) 与 [`dsh-tool-subagent`](../../subagent/tool-subagent/) 负责后台收集。
|
||||
- 持久化:从 `session/event` 立即后写;`session/flush` 是显式观测屏障
|
||||
|
||||
@@ -27,9 +27,9 @@ import type {
|
||||
SendOptions,
|
||||
} from '@deepseek-ai/dsh-agent'
|
||||
import {
|
||||
BlockAssembler, LlmError, assertNever, deepFreeze, errorChain, isHarnessError, llmFailureOf, markAgentLoopRequest,
|
||||
BlockAssembler, LlmError, assertNever, deepFreeze, errorChain, isHarnessError, llmFailureOf, llmRetryPolicyOf, markAgentLoopRequest,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmCallConfig, LlmFailure, Message, PreparedLlmCall } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmCallConfig, LlmFailure, Message, PreparedLlmCall, ResolvedRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import { canonicalHeader, headerEquals } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionId, TurnEndReason, TurnTrigger, UserMessageData } from '@deepseek-ai/dsh-session'
|
||||
import { renderPrompt } from '@deepseek-ai/dsh-system-prompt'
|
||||
@@ -39,7 +39,7 @@ import { executeToolCalls } from './tool-calls.ts'
|
||||
/** One completed step or a final-adapter failure eligible for recovery. */
|
||||
type StepOutcome =
|
||||
| { kind: 'completed'; continueTurn: boolean; concluded: boolean; maxTokens: boolean }
|
||||
| { kind: 'request-failed'; error: RequestError; failure: LlmFailure }
|
||||
| { kind: 'request-failed'; error: RequestError; failure: LlmFailure; retryPolicy: ResolvedRetryPolicy | undefined }
|
||||
|
||||
/**
|
||||
* The concrete {@link Agent}: each `run()` owns one turn and repeats model
|
||||
@@ -303,6 +303,7 @@ export class ReactLoopAgent implements Agent {
|
||||
trigger: TurnTrigger,
|
||||
admitted: UserMessageData[] = [],
|
||||
inheritedOutboxLength = 0,
|
||||
priorFailures: readonly LlmFailure[] = Object.freeze([]),
|
||||
): Promise<void> {
|
||||
// Both entries hold the invariant: kick() clears the admission slot before
|
||||
// awaiting run(), and a retry is entered only after the prior run clears it.
|
||||
@@ -317,8 +318,9 @@ export class ReactLoopAgent implements Agent {
|
||||
let opened = false
|
||||
let reason: TurnEndReason = { kind: 'completed' }
|
||||
let settleReason: SettleReason = { kind: 'completed' }
|
||||
let retry = false
|
||||
const cancelRetry = (): void => { retry = false }
|
||||
let requestFailureHistory = priorFailures
|
||||
let retryFailures: readonly LlmFailure[] | undefined
|
||||
const cancelRetry = (): void => { retryFailures = undefined }
|
||||
signal.addEventListener('abort', cancelRetry, { once: true })
|
||||
|
||||
try {
|
||||
@@ -344,6 +346,7 @@ export class ReactLoopAgent implements Agent {
|
||||
const outcome = await this.step(turn, step, signal)
|
||||
switch (outcome.kind) {
|
||||
case 'completed':
|
||||
requestFailureHistory = Object.freeze([])
|
||||
if (outcome.maxTokens) reason = { kind: 'max-tokens' }
|
||||
// A concluding tool result is terminal: steering already in the
|
||||
// log waits for the next turn's request instead of reopening this
|
||||
@@ -361,10 +364,13 @@ export class ReactLoopAgent implements Agent {
|
||||
try {
|
||||
const action = await this.loopCtx.waterfall(
|
||||
agentCarrier(this), 'agent/request-error', this, turn, step, outcome.error,
|
||||
outcome.failure, signal,
|
||||
outcome.failure, requestFailureHistory, outcome.retryPolicy, signal,
|
||||
() => Promise.resolve<RequestErrorAction>(undefined),
|
||||
)
|
||||
retry = action?.kind === 'retry' && !signal.aborted
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while recovery is awaited.
|
||||
if (action?.kind === 'retry' && !signal.aborted) {
|
||||
retryFailures = Object.freeze([...requestFailureHistory, outcome.failure])
|
||||
}
|
||||
} catch (recoveryError: unknown) {
|
||||
this.loopCtx.logger.warn(
|
||||
`agent "${this.id}": request recovery failed at turn ${turn}, step ${step}: ${errorChain(recoveryError)}`,
|
||||
@@ -411,7 +417,7 @@ export class ReactLoopAgent implements Agent {
|
||||
this.session.append('turn/end', { turn, reason })
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
retry = false
|
||||
retryFailures = undefined
|
||||
this.loopCtx.logger.warn(`agent "${this.id}": closing turn ${turn} failed: ${errorChain(error)}`)
|
||||
emitAgentEvent(this.loopCtx, this, 'agent/error', turn, step, error)
|
||||
}
|
||||
@@ -422,8 +428,8 @@ export class ReactLoopAgent implements Agent {
|
||||
signal.removeEventListener('abort', cancelRetry)
|
||||
}
|
||||
|
||||
if (retry) {
|
||||
await this.run({ kind: 'retry' })
|
||||
if (retryFailures !== undefined) {
|
||||
await this.run({ kind: 'retry' }, [], 0, retryFailures)
|
||||
} else {
|
||||
// agent/settled names only committed turns: a run aborted or rejected
|
||||
// before turn/start has no durable turn/end for consumers to settle
|
||||
@@ -483,7 +489,7 @@ export class ReactLoopAgent implements Agent {
|
||||
} catch (error: unknown) {
|
||||
const facts = llmFailureOf(stream, error)
|
||||
if (facts !== undefined && error instanceof Error) {
|
||||
return { kind: 'request-failed', error, failure: facts }
|
||||
return { kind: 'request-failed', error, failure: facts, retryPolicy: llmRetryPolicyOf(stream) }
|
||||
}
|
||||
throw error
|
||||
}
|
||||
@@ -493,7 +499,7 @@ export class ReactLoopAgent implements Agent {
|
||||
const finish = assembler.finish
|
||||
if (finish.kind === 'error' || finish.kind === 'aborted') {
|
||||
const error = new LlmError(finish.failure.message, finish.failure.code, finish.failure)
|
||||
return { kind: 'request-failed', error, failure: finish.failure }
|
||||
return { kind: 'request-failed', error, failure: finish.failure, retryPolicy: llmRetryPolicyOf(stream) }
|
||||
}
|
||||
|
||||
// Truncated (max-tokens) output cannot owe tool calls.
|
||||
|
||||
@@ -285,7 +285,9 @@ describe('request-error action edges', () => {
|
||||
])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('retry-raced'), { provider: 'mock', model: 'mock' })
|
||||
ctx.on('agent/request-error', async (subject, _turn, _step, _error, _failure, signal, next) => {
|
||||
ctx.on('agent/request-error', async (
|
||||
subject, _turn, _step, _error, _failure, _priorFailures, _retryPolicy, signal, next,
|
||||
) => {
|
||||
await next()
|
||||
subject.cancel({ kind: 'user' })
|
||||
expect(signal.aborted).toBe(true)
|
||||
|
||||
@@ -3,7 +3,7 @@ import { Context } from 'cordis'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import LlmService, { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmFailure, ResolvedRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
@@ -55,7 +55,13 @@ describe('agent/request-error', () => {
|
||||
])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create(SessionId('request-error-retry'), { provider: 'mock', model: 'mock' })
|
||||
const seen: { turn: number; step: number; failure: LlmFailure }[] = []
|
||||
const seen: {
|
||||
turn: number
|
||||
step: number
|
||||
failure: LlmFailure
|
||||
priorFailures: readonly LlmFailure[]
|
||||
retryPolicy: ResolvedRetryPolicy | undefined
|
||||
}[] = []
|
||||
const statuses: string[] = []
|
||||
const settledTurns: number[] = []
|
||||
ctx.on('agent/status', (subject, status) => {
|
||||
@@ -64,13 +70,15 @@ describe('agent/request-error', () => {
|
||||
ctx.on('agent/settled', (subject, turn) => {
|
||||
if (subject === agent) settledTurns.push(turn)
|
||||
})
|
||||
ctx.on('agent/request-error', async (subject, turn, step, _error, failure) => {
|
||||
ctx.on('agent/request-error', async (
|
||||
subject, turn, step, _error, failure, priorFailures, retryPolicy,
|
||||
) => {
|
||||
expect(subject).toBe(agent)
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'step/end',
|
||||
data: { turn, step },
|
||||
})
|
||||
seen.push({ turn, step, failure })
|
||||
seen.push({ turn, step, failure, priorFailures, retryPolicy })
|
||||
return { kind: 'retry' }
|
||||
})
|
||||
|
||||
@@ -99,6 +107,12 @@ describe('agent/request-error', () => {
|
||||
{ kind: 'retry' },
|
||||
{ kind: 'retry' },
|
||||
])
|
||||
expect(seen.map(item => item.priorFailures.map(failure => failure.code)))
|
||||
.toEqual([[], ['RATE_LIMIT']])
|
||||
expect(seen.map(item => item.retryPolicy)).toEqual([
|
||||
expect.objectContaining({ mode: 'normal' }),
|
||||
expect.objectContaining({ mode: 'normal' }),
|
||||
])
|
||||
expect(statuses).toEqual(['running', 'idle'])
|
||||
expect(settledTurns).toEqual([3])
|
||||
})
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
import type { Context } from 'cordis'
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { Scoped } from '@deepseek-ai/dsh-scope'
|
||||
import type { ContentBlock, LlmCallConfig, LlmFailure, MessageSource } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, LlmCallConfig, LlmFailure, MessageSource, ResolvedRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type { Session, SessionId, UserMessageData } from '@deepseek-ai/dsh-session'
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
declare module '@deepseek-ai/dsh-system-prompt' {
|
||||
@@ -370,11 +370,15 @@ declare module 'cordis' {
|
||||
* @param step - the failed step number.
|
||||
* @param error - the original model-request failure.
|
||||
* @param failure - serializable facts normalized at the final adapter boundary.
|
||||
* @param priorFailures - immutable failures that already authorized another
|
||||
* retry turn in this consecutive sequence.
|
||||
* @param retryPolicy - immutable policy of the adapter registration that served
|
||||
* the failed request, or `undefined` if no final adapter served it.
|
||||
* @param signal - the turn abort signal.
|
||||
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
||||
* @mode waterfall
|
||||
*/
|
||||
'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, signal: AbortSignal, next: () => Promise<RequestErrorAction>): Promise<RequestErrorAction>
|
||||
'agent/request-error'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, error: RequestError, failure: LlmFailure, priorFailures: readonly LlmFailure[], retryPolicy: ResolvedRetryPolicy | undefined, signal: AbortSignal, next: () => Promise<RequestErrorAction>): Promise<RequestErrorAction>
|
||||
/**
|
||||
* The turn is about to close: the model owes no response (no live tool
|
||||
* calls, no fresh steering). Awaited before the boundary commits — a
|
||||
|
||||
@@ -55,6 +55,8 @@ describe('scoped-dispatch invariants', () => {
|
||||
1,
|
||||
new Error('request'),
|
||||
{ message: 'request', code: 'UNKNOWN' },
|
||||
[],
|
||||
undefined,
|
||||
signal,
|
||||
() => Promise.resolve(undefined),
|
||||
],
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: bbc41f1e0aa0c98a6e70ee54357675f1d7f05dbc
|
||||
README.zh.md: 03e1246d5358138c633d2b19a9c186a3beec5a1e
|
||||
# pnpm run verify-translation-pairing --write packages/examples/acp-demo/README.md
|
||||
README.md: 395ab230146568989c4e6d1361218efb72d857e7
|
||||
README.zh.md: 1dfd2d99f4ab80953df77feba19649b934775d68
|
||||
@@ -36,7 +36,6 @@ The app does not install commands, user interaction, session navigation, configu
|
||||
| `toolBash` | owner defaults | Model-facing bash tool config. |
|
||||
| `toolTasks` | owner defaults | Generic background-task control config, or `false`. |
|
||||
| `goals` | owner defaults | Persisted same-session goal domain and model tools, or `false`. |
|
||||
| `llmRetry` | owner defaults | Bounded transient model-request retry policy. |
|
||||
|
||||
The shipped [`examples/acp-agent/cordis.yml`](../../../examples/acp-agent/cordis.yml) adds the DeepSeek adapter, sandboxed bash and filesystem providers, one-shot approval policy, compaction, subagents, workflows, hooks, and model-facing tools. The app supplies the derived session-query index, while the model-facing query consumer remains an explicit leaf opt-in. Snapshot overlays replace only nondeterministic providers or policy values.
|
||||
|
||||
|
||||
@@ -36,7 +36,6 @@ ACP 自动化服务器应用:默认 agent 主干、客户端通过 [`@deepseek
|
||||
| `toolBash` | 拥有者默认值 | 面向模型的 bash 工具配置。 |
|
||||
| `toolTasks` | 拥有者默认值 | 通用后台任务控制配置,或 `false`。 |
|
||||
| `goals` | 拥有者默认值 | 持久的同会话目标领域与模型工具,或 `false`。 |
|
||||
| `llmRetry` | 拥有者默认值 | 有界的瞬时模型请求重试策略。 |
|
||||
|
||||
已交付的 [`examples/acp-agent/cordis.yml`](../../../examples/acp-agent/cordis.yml) 添加 DeepSeek 适配器、沙箱化 bash 与文件系统提供方、一次性批准策略、压缩、subagent、工作流、钩子,以及面向模型的工具。应用提供派生会话查询索引,而面向模型的查询消费方仍由叶节点显式选用。快照 overlay 只替换非确定性提供方或策略值。
|
||||
|
||||
|
||||
@@ -69,8 +69,6 @@ export interface Config {
|
||||
toolTasks?: NonNullable<agentCore.Config['toolTasks']>
|
||||
/** Persisted same-session goals; owner defaults enable them, or false disables the stack and tools. */
|
||||
goals?: agentCore.GoalConfig | false
|
||||
/** Bounded transient model-request retry policy forwarded through agent-core. */
|
||||
llmRetry?: NonNullable<agentCore.Config['llmRetry']>
|
||||
}
|
||||
|
||||
// Each front door owns a complete, directly readable config schema; extracting
|
||||
@@ -96,7 +94,6 @@ export const Config: z<Config> = z.object({
|
||||
toolBash: agentCore.ToolBashConfigSchema,
|
||||
toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]),
|
||||
goals: z.union([z.const(false), agentCore.GoalConfigSchema]),
|
||||
llmRetry: agentCore.LlmRetryConfigSchema,
|
||||
})
|
||||
/* jscpd:ignore-end */
|
||||
|
||||
|
||||
@@ -210,6 +210,7 @@ describe.skipIf(!existsSync(acpBin))('dsh-acp-demo BUILT bin (node lib/bin.js, n
|
||||
expect(code).not.toBe(0)
|
||||
expect(stderr).toContain('config file not found')
|
||||
}, 30_000)
|
||||
|
||||
})
|
||||
|
||||
/** Spawn the built acp bin against `configArg` (stdin closed at EOF) and resolve with its exit code + stderr. */
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 32874bf2839c194572ddde8c4ed007297f763ccc
|
||||
README.zh.md: 57a06a00203b8e67f2f33c87d7450d1a0789d7e6
|
||||
# pnpm run verify-translation-pairing --write packages/examples/agent-spine-demo/README.md
|
||||
README.md: 359e7153be2f480ba3fea4b06782acdc9f89ebb9
|
||||
README.zh.md: 57fec3f32f5bbc8f3d82ff8971d36d376d722753
|
||||
@@ -23,7 +23,7 @@ Read this package for the whole plugin tree and its composition order.
|
||||
@deepseek-ai/dsh-goal optional persisted same-session goal domain
|
||||
@deepseek-ai/dsh-tool-goal optional model-facing goal controls
|
||||
@deepseek-ai/dsh-goal-session optional same-session goal-round driver
|
||||
@deepseek-ai/dsh-llm-retry bounded transient request retry policy
|
||||
@deepseek-ai/dsh-llm-retry provider-routed request retry policy
|
||||
@deepseek-ai/dsh-tasks-local generic background-task registry
|
||||
@deepseek-ai/dsh-invariants configurable invariant registry service
|
||||
@deepseek-ai/dsh-session/invariant
|
||||
@@ -55,11 +55,11 @@ This is the [interface/implementation/consumer seam](../../../.agents/notes/impl
|
||||
|
||||
```ts
|
||||
import type { Config } from '@deepseek-ai/dsh-agent-spine-demo'
|
||||
// { agents?, maxParallelToolCalls?, persona?, toolOrder?, tools?, dshHome?, sessionTitle?, skills?, workspaceContext, toolBash?, toolTasks?, goals?, invariants?, llmRetry? }
|
||||
// { agents?, maxParallelToolCalls?, persona?, toolOrder?, tools?, dshHome?, sessionTitle?, skills?, workspaceContext, toolBash?, toolTasks?, goals?, invariants? }
|
||||
// workspaceContext requires { maxBytes } or false; the other owner schemas supply defaults.
|
||||
```
|
||||
|
||||
The bundle FORWARDS each field to the child that owns it: `agents` and `maxParallelToolCalls` to `agent-loop` (`agents` defaults to `[]`; the cap defaults there), so each app supplies its own pre-created agents — TUI and headless apps pre-create `main`, while the ACP app creates agents on demand at `session/new`; `llmRetry` to the bounded retry policy; `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; `sessionTitle` to the fallback title service; `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer; the required `workspaceContext` choice to `dsh-workspace-context` (`{ maxBytes }` enables loading and `false` disables it); `invariants` to the invariant service; and `toolBash`/`toolTasks` to the two model-facing tool plugins the bundle owns. Omitted `sessionTitle` uses the explicit example policy of 5 words, 40 fallback bytes, and 80 accepted-title bytes. A `goals` object opts into the persisted domain, model tools, and same-session driver while forwarding `goals.domain` and `goals.tool` to their owners; omission or `false` leaves the stack absent so headless callers retain one-turn settlement. Set `skills.enabled: false` to omit both the local provider and model-facing skill tool, and set `toolTasks: false` to retain the task service for foreground producers without exposing `task_output` / `task_list` / `task_kill`. It resolves `dshHome` once through [`@deepseek-ai/dsh-paths`](../../util/paths/README.md) and forwards that absolute value to tool-bash's managed environment and enabled local skill discovery. An absent top-level `dshHome` adopts `skills.local.dshHome`; supplying both with different resolved paths fails loudly. `toolBash.enableRunInBackground` controls only the bash producer; independently loaded producers keep their own config. Workspace instructions register before the skill catalog so their session-prefix message renders first. App packages use `pickSpineConfig()` to copy only these bundle-owned fields.
|
||||
The bundle FORWARDS each field to the child that owns it: `agents` and `maxParallelToolCalls` to `agent-loop` (`agents` defaults to `[]`; the cap defaults there), so each app supplies its own pre-created agents — TUI and headless apps pre-create `main`, while the ACP app creates agents on demand at `session/new`; `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; `sessionTitle` to the fallback title service; `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer; the required `workspaceContext` choice to `dsh-workspace-context` (`{ maxBytes }` enables loading and `false` disables it); `invariants` to the invariant service; and `toolBash`/`toolTasks` to the two model-facing tool plugins the bundle owns. It always mounts `dsh-llm-retry`, while each leaf adapter owns its nested `retryPolicy`. Omitted `sessionTitle` uses the explicit example policy of 5 words, 40 fallback bytes, and 80 accepted-title bytes. A `goals` object opts into the persisted domain, model tools, and same-session driver while forwarding `goals.domain` and `goals.tool` to their owners; omission or `false` leaves the stack absent so headless callers retain one-turn settlement. Set `skills.enabled: false` to omit both the local provider and model-facing skill tool, and set `toolTasks: false` to retain the task service for foreground producers without exposing `task_output` / `task_list` / `task_kill`. It resolves `dshHome` once through [`@deepseek-ai/dsh-paths`](../../util/paths/README.md) and forwards that absolute value to tool-bash's managed environment and enabled local skill discovery. An absent top-level `dshHome` adopts `skills.local.dshHome`; supplying both with different resolved paths fails loudly. `toolBash.enableRunInBackground` controls only the bash producer; independently loaded producers keep their own config. Workspace instructions register before the skill catalog so their session-prefix message renders first. App packages use `pickSpineConfig()` to copy only these bundle-owned fields.
|
||||
|
||||
For example, `{ invariants: { enabled: true, package_allowlist: ['^@deepseek-ai/dsh-'], package_blocklist: ['agent-loop$'] } }` keeps the package-owned companions mounted but suppresses the blocked owner. Blocklist matches override allowlist matches; see [`dsh-invariants`](../../support/invariants/README.md) for regex and lifecycle rules.
|
||||
|
||||
@@ -67,7 +67,7 @@ For example, `{ invariants: { enabled: true, package_allowlist: ['^@deepseek-ai/
|
||||
|
||||
A YAML include can deduplicate config but cannot own a bin or provide front-door defaults. The ACP app package makes protocol-pure stdout wiring the default, though a leaf can still add an unsafe logger. Bundle children register services in the root isolate-keyed store, so injected leaf siblings see them without load-order coupling.
|
||||
|
||||
The bounded retry policy may repeat a transiently failed request in a new numbered step. Retry status and failed partial chunks stay outside model history, each provider attempt can still incur billing, front doors derive usage across every logged step, and the reconstructed request preserves the prior prefix for provider cache reuse.
|
||||
The retry policy may repeat a failed request in a new numbered step. Retry status, provider errors, and failed partial chunks stay outside model history; each provider attempt can still incur billing, always mode has no attempt limit, front doors derive usage across every logged step, and the reconstructed request preserves the prior prefix for provider cache reuse.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
@deepseek-ai/dsh-goal optional persisted same-session goal domain
|
||||
@deepseek-ai/dsh-tool-goal optional model-facing goal controls
|
||||
@deepseek-ai/dsh-goal-session optional same-session goal-round driver
|
||||
@deepseek-ai/dsh-llm-retry bounded transient request retry policy
|
||||
@deepseek-ai/dsh-llm-retry provider-routed request retry policy
|
||||
@deepseek-ai/dsh-tasks-local generic background-task registry
|
||||
@deepseek-ai/dsh-invariants configurable invariant registry service
|
||||
@deepseek-ai/dsh-session/invariant
|
||||
@@ -55,11 +55,11 @@
|
||||
|
||||
```ts
|
||||
import type { Config } from '@deepseek-ai/dsh-agent-spine-demo'
|
||||
// { agents?, maxParallelToolCalls?, persona?, toolOrder?, tools?, dshHome?, sessionTitle?, skills?, workspaceContext, toolBash?, toolTasks?, goals?, invariants?, llmRetry? }
|
||||
// { agents?, maxParallelToolCalls?, persona?, toolOrder?, tools?, dshHome?, sessionTitle?, skills?, workspaceContext, toolBash?, toolTasks?, goals?, invariants? }
|
||||
// workspaceContext requires { maxBytes } or false; the other owner schemas supply defaults.
|
||||
```
|
||||
|
||||
组合包将每个字段转发给拥有它的子节点:`agents` 与 `maxParallelToolCalls` 交给 `agent-loop`(`agents` 默认为 `[]`,上限在该处默认),因此每个应用提供自己的预创建 agent;TUI 和无头应用预创建 `main`,ACP 应用则在 `session/new` 按需创建 agent;`llmRetry` 交给有界重试策略;`persona` 与 `toolOrder` 交给 `dsh-system-prompt`;`tools` 交给工具注册表以配置呈现 mode;`sessionTitle` 交给后备标题服务;`skills.registry`、`skills.local` 与 `skills.tool` 分别交给 skill 注册表、本地提供方和面向模型的消费方;必填的 `workspaceContext` 选择交给 `dsh-workspace-context`(`{ maxBytes }` 启用加载,`false` 禁用);`invariants` 交给不变式服务;`toolBash`/`toolTasks` 交给组合包拥有的两个面向模型工具插件。省略 `sessionTitle` 时采用显式示例策略:5 个词、40 个后备字节、80 个可接受标题字节。`goals` 对象会选用持久领域、模型工具和同会话驱动器,并将 `goals.domain` 与 `goals.tool` 转发给各自拥有者;省略或设为 `false` 会让整个栈缺席,使无头调用方继续以一轮结算。设置 `skills.enabled: false` 会同时省略本地提供方和面向模型的 skill 工具;设置 `toolTasks: false` 会保留供前台生产方使用的任务服务,但不公开 `task_output`/`task_list`/`task_kill`。它对 `dshHome` 只解析一次,解析通过 [`@deepseek-ai/dsh-paths`](../../util/paths/README.md) 完成,并将所得绝对值转发给 tool-bash 的托管环境和已启用的本地 skill 发现。顶层 `dshHome` 缺席时采用 `skills.local.dshHome`;两者同时提供但解析后的路径不同会明确失败。`toolBash.enableRunInBackground` 只控制 bash 生产方;独立加载的生产方保留各自配置。Workspace 指令先于 skill 目录注册,因此其会话前缀消息先渲染。应用包使用 `pickSpineConfig()`,只复制这些由组合包拥有的字段。
|
||||
组合包将每个字段转发给拥有它的子节点:`agents` 与 `maxParallelToolCalls` 交给 `agent-loop`(`agents` 默认为 `[]`,上限在该处默认),因此每个应用提供自己的预创建 agent;TUI 和无头应用预创建 `main`,ACP 应用则在 `session/new` 按需创建 agent;`persona` 与 `toolOrder` 交给 `dsh-system-prompt`;`tools` 交给工具注册表以配置呈现 mode;`sessionTitle` 交给后备标题服务;`skills.registry`、`skills.local` 与 `skills.tool` 分别交给 skill 注册表、本地提供方和面向模型的消费方;必填的 `workspaceContext` 选择交给 `dsh-workspace-context`(`{ maxBytes }` 启用加载,`false` 禁用);`invariants` 交给不变式服务;`toolBash`/`toolTasks` 交给组合包拥有的两个面向模型工具插件。组合包始终挂载 `dsh-llm-retry`,而每个叶节点适配器拥有自己的嵌套 `retryPolicy`。省略 `sessionTitle` 时采用显式示例策略:5 个词、40 个后备字节、80 个可接受标题字节。`goals` 对象会选用持久领域、模型工具和同会话驱动器,并将 `goals.domain` 与 `goals.tool` 转发给各自拥有者;省略或设为 `false` 会让整个栈缺席,使无头调用方继续以一轮结算。设置 `skills.enabled: false` 会同时省略本地提供方和面向模型的 skill 工具;设置 `toolTasks: false` 会保留供前台生产方使用的任务服务,但不公开 `task_output`/`task_list`/`task_kill`。它对 `dshHome` 只解析一次,解析通过 [`@deepseek-ai/dsh-paths`](../../util/paths/README.md) 完成,并将所得绝对值转发给 tool-bash 的托管环境和已启用的本地 skill 发现。顶层 `dshHome` 缺席时采用 `skills.local.dshHome`;两者同时提供但解析后的路径不同会明确失败。`toolBash.enableRunInBackground` 只控制 bash 生产方;独立加载的生产方保留各自配置。Workspace 指令先于 skill 目录注册,因此其会话前缀消息先渲染。应用包使用 `pickSpineConfig()`,只复制这些由组合包拥有的字段。
|
||||
|
||||
例如,`{ invariants: { enabled: true, package_allowlist: ['^@deepseek-ai/dsh-'], package_blocklist: ['agent-loop$'] } }` 会让包拥有的配套插件保持挂载,但抑制被阻止的拥有者。Blocklist 匹配优先于 allowlist 匹配;正则表达式与生命周期规则见 [`dsh-invariants`](../../support/invariants/README.md)。
|
||||
|
||||
@@ -67,7 +67,7 @@ import type { Config } from '@deepseek-ai/dsh-agent-spine-demo'
|
||||
|
||||
YAML include 可以去重配置,却无法拥有 bin 或提供前端入口默认值。ACP 应用包默认接出协议纯净的 stdout,但叶节点仍可添加不安全的 logger。组合包子节点把服务注册到根 isolate-keyed store,因此注入这些服务的叶节点同级插件无需依赖加载顺序即可看到它们。
|
||||
|
||||
有界重试策略可能在新的编号步骤中重复瞬时失败的请求。重试状态和失败的部分 chunk 不进入模型历史;每次提供方尝试仍可能产生计费;前端入口从所有已记录步骤推导用量;重建的请求保留先前前缀,以便复用提供方 cache。
|
||||
重试策略可能在新的编号步骤中重复失败的请求。重试状态、提供方错误和失败的部分 chunk 不进入模型历史;每次提供方尝试仍可能产生计费;always mode 没有尝试次数上限;前端入口从所有已记录步骤推导用量;重建的请求保留先前前缀,以便复用提供方 cache。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-agent-spine-demo",
|
||||
"description": "The default executor-less/UI-less agent spine with fallback session titles, bounded retry, and optional persisted goals",
|
||||
"description": "The default executor-less/UI-less agent spine with fallback session titles, provider-routed retry, and optional persisted goals",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
|
||||
@@ -74,8 +74,9 @@ export interface GoalConfig {
|
||||
* `dshHome` to bash environment and local skill discovery, `sessionTitle` to
|
||||
* the fallback title service, `skills` to the
|
||||
* skill registry/local provider/tool consumer, `workspaceContext` to the
|
||||
* workspace-context loader, `llmRetry` to the bounded request-recovery policy,
|
||||
* and `toolBash`/`toolTasks` to the model-facing tool plugins this bundle owns.
|
||||
* workspace-context loader, and `toolBash`/`toolTasks` to the model-facing tool
|
||||
* plugins this bundle owns. Provider adapters own their `retryPolicy`; this
|
||||
* bundle always mounts its executor.
|
||||
* `goals` opts into and configures the persisted goal domain plus its model tool
|
||||
* and same-session driver; `invariants` configures global and package-filtered
|
||||
* relational checks. Owner schemas supply defaults for optional input;
|
||||
@@ -111,8 +112,6 @@ export interface Config {
|
||||
invariants?: InvariantConfig
|
||||
/** Opt-in persisted same-session goal stack; set false or omit to leave it unmounted. */
|
||||
goals?: GoalConfig | false
|
||||
/** Bounded transient model-request retry policy. */
|
||||
llmRetry?: llmRetry.Config
|
||||
}
|
||||
|
||||
/** The skill config schema exported for app packages that forward `skills`. */
|
||||
@@ -139,9 +138,6 @@ export const GoalConfigSchema: z<GoalConfig> = z.object({
|
||||
tool: toolGoal.Config,
|
||||
})
|
||||
|
||||
/** The bounded LLM retry schema exported for app packages that forward `llmRetry`. */
|
||||
export const LlmRetryConfigSchema: z<llmRetry.Config> = llmRetry.Config
|
||||
|
||||
/** Intersect the owners' schemas so validation + defaulting stay identical. */
|
||||
export const Config = z.intersect([
|
||||
AgentLoop.Config,
|
||||
@@ -156,8 +152,7 @@ export const Config = z.intersect([
|
||||
toolTasks: z.union([z.const(false), ToolTasksConfigSchema]),
|
||||
invariants: InvariantService.Config,
|
||||
goals: z.union([z.const(false), GoalConfigSchema]),
|
||||
llmRetry: LlmRetryConfigSchema,
|
||||
}) as unknown as z<Pick<Config, 'tools' | 'dshHome' | 'sessionTitle' | 'skills' | 'workspaceContext' | 'toolBash' | 'toolTasks' | 'invariants' | 'goals' | 'llmRetry'>>,
|
||||
}) as unknown as z<Pick<Config, 'tools' | 'dshHome' | 'sessionTitle' | 'skills' | 'workspaceContext' | 'toolBash' | 'toolTasks' | 'invariants' | 'goals'>>,
|
||||
]) as unknown as z<Config>
|
||||
|
||||
/**
|
||||
@@ -179,7 +174,6 @@ export function pickSpineConfig(config: Omit<Config, 'agents'>): Omit<Config, 'a
|
||||
...config.toolTasks !== undefined ? { toolTasks: config.toolTasks } : {},
|
||||
...config.invariants !== undefined ? { invariants: config.invariants } : {},
|
||||
...config.goals !== undefined ? { goals: config.goals } : {},
|
||||
...config.llmRetry !== undefined ? { llmRetry: config.llmRetry } : {},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -217,7 +211,7 @@ export function apply(ctx: Context, config: Config): void {
|
||||
ctx.plugin(SkillLocal, Object.assign({}, config.skills?.local, { dshHome }))
|
||||
}
|
||||
ctx.plugin(AgentRegistry)
|
||||
ctx.plugin(llmRetry, config.llmRetry ?? {})
|
||||
ctx.plugin(llmRetry)
|
||||
if (config.goals !== undefined && config.goals !== false) {
|
||||
ctx.plugin(GoalService, config.goals.domain ?? {})
|
||||
ctx.plugin(toolGoal, config.goals.tool ?? {})
|
||||
|
||||
@@ -10,7 +10,16 @@ import { agentEvents, type Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import LocalFileSystem from '@deepseek-ai/dsh-fs-local'
|
||||
import { MockAdapter, textResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
|
||||
import { CallId, LlmAdapter, LlmError, type GenerateOptions, type Message, type StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
CallId,
|
||||
LlmAdapter,
|
||||
LlmError,
|
||||
resolveRetryPolicy,
|
||||
type GenerateOptions,
|
||||
type Message,
|
||||
type ResolvedRetryPolicy,
|
||||
type StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import type { ToolExecution } from '@deepseek-ai/dsh-tools'
|
||||
import * as sessionInvariant from '@deepseek-ai/dsh-session/invariant'
|
||||
import * as agentInvariant from '@deepseek-ai/dsh-agent/invariant'
|
||||
@@ -106,6 +115,15 @@ function messageText(message: Message | undefined): string {
|
||||
|
||||
class TransientOnceAdapter extends LlmAdapter {
|
||||
requests = 0
|
||||
private readonly retryPolicy = resolveRetryPolicy({
|
||||
mode: 'normal',
|
||||
maxRetries: 1,
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
}, 'agent-spine test provider retryPolicy')
|
||||
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests += 1
|
||||
@@ -209,15 +227,7 @@ describe('dsh-agent-spine-demo bundle', () => {
|
||||
|
||||
it('loads and configures bounded request recovery for every bundled front door', async () => {
|
||||
const adapter = new TransientOnceAdapter()
|
||||
const ctx = await mount({
|
||||
workspaceContext: false,
|
||||
llmRetry: {
|
||||
maxTransientRetries: 1,
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
jitterRatio: 0,
|
||||
},
|
||||
})
|
||||
const ctx = await mount({ workspaceContext: false })
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const handle = await ctx.agents.create({
|
||||
sessionId: SessionId('bundled-retry-session'),
|
||||
@@ -232,7 +242,7 @@ describe('dsh-agent-spine-demo bundle', () => {
|
||||
const retryEvents = handle.agent.session.events.filter(event => event.type === 'llm/retry')
|
||||
expect(retryEvents).toHaveLength(1)
|
||||
expect(retryEvents[0]?.data.retry).toBe(1)
|
||||
expect(retryEvents[0]?.data.maxRetries).toBe(1)
|
||||
expect(retryEvents[0]?.data).toMatchObject({ provider: 'mock', mode: 'normal', maxRetries: 1 })
|
||||
expect(handle.agent.session.events.find(event => event.type === 'session/title')?.data.title).toBe('recover')
|
||||
expect(messageText(handle.agent.session.deriveMessages().at(-1))).toBe('recovered by bundled policy')
|
||||
await handle.dispose()
|
||||
@@ -513,7 +523,6 @@ describe('dsh-agent-spine-demo bundle', () => {
|
||||
toolBash: { enableRunInBackground: false },
|
||||
toolTasks: false as const,
|
||||
invariants: { enabled: false },
|
||||
llmRetry: { maxTransientRetries: 1, jitterRatio: 0 },
|
||||
}
|
||||
|
||||
expect(agentCore.pickSpineConfig(appConfig)).toEqual({
|
||||
@@ -527,7 +536,6 @@ describe('dsh-agent-spine-demo bundle', () => {
|
||||
toolBash: appConfig.toolBash,
|
||||
toolTasks: appConfig.toolTasks,
|
||||
invariants: appConfig.invariants,
|
||||
llmRetry: appConfig.llmRetry,
|
||||
})
|
||||
expect(agentCore.pickSpineConfig({ workspaceContext: false })).toEqual({ workspaceContext: false })
|
||||
})
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 4e8e5388e17ab2879582286adf593c72fb2cf78f
|
||||
README.zh.md: 3cad72071184403a78b7906d637bba67bea1a64a
|
||||
# pnpm run verify-translation-pairing --write packages/examples/cli-demo/README.md
|
||||
README.md: b8f2bde962738a1a23f0e57218ab0f90e8e0b705
|
||||
README.zh.md: 322ba3fb3b253d867832534bd33f65df3a5b8d37
|
||||
@@ -21,7 +21,6 @@ The package mounts no console logger, interactive UI, user-interaction service,
|
||||
| `skills` | owner defaults | skill registry, local provider, and model-facing skill tool |
|
||||
| `toolBash` | owner defaults | model-facing bash config, including this producer's background opt-in |
|
||||
| `toolTasks` | owner defaults | generic `task_output` wait bounds |
|
||||
| `llmRetry` | owner defaults | bounded transient model-request retry policy |
|
||||
| `persistenceRoot` | `./.sessions` | JSONL session root |
|
||||
| `persistenceCompression` | `'zstd'` | JSONL artifact encoding (`'zstd'` or raw `'none'`) |
|
||||
| `workspaceContext` | required | workspace-instruction byte budget, or `false` to disable loading |
|
||||
|
||||
@@ -21,7 +21,6 @@
|
||||
| `skills` | 拥有者默认值 | Skill 注册表、本地提供方和面向模型的 skill 工具 |
|
||||
| `toolBash` | 拥有者默认值 | 面向模型的 bash 配置,包括此生产方对后台任务的选用 |
|
||||
| `toolTasks` | 拥有者默认值 | 通用 `task_output` 等待边界 |
|
||||
| `llmRetry` | 拥有者默认值 | 有界的瞬时模型请求重试策略 |
|
||||
| `persistenceRoot` | `./.sessions` | JSONL 会话根目录 |
|
||||
| `persistenceCompression` | `'zstd'` | JSONL 工件编码(`'zstd'` 或原始 `'none'`) |
|
||||
| `workspaceContext` | 必填 | Workspace 指令字节预算,或以 `false` 禁用加载 |
|
||||
|
||||
@@ -50,8 +50,6 @@ export interface Config {
|
||||
toolBash?: NonNullable<agentCore.Config['toolBash']>
|
||||
/** Generic background-task control-tool config forwarded through agent-spine-demo. */
|
||||
toolTasks?: NonNullable<agentCore.Config['toolTasks']>
|
||||
/** Bounded transient model-request retry policy forwarded through agent-spine-demo. */
|
||||
llmRetry?: NonNullable<agentCore.Config['llmRetry']>
|
||||
/** Controls automatic AGENTS.md/CLAUDE.md loading; configure a byte budget or set `false`. */
|
||||
workspaceContext: agentCore.Config['workspaceContext']
|
||||
}
|
||||
@@ -74,7 +72,6 @@ export const Config: z<Config> = z.object({
|
||||
tools: ToolRegistry.Config,
|
||||
toolBash: agentCore.ToolBashConfigSchema,
|
||||
toolTasks: z.union([z.const(false), agentCore.ToolTasksConfigSchema]),
|
||||
llmRetry: agentCore.LlmRetryConfigSchema,
|
||||
workspaceContext: z.union([z.const(false), workspaceContext.Config]).required(),
|
||||
})
|
||||
/* jscpd:ignore-end */
|
||||
|
||||
@@ -3,7 +3,15 @@ import { tmpdir } from 'node:os'
|
||||
import { join, resolve } from 'node:path'
|
||||
import { Context } from 'cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { CallId, LlmAdapter, type GenerateOptions, type StreamChunk, type TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
CallId,
|
||||
LlmAdapter,
|
||||
resolveRetryPolicy,
|
||||
type GenerateOptions,
|
||||
type ResolvedRetryPolicy,
|
||||
type StreamChunk,
|
||||
type TokenUsage,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { SessionId, type SessionEvent, type TurnEndReason } from '@deepseek-ai/dsh-session'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import * as cliDemo from '../src/index.ts'
|
||||
@@ -20,11 +28,19 @@ type ScriptEntry = readonly StreamChunk[] | 'hang'
|
||||
class ScriptedAdapter extends LlmAdapter {
|
||||
readonly requests: GenerateOptions[] = []
|
||||
private cursor = 0
|
||||
private readonly retryPolicy = resolveRetryPolicy({
|
||||
mode: 'normal',
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
}, 'cli test provider retryPolicy')
|
||||
|
||||
constructor(private readonly script: readonly ScriptEntry[]) {
|
||||
super()
|
||||
}
|
||||
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests.push(options)
|
||||
const entry = this.script[this.cursor++]
|
||||
@@ -107,7 +123,6 @@ async function harness(script: readonly ScriptEntry[]): Promise<Harness> {
|
||||
persistenceRoot: root,
|
||||
skills: { local: { dshHome: join(skillHome, '.dsh'), agentsHome: join(skillHome, '.agents') } },
|
||||
workspaceContext: false,
|
||||
llmRetry: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
})
|
||||
await new Promise(resolve => setTimeout(resolve, 80))
|
||||
ctx.llm.registerAdapter(['mock'], new ScriptedAdapter(script))
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { join } from 'node:path'
|
||||
import type { Context } from 'cordis'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import { TOOL_ORDER_REST } from '@deepseek-ai/dsh-system-prompt'
|
||||
import * as tuiAgent from '../src/index.ts'
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 13a04aa9f73fec5824069644449009989d6fd924
|
||||
README.zh.md: 3e417c5f8be1f7831b99940c2a4aec815dc2c5b6
|
||||
# pnpm run verify-translation-pairing --write packages/llm/README.md
|
||||
README.md: 66b7beabd73cc3fec7230f209a9da0da48a37c95
|
||||
README.zh.md: 48c54358ce3e8e21e33a6ef5b75a7e095b6581d5
|
||||
@@ -8,8 +8,8 @@ The LLM seam and its provider adapters. The interface package (`llm`) owns the a
|
||||
|---|---|---|
|
||||
| `llm/` | Abstract LLM service + content-block vocabulary + chunk assembler | `ctx.llm` |
|
||||
| `token-meter/` | Replay-aware request and surface token measurement | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | Bounded transient request retry policy | (listens to `agent/request-error`) |
|
||||
| `llm-retry/` | Exact-provider normal or unbounded request retry policy | (listens to `agent/request-error`) |
|
||||
| `llm-deepseek/` | DeepSeek API adapter (direct fetch + eventsource-parser SSE) | (registers on `ctx.llm`) |
|
||||
| `llm-pi-ai/` | Multi-provider adapter via `@earendil-works/pi-ai` | (registers on `ctx.llm`) |
|
||||
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter optionally resolves exact provider/model context capacity; the token meter remains model-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the interface or consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
The interface lives at `llm/llm/`; adapters, retry policy, and the reusable token meter are flat siblings under the group. Requests route by `provider`, while `model` is passed through to the selected adapter. The route-owning adapter supplies retry policy and resolves available exact-model identity, context capacity, and reasoning metadata; the retry executor and token meter remain provider-agnostic. A new provider adapter registers one or more provider routes on `ctx.llm` without touching the consumers. See [twin LLM adapters](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md) for the two shipping implementations, the [replay token meter Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md) for measurement ownership, and the [routed model context Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md) for capacity and compaction-policy ownership.
|
||||
@@ -8,8 +8,8 @@ LLM seam 及其提供方适配器。接口包(`llm`)拥有抽象服务、内
|
||||
|---|---|---|
|
||||
| `llm/` | 抽象 LLM 服务 + 内容块词汇 + 分片组装器 | `ctx.llm` |
|
||||
| `token-meter/` | 感知回放的请求与表层 token 测量 | `ctx.tokenMeter` |
|
||||
| `llm-retry/` | 有界的暂时性请求重试策略 | (监听 `agent/request-error`) |
|
||||
| `llm-retry/` | 确切提供方的 normal 或无界请求重试策略 | (监听 `agent/request-error`) |
|
||||
| `llm-deepseek/` | DeepSeek API 适配器(直接 fetch + eventsource-parser SSE) | (注册到 `ctx.llm`) |
|
||||
| `llm-pi-ai/` | 通过 `@earendil-works/pi-ai` 实现的多提供方适配器 | (注册到 `ctx.llm`) |
|
||||
|
||||
接口位于 `llm/llm/`;适配器、重试策略和可复用的 token 计量器都是该分组下的扁平兄弟包。请求按 `provider` 路由,而 `model` 会原样传给选中的适配器。拥有路由的适配器可以解析精确的提供方/模型上下文容量;token 计量器仍与模型无关。新的提供方适配器只需在 `ctx.llm` 上注册一个或多个提供方路由,无需改动接口或消费方。两个已交付实现见[双生 LLM 适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md),测量归属见[回放 token 计量器 Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md),容量与压缩策略归属见[路由模型上下文 Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md)。
|
||||
接口位于 `llm/llm/`;适配器、重试策略和可复用的 token 计量器都是该分组下的扁平兄弟包。请求按 `provider` 路由,而 `model` 会原样传给选中的适配器。拥有路由的适配器提供重试策略,并解析可用的确切模型身份、上下文容量和推理元数据;重试执行器与 token 计量器仍与提供方无关。新的提供方适配器只需在 `ctx.llm` 上注册一个或多个提供方路由,无需改动消费方。两个已交付实现见[双生 LLM 适配器](../../.agents/notes/implemented/architecture/2026-06-13-twin-llm-adapters.md),测量归属见[回放 token 计量器 Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-replay-token-meter-service.md),容量与压缩策略归属见[路由模型上下文 Agent Note](../../.agents/notes/implemented/architecture/2026-07-20-routed-model-context-and-compaction-policy.md)。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 4358620295547248ca87c42e07022c5eab0c947b
|
||||
README.zh.md: 4ecc5dd2e5980751ca6e724b5041efefc8114077
|
||||
README.md: a7f2fcb9c21d45a95fc81abd3dc1424d4336966d
|
||||
README.zh.md: bca56e700c0067b644adb4d1460a47901db28209
|
||||
@@ -19,6 +19,12 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; off | high | max — omitted ⇒ high
|
||||
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
|
||||
retryPolicy: # optional; omission uses bounded normal defaults
|
||||
mode: always # normal | always
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
defaultContextWindow: 256000 # optional positive-integer fallback for models without an exact value
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
@@ -28,7 +34,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
contextWindow: 64000
|
||||
```
|
||||
|
||||
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for UI selectors and deployment introspection, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
The plugin registers the single provider route `deepseek` together with its resolved `retryPolicy`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for UI selectors and deployment introspection, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. When neither value exists, `context` is absent without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
@@ -36,7 +42,7 @@ The same exact-model result exposes ordered `off`, `high`, and `max` efforts und
|
||||
|
||||
`thinking: disabled` is a deployment lock that publishes only `off` with `off` as its default. Omitting `reasoningEffort` or configuring it as `off` is valid; configuring `high` or `max` fails plugin loading, and a direct per-request attempt to enable thinking fails before network I/O. A request with `GenerateOptions.purpose: 'session-title'` also forces thinking disabled and omits the already-resolved effort, reserving its bounded output for visible title text without changing conversation or compaction defaults.
|
||||
|
||||
`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; agent-level retry is a separate plugin policy.
|
||||
`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; it registers the configured policy as provider metadata, and `dsh-llm-retry` separately executes it at durable agent-step boundaries.
|
||||
|
||||
## App attribution
|
||||
|
||||
|
||||
@@ -19,6 +19,12 @@ harness LLM seam 的 DeepSeek chat-completions 适配器:直接 `fetch` + SSE
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; off | high | max — omitted ⇒ high
|
||||
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
|
||||
retryPolicy: # optional; omission uses bounded normal defaults
|
||||
mode: always # normal | always
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
defaultContextWindow: 256000 # optional positive-integer fallback for models without an exact value
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
@@ -28,7 +34,7 @@ harness LLM seam 的 DeepSeek chat-completions 适配器:直接 `fetch` + SSE
|
||||
contextWindow: 64000
|
||||
```
|
||||
|
||||
该插件注册唯一提供方路由 `deepseek`。请求使用 `provider: deepseek` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash` 和 `deepseek-v4-pro`,两者的上下文窗口均为 128,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek')` 公开给 UI selector 与部署自省,但仍只提供建议:未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。
|
||||
该插件注册唯一提供方路由 `deepseek`,同时注册解析后的 `retryPolicy`。请求使用 `provider: deepseek` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash` 和 `deepseek-v4-pro`,两者的上下文窗口均为 128,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek')` 公开给 UI selector 与部署自省,但仍只提供建议:未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。
|
||||
|
||||
`contextWindow` 对每个已配置模型都可选,不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek', model).context` 先返回精确模型值,再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。两者都不存在时,`context` 字段缺失但不会使路由失效。因此,压力敏感插件可以获得部署拥有的容量,不会将模型 selector 视为权威。为 `deepseek` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
|
||||
@@ -36,7 +42,7 @@ harness LLM seam 的 DeepSeek chat-completions 适配器:直接 `fetch` + SSE
|
||||
|
||||
`thinking: disabled` 是部署锁定:它只公布 `off`,并以 `off` 为默认值。省略 `reasoningEffort` 或将其配置为 `off` 均有效;配置 `high` 或 `max` 会使插件加载失败,直接按请求启用思考也会在网络 I/O 前失败。携带 `GenerateOptions.purpose: 'session-title'` 的请求也会强制禁用思考并省略已解析的推理强度,将有界输出保留给可见标题文本,不改变会话或压缩默认值。
|
||||
|
||||
`streamIdleTimeoutMs` 会限制每次未完成提供方读取,包括初始 `fetch`,但不计入消费方在 chunk 间花费的时间。一个稳定 abort 信号会在整个调用中达到请求与 body reader;过期会停止传输并抛出 `LlmError('TIMEOUT')`,较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用精确发起一次提供方请求;agent 级重试是独立插件策略。
|
||||
`streamIdleTimeoutMs` 会限制每次未完成提供方读取,包括初始 `fetch`,但不计入消费方在 chunk 间花费的时间。一个稳定 abort 信号会在整个调用中达到请求与 body reader;过期会停止传输并抛出 `LlmError('TIMEOUT')`,较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用精确发起一次提供方请求;它把已配置策略注册为提供方元数据,再由 `dsh-llm-retry` 在持久 agent 步骤边界单独执行该策略。
|
||||
|
||||
## 应用归因
|
||||
|
||||
|
||||
@@ -5,12 +5,14 @@
|
||||
* @module dsh-llm-deepseek/adapter
|
||||
*/
|
||||
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
GenerateOptions,
|
||||
LlmModelInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
ResolvedRetryPolicy,
|
||||
RetryPolicyConfig,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
@@ -46,6 +48,8 @@ export interface DeepSeekAdapterOptions {
|
||||
models?: readonly DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
@@ -115,6 +119,7 @@ export function httpErrorCode(status: number, error?: WireError['error']): strin
|
||||
*/
|
||||
export class DeepSeekAdapter extends LlmAdapter {
|
||||
private readonly streamIdleTimeoutMs: number
|
||||
private readonly retryPolicy: ResolvedRetryPolicy
|
||||
|
||||
constructor(private readonly options: DeepSeekAdapterOptions) {
|
||||
super()
|
||||
@@ -135,12 +140,17 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
`llm-deepseek: streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
this.retryPolicy = resolveRetryPolicy(options.retryPolicy, 'llm-deepseek: retryPolicy')
|
||||
}
|
||||
|
||||
override providerInfo(provider: string): LlmProviderInfo {
|
||||
return { id: provider, name: 'DeepSeek' }
|
||||
}
|
||||
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve((this.options.models ?? []).map(model => modelInfo(provider, model)))
|
||||
}
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type {} from '@deepseek-ai/dsh-llm'
|
||||
import { RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { DEFAULT_STREAM_IDLE_TIMEOUT_MS, DeepSeekAdapter } from './adapter.ts'
|
||||
import type { DeepSeekCatalogModel } from './adapter.ts'
|
||||
@@ -47,6 +48,8 @@ export interface Config {
|
||||
models?: DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
const catalogModel: z<DeepSeekCatalogModel> = z.object({
|
||||
@@ -64,6 +67,7 @@ export const Config: z<Config> = z.object({
|
||||
defaultContextWindow: z.number().step(1).min(1),
|
||||
models: z.array(catalogModel).default(DEFAULT_MODELS),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
retryPolicy: RetryPolicySchema,
|
||||
})
|
||||
|
||||
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
|
||||
@@ -117,5 +121,6 @@ export function apply(ctx: Context, config: Config): void {
|
||||
: { defaultContextWindow: config.defaultContextWindow },
|
||||
models: resolveModels(config.models),
|
||||
streamIdleTimeoutMs: config.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS,
|
||||
...config.retryPolicy === undefined ? {} : { retryPolicy: config.retryPolicy },
|
||||
}))
|
||||
}
|
||||
@@ -602,6 +602,26 @@ describe('plugin registration and config', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('registers retryPolicy from the provider config', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
retryPolicy: {
|
||||
mode: 'always',
|
||||
backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 },
|
||||
},
|
||||
})
|
||||
|
||||
expect(ctx.llm.providerRetryPolicy('deepseek')).toEqual({
|
||||
mode: 'always',
|
||||
initialDelayMs: 25,
|
||||
maxDelayMs: 100,
|
||||
jitterRatio: 0.2,
|
||||
})
|
||||
})
|
||||
|
||||
it('owns the deepseek provider and advertises the default models', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -908,4 +928,16 @@ describe('plugin registration and config', () => {
|
||||
streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1,
|
||||
})).rejects.toThrow(/streamIdleTimeoutMs/)
|
||||
})
|
||||
|
||||
it('rejects invalid nested retryPolicy before registering the provider', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
await expect(ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
retryPolicy: { mode: 'normal', maxRetries: -1 },
|
||||
})).rejects.toThrow(/retryPolicy/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: 24a4762342f1ce8e71d1a5b1733fe02257823cb8
|
||||
README.zh.md: 557dc892c2eac10edc4e2fe4a0a142024942b1a9
|
||||
README.md: ac47cf6a21285fc887948a5a7798a9f1cb9157b0
|
||||
README.zh.md: a3d864ed9068d8bdaff4c5b73a4b7b339802ae05
|
||||
@@ -19,6 +19,13 @@ Configure credentials and deployment-specific transport settings per provider. O
|
||||
apiKey: !!js process.env.OPENAI_API_KEY
|
||||
baseURL: https://proxy.example.com:8443
|
||||
reasoning: high
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 3
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
- provider: anthropic
|
||||
apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
streamIdleTimeoutMs: 300000
|
||||
@@ -34,7 +41,7 @@ The adapter exposes each configured provider's installed pi-ai models through `c
|
||||
|
||||
The `reasoning.efforts` list is pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`. A non-reasoning model therefore exposes pi-ai's `off` choice. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and any explicit value absent from the exact model capability fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
@@ -102,4 +109,4 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
- **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field.
|
||||
- **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override.
|
||||
- **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes.
|
||||
- **Retry policy is not an adapter option** — SDK retries are disabled so durable agent steps and `llm/retry` events own every visible attempt; direct `ctx.llm.stream()` calls remain single-attempt.
|
||||
- **Retry policy is provider-owned, not an SDK retry** — each provider profile may configure nested `retryPolicy`, which `dsh-llm-retry` executes at the agent failed-step seam; pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt.
|
||||
@@ -19,6 +19,13 @@
|
||||
apiKey: !!js process.env.OPENAI_API_KEY
|
||||
baseURL: https://proxy.example.com:8443
|
||||
reasoning: high
|
||||
retryPolicy:
|
||||
mode: normal
|
||||
maxRetries: 3
|
||||
backoff:
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
- provider: anthropic
|
||||
apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
streamIdleTimeoutMs: 300000
|
||||
@@ -34,7 +41,7 @@
|
||||
|
||||
`reasoning.efforts` 列表是 pi-ai 有序的 `getSupportedThinkingLevels(model)` 结果,不经筛选或规范化,其中包括 `off`,以及模型对 `xhigh` 或 `max` 的特定支持。Harness 将每个规范 pi-ai 级别公开为不透明 ID;提供方/模型协议拼写仍保留在 pi-ai 的 `thinkingLevelMap` 中。因此,不具备推理(reasoning)能力的模型也会公开 pi-ai 的 `off` 选项。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;任何未出现在确切模型能力中的显式值都会在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `provider`、`apiKey`、`baseURL`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs` 和 `streamIdleTimeoutMs`。流 idle 间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。Harness 应用归因会胜过名称冲突的已配置标头。
|
||||
受支持的 profile 字段是 `provider`、`apiKey`、`baseURL`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获;省略时使用有界的 normal 默认值。流 idle 间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。Harness 应用归因会胜过名称冲突的已配置标头。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent 级重试预算。Idle 过期会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
@@ -102,4 +109,4 @@ pi-ai 事件会变为 harness reasoning、文本、工具调用、usage 与 fini
|
||||
- **不支持 `GenerateOptions.stop`**:pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence,因此适配器会拒绝该字段。
|
||||
- **历史中的 `system` 消息使用 pi-ai 通用上下文转换**:提供方特定位置由 pi-ai 决定,而非由 harness 拥有的协议覆盖决定。
|
||||
- **无法获取提供方 HTTP 状态**:pi-ai 错误事件不会在所有提供方上公开稳定 HTTP 状态;失败只公开稳定 harness 错误 code。
|
||||
- **重试策略不是适配器选项**:SDK 重试已禁用,因此持久 agent 步骤与 `llm/retry` 事件拥有每次可见尝试;直接 `ctx.llm.stream()` 调用仍只尝试一次。
|
||||
- **重试策略由提供方持有,而不是 SDK 重试**:每个提供方 profile 都可以配置嵌套的 `retryPolicy`,由 `dsh-llm-retry` 在 agent 的失败步骤 seam 上执行;pi-ai SDK 重试仍保持禁用,因此持久 agent 步骤与 `llm/retry` 事件拥有每次可见尝试,直接 `ctx.llm.stream()` 调用仍只尝试一次。
|
||||
@@ -26,6 +26,7 @@ import type {
|
||||
LlmModelInfo,
|
||||
LlmResolvedModelInfo,
|
||||
ReasoningEffortId as ReasoningEffortIdType,
|
||||
ResolvedRetryPolicy,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
|
||||
@@ -44,7 +45,10 @@ export interface PiAiAdapterOptions {
|
||||
* Resolve a catalog model dynamically and apply only the configured endpoint
|
||||
* override, preserving the catalog's API/capability/compatibility metadata.
|
||||
*/
|
||||
function resolvePiModel(profile: PiAiProviderProfile, modelId: string): Model<Api> {
|
||||
function resolvePiModel(
|
||||
profile: Omit<PiAiProviderProfile, 'retryPolicy'>,
|
||||
modelId: string,
|
||||
): Model<Api> {
|
||||
const model = getBuiltinModels(profile.provider as BuiltinProvider).find(candidate => candidate.id === modelId) as Model<Api> | undefined
|
||||
if (model === undefined) {
|
||||
throw new LlmError(`pi-ai provider "${profile.provider}" has no catalog model "${modelId}"`, 'UNKNOWN_MODEL')
|
||||
@@ -54,7 +58,7 @@ function resolvePiModel(profile: PiAiProviderProfile, modelId: string): Model<Ap
|
||||
|
||||
/** Copy profile stream knobs into pi-ai's common option vocabulary. */
|
||||
function profileOptions(
|
||||
profile: PiAiProviderProfile,
|
||||
profile: Omit<PiAiProviderProfile, 'retryPolicy'>,
|
||||
reasoning: ModelThinkingLevel | undefined,
|
||||
): SimpleStreamOptions {
|
||||
const enabledReasoning: ThinkingLevel | undefined = reasoning === 'off' ? undefined : reasoning
|
||||
@@ -107,6 +111,10 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
this.profiles = new Map(resolveProfiles(options.profiles).map(profile => [profile.provider, profile]))
|
||||
}
|
||||
|
||||
override providerRetryPolicy(provider: string): ResolvedRetryPolicy | undefined {
|
||||
return this.profiles.get(provider)?.retryPolicy
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
const profile = this.profiles.get(provider)
|
||||
if (profile === undefined) {
|
||||
|
||||
@@ -8,6 +8,8 @@ import { getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { CacheRetention, ModelThinkingLevel, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
|
||||
import z from 'schemastery'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { ResolvedRetryPolicy, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
@@ -36,12 +38,16 @@ export interface PiAiProviderProfile {
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
|
||||
/** Validated profile with every adapter-owned default resolved. */
|
||||
export interface ResolvedPiAiProviderProfile extends PiAiProviderProfile {
|
||||
export interface ResolvedPiAiProviderProfile extends Omit<PiAiProviderProfile, 'retryPolicy'> {
|
||||
/** Positive finite provider-idle interval after defaulting. */
|
||||
streamIdleTimeoutMs: number
|
||||
/** Immutable retry policy captured with this provider route. */
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
/** Plugin configuration: the non-empty provider profiles this instance owns. */
|
||||
@@ -69,6 +75,7 @@ const profile = z.object({
|
||||
timeoutMs: z.natural(),
|
||||
websocketConnectTimeoutMs: z.natural(),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
retryPolicy: RetryPolicySchema,
|
||||
})
|
||||
|
||||
/** Runtime schema for {@link Config}. */
|
||||
@@ -115,6 +122,10 @@ export function resolveProfiles(profiles: readonly PiAiProviderProfile[]): Resol
|
||||
return {
|
||||
...source,
|
||||
streamIdleTimeoutMs,
|
||||
retryPolicy: resolveRetryPolicy(
|
||||
source.retryPolicy,
|
||||
`llm-pi-ai: provider "${source.provider}" retryPolicy`,
|
||||
),
|
||||
...source.headers === undefined ? {} : { headers: { ...source.headers } },
|
||||
...source.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...source.thinkingBudgets } },
|
||||
}
|
||||
|
||||
@@ -10,6 +10,9 @@
|
||||
* providers:
|
||||
* - provider: openai
|
||||
* apiKey: !!js process.env.OPENAI_API_KEY
|
||||
* retryPolicy:
|
||||
* mode: normal
|
||||
* maxRetries: 2
|
||||
* - provider: anthropic
|
||||
* apiKey: !!js process.env.ANTHROPIC_API_KEY
|
||||
* - provider: openrouter
|
||||
@@ -36,6 +39,6 @@ export const inject = ['llm']
|
||||
/** Register one generic pi-ai adapter for all configured provider routes. */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
const profiles = resolveProfiles(config.providers)
|
||||
const adapter = new PiAiAdapter({ profiles })
|
||||
const adapter = new PiAiAdapter({ profiles: config.providers })
|
||||
ctx.llm.registerAdapter(profiles.map(entry => entry.provider), adapter)
|
||||
}
|
||||
@@ -330,12 +330,31 @@ describe('provider profile lifecycle', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const fiber = await ctx.plugin(LlmPiAi, {
|
||||
providers: [{ provider: 'openai' }, { provider: 'anthropic' }],
|
||||
providers: [
|
||||
{
|
||||
provider: 'openai',
|
||||
retryPolicy: {
|
||||
mode: 'always',
|
||||
backoff: { initialDelayMs: 25, maxDelayMs: 100, jitterRatio: 0.2 },
|
||||
},
|
||||
},
|
||||
{ provider: 'anthropic' },
|
||||
],
|
||||
})
|
||||
expect(ctx.llm.listProviders()).toEqual([
|
||||
{ id: 'openai', name: 'openai' },
|
||||
{ id: 'anthropic', name: 'anthropic' },
|
||||
])
|
||||
expect(ctx.llm.providerRetryPolicy('openai')).toEqual({
|
||||
mode: 'always',
|
||||
initialDelayMs: 25,
|
||||
maxDelayMs: 100,
|
||||
jitterRatio: 0.2,
|
||||
})
|
||||
expect(ctx.llm.providerRetryPolicy('anthropic')).toMatchObject({
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
})
|
||||
await fiber.dispose()
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
@@ -459,6 +478,23 @@ describe('provider profile lifecycle', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects invalid nested retryPolicy at the provider-profile boundary', async () => {
|
||||
expect(() => resolveProfiles([{
|
||||
provider: 'openai',
|
||||
retryPolicy: { mode: 'always', backoff: { jitterRatio: -1 } },
|
||||
}])).toThrow(/retryPolicy\.backoff\.jitterRatio/)
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmPiAi, {
|
||||
providers: [{
|
||||
provider: 'openai',
|
||||
retryPolicy: { mode: 'normal', maxRetries: -1 },
|
||||
}],
|
||||
})).rejects.toThrow(/retryPolicy/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it('constructs the adapter directly and rejects routes it does not own', async () => {
|
||||
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai' }] })
|
||||
await expect(adapter.listModels('anthropic')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-retry/README.md
|
||||
README.md: 596a46e5395a4b5be9d400a85ee7c54d613ec2e6
|
||||
README.zh.md: a6a48f203e4701688816d4365b03da1ae2cfad82
|
||||
README.md: 7a86652a794e70c4dfd00ab7427730387e3ec949
|
||||
README.zh.md: 6255cca8c3b669ddec401496acb1e79ad54a8b3e
|
||||
@@ -2,42 +2,52 @@
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
Function plugin that retries selected transient model-request failures through the `agent/request-error` waterfall. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered turn.
|
||||
Function plugin that applies exact-provider retry policy through the agent loop's closed-step `agent/request-error` waterfall. It does not wrap `ctx.llm.stream()`: every adapter call remains one provider attempt, and every retry opens a fresh numbered turn.
|
||||
|
||||
The default policy permits two retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, using bounded exponential backoff from 500 ms to 10 seconds with 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion (a terminal stop with zero content blocks); the attempt produced nothing durable, so repeating it is safe. Delay bounds must fit Node's supported timer range. A valid `providerRetryAfterMs` replaces local backoff when it is within the configured cap; an over-cap instruction delegates to the next recovery policy instead.
|
||||
Each provider adapter owns an optional nested `retryPolicy`, captured when its route registers on `ctx.llm` and carried with each call that reaches that registration's final adapter boundary. An in-flight failure retains that serving policy if the route is later disposed or replaced; a failure before any final adapter is selected has no provider policy and delegates. Omission uses normal mode: two retries for `EMPTY_RESPONSE`, `RATE_LIMIT`, `SERVER`, `TIMEOUT`, and `TRANSPORT`, with bounded exponential backoff from 500 ms to 10 seconds and 10 percent jitter. `EMPTY_RESPONSE` is the adapters' classification of a degenerate provider completion that produced no durable content, so repeating it is safe. A normal policy can change its finite budget, eligible codes, and backoff. Always mode asks downstream recovery first, then retries every model-request failure without an attempt limit; success, cancellation, or plugin disposal stops it after active delegated recovery reaches quiescence.
|
||||
|
||||
The recovery listener appends a non-surface `llm/retry` event after the failed step, waits for the backoff while the failed turn's signal remains live, then returns `{ kind: 'retry' }`. The loop closes that failed turn and opens a retry turn over the same durable history. The policy keeps its own retry count across that uninterrupted recovery chain and clears it at terminal `agent/settled`. Turn cancellation and plugin disposal abort the wait.
|
||||
Both modes use bounded exponential backoff with symmetric jitter. A valid `providerRetryAfterMs` at or below `maxDelayMs` replaces local backoff without jitter. An over-cap provider delay makes normal mode delegate, while always mode uses its configured local backoff so it cannot terminate on that instruction.
|
||||
|
||||
The separately published `./invariant` companion checks that every retry record appears inside an open turn after its failed step, matches its position in the current retry chain, and carries a positive bounded retry budget and non-negative bounded timer delay. Full jitter may schedule zero milliseconds at its lower boundary.
|
||||
Before waiting, the plugin appends a non-surface `llm/retry` event with the provider, mode, canonical resolved-policy key, failure, and scheduled delay. The key includes every behavior-affecting field and sorts normal-mode codes because eligibility uses set membership. Retry numbers continue only across events with the same provider and complete policy key, so a route replacement with different limits, code membership, or backoff starts its own history. Normal events include the finite maximum; always events omit it, and UIs render `∞`. After the wait, the listener returns `{ kind: 'retry' }`, and the loop closes the failed turn and opens a retry turn over the same durable history. Cancellation and plugin disposal abort active backoff, drain active delegated recovery before applying the abort, and make a callback captured before disposal fail closed.
|
||||
|
||||
The separately published `./invariant` companion checks that every retry record names the current open turn and latest closed step, matches the failed request's durable provider, carries non-empty provider and policy identities, has mode-specific bounds, a unique step record, the correct provider-policy retry number, and a bounded timer delay. Full jitter may schedule zero milliseconds at its lower boundary.
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
- name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
retryPolicy:
|
||||
mode: always
|
||||
backoff:
|
||||
initialDelayMs: 1000
|
||||
maxDelayMs: 30000
|
||||
jitterRatio: 0.2
|
||||
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
```
|
||||
|
||||
The executor has no policy config. Multi-provider adapters such as `dsh-llm-pi-ai` place `retryPolicy` inside each provider profile, avoiding a second provider-name list.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Transient request recovery
|
||||
### Model-request recovery
|
||||
|
||||
#### What the model sees
|
||||
|
||||
No retry event, delay, or failure prose is model-visible. The retry turn reconstructs the same explicit provider/model request from durable session history; failed chunks never enter derived messages.
|
||||
No retry event, delay, provider error, or failed partial output is model-visible. The retry turn reconstructs the same explicit provider/model request from durable surface history unless a downstream recovery policy deliberately changes that surface; failed chunks never enter derived messages.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Each retry is a new provider request and may repeat input-token billing. The finite budget caps attempts; `llm/retry` itself contributes no tokens.
|
||||
Each retry is a new provider request and may repeat input-token billing. Normal mode has a finite budget; always mode can consume unbounded requests until success or cancellation. `llm/retry` itself contributes no tokens.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface status event does not change cache identity.
|
||||
The reconstructed request preserves the prior prefix and is eligible for provider cache reuse under that provider's rules. The non-surface retry event does not change cache identity.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Agent turns are the only retry boundary** — direct `ctx.llm.stream()` consumers remain single-attempt because a raw stream cannot separate already-emitted chunks durably.
|
||||
- **Finite plugin budgets add** — this policy counts only configured transient codes; context-overflow compaction counts only its own code. A future policy with overlapping codes must document and test registration-order behavior.
|
||||
- **`llm/retry` records completed backoff, not request completion** — later step and turn events establish success, exhaustion, or cancellation.
|
||||
- **Always mode retries permanent failures** — authentication, quota, invalid-request, protocol, and unrecoverable context errors continue until success, cancellation, or disposal; deployments own provider-specific cost and latency controls.
|
||||
- **Finite plugin budgets add** — normal mode counts only its configured codes and exact provider policy, while context-overflow compaction owns a separate budget. A future overlapping policy must document and test registration-order behavior.
|
||||
- **Recovery policies compose by waterfall order** — always mode accepts a downstream retry before applying its fallback. A later policy that ignores cancellation and never settles also prevents fallback, turn quiescence, and plugin disposal from completing.
|
||||
- **`llm/retry` records scheduling, not completion** — later step and turn events establish success, exhaustion, or cancellation.
|
||||
@@ -2,42 +2,52 @@
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
一个函数插件,通过 `agent/request-error` waterfall 重试特定的短暂模型请求失败。它不包装 `ctx.llm.stream()`:每次适配器调用仍是一次提供方尝试,每次重试都会开启新的编号轮次。
|
||||
一个函数插件,通过 agent loop(智能体循环)在已关闭步骤上触发的 `agent/request-error` waterfall(瀑布式事件)应用确切提供方重试策略。它不包装 `ctx.llm.stream()`:每次适配器调用仍是一次提供方尝试,每次重试都会开启新的编号轮次。
|
||||
|
||||
默认策略允许为 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 和 `TRANSPORT` 重试两次,使用从 500 ms 到 10 秒的有界指数退避与 10% jitter。`EMPTY_RESPONSE` 是适配器对退化提供方完成的分类(携带零个内容块的终止 stop);该尝试未产生持久内容,因此可安全重复。延迟边界必须适合 Node 支持的定时器范围。有效 `providerRetryAfterMs` 在已配置上限内时替换本地退避;超出上限的指令会委托给下一项恢复策略。
|
||||
每个提供方适配器都拥有可选的嵌套 `retryPolicy`;路由在 `ctx.llm` 上注册时会捕获该策略,任何到达该注册最终适配器边界的调用都会携带它。如果之后释放或替换路由,进行中的失败仍会保留为其提供服务的策略;在选中任何最终适配器前发生的失败没有提供方策略,会继续委托。省略策略时使用 normal mode:为 `EMPTY_RESPONSE`、`RATE_LIMIT`、`SERVER`、`TIMEOUT` 和 `TRANSPORT` 重试两次,并采用从 500 ms 到 10 秒的有界指数退避与 10% jitter。`EMPTY_RESPONSE` 是适配器对未产生任何持久内容的退化提供方完成所作的分类,因此可安全重复。normal 策略可以更改其有限预算、合格 code 和退避配置。always mode 会先请求下游恢复,再无次数上限地重试每个模型请求失败;成功、取消或插件 dispose(资源释放)会在活跃的委托恢复完全停稳后终止它。
|
||||
|
||||
恢复 listener 会在失败步骤之后追加一个非表层 `llm/retry` 事件,在失败轮次的信号仍存活期间等待退避,然后返回 `{ kind: 'retry' }`。循环会关闭该失败轮次,并在同一持久历史上开启重试轮次。策略在这条不间断的恢复链中维护自己的重试计数,并在终态 `agent/settled` 时清零。轮次取消与插件 dispose 会中止等待。
|
||||
两种 mode 都使用带对称 jitter 的有界指数退避。有效 `providerRetryAfterMs` 不超过 `maxDelayMs` 时会替换本地退避,并且不加 jitter。超出上限的提供方延迟会使 normal mode 继续委托;always mode 则改用已配置的本地退避,避免该指令终止重试。
|
||||
|
||||
单独发布的 `./invariant` 配套模块会检查每个重试记录是否出现在开启轮次内的失败步骤之后,是否与其在当前重试链中的位置匹配,以及是否携带正数有界重试预算和非负有界定时器延迟。完整 jitter 可以在下界调度为零毫秒。
|
||||
等待前,插件会追加一条不进入表层的 `llm/retry` 事件,其中包含提供方、mode、规范的解析策略 key、失败和计划延迟。该 key 包含所有影响行为的字段,并对 normal mode 的 code 排序,因为合格性采用集合成员关系判断。只有提供方与完整策略 key 都相同的事件才会延续重试编号;因此,用限制、code 成员关系或退避不同的路由替换后,会开始自己的历史。normal 事件包含有限上限;always 事件省略该上限,UI 会渲染 `∞`。等待结束后,监听器返回 `{ kind: 'retry' }`,循环关闭失败轮次,并在同一持久历史上开启重试轮次。取消与插件 dispose 会中止活跃退避,在应用中止前排空活跃的委托恢复,并使 dispose 前捕获的 callback 只能以失败结束。
|
||||
|
||||
单独发布的 `./invariant` 配套模块会检查每个重试记录是否指向当前开启轮次及其最新已关闭步骤,是否与失败请求的持久提供方匹配,是否携带非空的提供方与策略身份,是否满足 mode 特定边界,是否拥有唯一步骤记录和正确的提供方策略重试编号,以及是否携带有界定时器延迟。完整 jitter 可以在下界调度为零毫秒。
|
||||
|
||||
```yaml
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
- name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
maxTransientRetries: 2
|
||||
initialDelayMs: 500
|
||||
maxDelayMs: 10000
|
||||
jitterRatio: 0.1
|
||||
retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT]
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
retryPolicy:
|
||||
mode: always
|
||||
backoff:
|
||||
initialDelayMs: 1000
|
||||
maxDelayMs: 30000
|
||||
jitterRatio: 0.2
|
||||
|
||||
- name: '@deepseek-ai/dsh-llm-retry'
|
||||
```
|
||||
|
||||
执行器没有策略配置。`dsh-llm-pi-ai` 等多提供方适配器会把 `retryPolicy` 放在每个提供方 profile 内,避免维护第二份提供方名称列表。
|
||||
|
||||
## 模型体验
|
||||
|
||||
### 短暂请求恢复
|
||||
### 模型请求恢复
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
模型不会看到重试事件、延迟或失败文本。重试轮次会从持久会话历史中重建相同的显式提供方/模型请求;失败 chunk 绝不会进入派生消息。
|
||||
模型不会看到重试事件、延迟、提供方错误或失败的部分输出。重试轮次会从持久表层历史中重建相同的显式提供方/模型请求,除非下游恢复策略有意更改该表层;失败分片绝不会进入派生消息。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
每次重试都是新的提供方请求,可能重复计费输入 token。有限预算会限制尝试次数;`llm/retry` 自身不产生 token。
|
||||
每次重试都是新的提供方请求,可能重复计费输入 token。normal mode 具有有限预算;always mode 可以在成功或取消前消耗无界数量的请求。`llm/retry` 自身不产生 token。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
重建请求保留之前的前缀,并可根据该提供方的规则复用 cache。非表层状态事件不会改变 cache 身份。
|
||||
重建请求保留之前的前缀,并可根据该提供方的规则复用 cache。非表层重试事件不会改变 cache 身份。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **Agent 轮次是唯一重试边界**:直接 `ctx.llm.stream()` 消费方仍只尝试一次,因为原始流无法将已发出 chunk 持久分隔为不同尝试。
|
||||
- **有限插件预算可叠加**:该策略只统计已配置短暂 code;上下文溢出压缩只统计自身 code。未来如有 code 重叠的策略,必须记录并测试注册顺序行为。
|
||||
- **`llm/retry` 记录已完成的退避,不是请求完成**:后续步骤与轮次事件用于确立成功、耗尽或取消。
|
||||
- **always mode 会重试永久性失败**:身份验证、配额、无效请求、协议和无法恢复的上下文错误都会继续重试,直至成功、取消或 dispose;部署负责提供方特定的成本与延迟控制。
|
||||
- **有限插件预算可叠加**:normal mode 只统计已配置 code 和确切提供方策略,上下文溢出压缩则拥有独立预算。未来如有重叠策略,必须记录并测试注册顺序行为。
|
||||
- **恢复策略按 waterfall 顺序组合**:always mode 会先接受下游重试,再应用自己的回退。后续策略如果忽略取消且永不结算,也会阻止回退、轮次完全停稳和插件 dispose 完成。
|
||||
- **`llm/retry` 记录调度,不是完成**:后续步骤与轮次事件用于确立成功、耗尽或取消。
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-llm-retry",
|
||||
"description": "Bounded transient LLM request retry policy for the DeepSeek Harness",
|
||||
"description": "Provider-routed LLM request retry policy for the DeepSeek Harness",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
/** Durable request-route lookup for one closed model step. @module @deepseek-ai/dsh-llm-retry/history */
|
||||
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* Find the provider in force when one step closed, excluding later recovery mutations.
|
||||
* Request headers remain effective across turn boundaries until a newer full
|
||||
* snapshot changes them; every provider change requires a newer full snapshot.
|
||||
* @param events - session events containing the closed step.
|
||||
* @param turn - turn that owns the failed step.
|
||||
* @param step - failed step whose provider is required.
|
||||
* @returns the provider from the request header in force at that step boundary.
|
||||
*/
|
||||
export function providerForClosedStep(
|
||||
events: readonly SessionEvent[],
|
||||
turn: number,
|
||||
step: number,
|
||||
): string | undefined {
|
||||
const stepEndIndex = events.findLastIndex(event =>
|
||||
event.type === 'step/end'
|
||||
&& event.data.turn === turn
|
||||
&& event.data.step === step,
|
||||
)
|
||||
if (stepEndIndex < 0) return undefined
|
||||
for (let index = stepEndIndex; index >= 0; index -= 1) {
|
||||
// The loop bounds prove this indexed read exists.
|
||||
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
|
||||
const event = events[index]!
|
||||
if (event.type === 'request/header') return event.data.header.config.provider
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
+164
-125
@@ -1,6 +1,6 @@
|
||||
/**
|
||||
* Bounded transient model-request retry policy on the agent request-recovery
|
||||
* seam. Each scheduled retry is durable before its cancellable wait.
|
||||
* Provider-routed model-request retry policy on the agent loop's closed-step
|
||||
* recovery seam. Each scheduled retry is durable before its cancellable wait.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-retry
|
||||
*/
|
||||
@@ -8,20 +8,32 @@
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { Agent, RequestError, RequestErrorAction } from '@deepseek-ai/dsh-agent'
|
||||
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
|
||||
import type {} from '@deepseek-ai/dsh-session'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import type { LlmFailure, ResolvedRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { providerForClosedStep } from './history.ts'
|
||||
|
||||
declare module '@deepseek-ai/dsh-session' {
|
||||
interface SessionEventMap {
|
||||
/** Durable, non-surface record of one transient retry scheduled after a closed failed step. */
|
||||
/** Durable, non-surface record of one provider-routed retry scheduled after a closed failed step. */
|
||||
'llm/retry': {
|
||||
turn: number
|
||||
step: number
|
||||
provider: string
|
||||
mode: 'normal'
|
||||
policyKey: string
|
||||
retry: number
|
||||
maxRetries: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
} | {
|
||||
turn: number
|
||||
step: number
|
||||
provider: string
|
||||
mode: 'always'
|
||||
policyKey: string
|
||||
retry: number
|
||||
delayMs: number
|
||||
failure: LlmFailure
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -29,82 +41,19 @@ declare module '@deepseek-ai/dsh-session' {
|
||||
export const name = 'llm-retry'
|
||||
export const inject = ['agents']
|
||||
|
||||
const DEFAULT_MAX_TRANSIENT_RETRIES = 2
|
||||
const DEFAULT_INITIAL_DELAY_MS = 500
|
||||
const DEFAULT_MAX_DELAY_MS = 10_000
|
||||
const DEFAULT_JITTER_RATIO = 0.1
|
||||
const DEFAULT_RETRYABLE_CODES = Object.freeze(['EMPTY_RESPONSE', 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT'])
|
||||
|
||||
/** Deployment-owned limits and classification for transient request recovery. */
|
||||
export interface Config {
|
||||
/** Maximum transient retries after the first request (default 2). */
|
||||
maxTransientRetries?: number
|
||||
/** Initial local exponential-backoff delay in milliseconds (default 500). */
|
||||
initialDelayMs?: number
|
||||
/** Maximum accepted or locally scheduled delay in milliseconds (default 10000). */
|
||||
maxDelayMs?: number
|
||||
/** Symmetric random multiplier range around one (default 0.1). */
|
||||
jitterRatio?: number
|
||||
/** Stable failure codes eligible for this policy. */
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
/** This policy executor has no config; providers own `retryPolicy`. */
|
||||
export type Config = Readonly<Record<string, never>>
|
||||
|
||||
/** Runtime schema for {@link Config}. */
|
||||
export const Config: z<Config> = z.object({
|
||||
maxTransientRetries: z.number().step(1).min(0).default(DEFAULT_MAX_TRANSIENT_RETRIES),
|
||||
initialDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_INITIAL_DELAY_MS),
|
||||
maxDelayMs: z.number().max(MAX_TIMER_DELAY_MS).default(DEFAULT_MAX_DELAY_MS),
|
||||
jitterRatio: z.number().min(0).max(1).default(DEFAULT_JITTER_RATIO),
|
||||
retryableCodes: z.array(z.string()).default([...DEFAULT_RETRYABLE_CODES]),
|
||||
})
|
||||
export const Config = z.object({}) as unknown as z<Config>
|
||||
|
||||
interface ResolvedConfig {
|
||||
readonly maxTransientRetries: number
|
||||
readonly initialDelayMs: number
|
||||
readonly maxDelayMs: number
|
||||
readonly jitterRatio: number
|
||||
readonly retryableCodes: ReadonlySet<string>
|
||||
}
|
||||
|
||||
function resolveConfig(config: Config): ResolvedConfig {
|
||||
const maxTransientRetries = config.maxTransientRetries ?? DEFAULT_MAX_TRANSIENT_RETRIES
|
||||
const initialDelayMs = config.initialDelayMs ?? DEFAULT_INITIAL_DELAY_MS
|
||||
const maxDelayMs = config.maxDelayMs ?? DEFAULT_MAX_DELAY_MS
|
||||
const jitterRatio = config.jitterRatio ?? DEFAULT_JITTER_RATIO
|
||||
const codes = config.retryableCodes ?? [...DEFAULT_RETRYABLE_CODES]
|
||||
|
||||
if (!Number.isInteger(maxTransientRetries) || maxTransientRetries < 0) {
|
||||
throw new Error('llm-retry: maxTransientRetries must be a non-negative integer')
|
||||
function validateConfig(config: Config): void {
|
||||
const [key] = Object.keys(config)
|
||||
if (key === undefined) return
|
||||
if (key === 'retryPolicy') {
|
||||
throw new Error('llm-retry: retryPolicy belongs under each provider configuration')
|
||||
}
|
||||
if (!Number.isFinite(initialDelayMs) || initialDelayMs <= 0 || initialDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`llm-retry: initialDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (!Number.isFinite(maxDelayMs) || maxDelayMs <= 0 || maxDelayMs > MAX_TIMER_DELAY_MS) {
|
||||
throw new Error(`llm-retry: maxDelayMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
if (initialDelayMs > maxDelayMs) {
|
||||
throw new Error('llm-retry: initialDelayMs must be less than or equal to maxDelayMs')
|
||||
}
|
||||
if (!Number.isFinite(jitterRatio) || jitterRatio < 0 || jitterRatio > 1) {
|
||||
throw new Error('llm-retry: jitterRatio must be between 0 and 1')
|
||||
}
|
||||
if (codes.length === 0) {
|
||||
throw new Error('llm-retry: retryableCodes must not be empty')
|
||||
}
|
||||
if (codes.some(code => code.length === 0)) {
|
||||
throw new Error('llm-retry: retryableCodes must contain only non-empty strings')
|
||||
}
|
||||
if (new Set(codes).size !== codes.length) {
|
||||
throw new Error('llm-retry: retryableCodes must not contain duplicates')
|
||||
}
|
||||
|
||||
return Object.freeze({
|
||||
maxTransientRetries,
|
||||
initialDelayMs,
|
||||
maxDelayMs,
|
||||
jitterRatio,
|
||||
retryableCodes: new Set(codes),
|
||||
})
|
||||
throw new Error(`llm-retry: unknown key "${key}"`)
|
||||
}
|
||||
|
||||
/** Non-serializable seams used to make timing policy deterministic in tests. */
|
||||
@@ -113,13 +62,40 @@ export interface RetryInternals {
|
||||
random?: () => number
|
||||
}
|
||||
|
||||
function localDelay(config: ResolvedConfig, retry: number, random: () => number): number {
|
||||
type DownstreamOutcome =
|
||||
| { readonly type: 'decision'; readonly decision: RequestErrorAction }
|
||||
| { readonly type: 'error'; readonly error: unknown }
|
||||
|
||||
async function settleDownstream(
|
||||
next: () => Promise<RequestErrorAction>,
|
||||
): Promise<DownstreamOutcome> {
|
||||
try {
|
||||
return { type: 'decision', decision: await next() }
|
||||
} catch (error: unknown) {
|
||||
return { type: 'error', error }
|
||||
}
|
||||
}
|
||||
|
||||
function localDelay(config: ResolvedRetryPolicy, retry: number, random: () => number): number {
|
||||
const exponent = Math.min(retry - 1, 1024)
|
||||
const exponential = Math.min(config.initialDelayMs * 2 ** exponent, config.maxDelayMs)
|
||||
const jitter = 1 - config.jitterRatio + 2 * config.jitterRatio * random()
|
||||
return Math.min(exponential * jitter, config.maxDelayMs)
|
||||
}
|
||||
|
||||
function retryPolicyKey(policy: ResolvedRetryPolicy): string {
|
||||
return policy.mode === 'always'
|
||||
? JSON.stringify([policy.mode, policy.initialDelayMs, policy.maxDelayMs, policy.jitterRatio])
|
||||
: JSON.stringify([
|
||||
policy.mode,
|
||||
policy.maxRetries,
|
||||
[...policy.retryableCodes].sort(),
|
||||
policy.initialDelayMs,
|
||||
policy.maxDelayMs,
|
||||
policy.jitterRatio,
|
||||
])
|
||||
}
|
||||
|
||||
function cancellableDelay(delayMs: number, signal: AbortSignal): Promise<boolean> {
|
||||
if (signal.aborted) return Promise.resolve(false)
|
||||
return new Promise((resolve) => {
|
||||
@@ -136,60 +112,141 @@ function cancellableDelay(delayMs: number, signal: AbortSignal): Promise<boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Install bounded transient request recovery.
|
||||
* Install provider-routed normal or unbounded request recovery.
|
||||
* @param ctx - plugin context that owns the listener and active waits.
|
||||
* @param config - retry budget, delay bounds, jitter, and eligible codes.
|
||||
* @param config - empty executor config; provider registrations own policy.
|
||||
* @param internals - non-serializable deterministic seams for tests.
|
||||
*/
|
||||
export function apply(ctx: Context, config: Config = {}, internals: RetryInternals = {}): void {
|
||||
const resolved = resolveConfig(config)
|
||||
validateConfig(config)
|
||||
const random = internals.random ?? Math.random
|
||||
const lifetime = new AbortController()
|
||||
const active = new Set<Promise<RequestErrorAction>>()
|
||||
const retries = new WeakMap<Agent, number>()
|
||||
|
||||
function track(operation: Promise<RequestErrorAction>): Promise<RequestErrorAction> {
|
||||
const tracked = operation.finally(() => active.delete(tracked))
|
||||
active.add(tracked)
|
||||
return tracked
|
||||
}
|
||||
|
||||
async function backoff(
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
failure: LlmFailure,
|
||||
provider: string,
|
||||
policy: ResolvedRetryPolicy,
|
||||
policyKey: string,
|
||||
retry: number,
|
||||
delayMs: number,
|
||||
signal: AbortSignal,
|
||||
): Promise<RequestErrorAction> {
|
||||
const fusedSignal = AbortSignal.any([signal, lifetime.signal])
|
||||
if (fusedSignal.aborted) return
|
||||
agent.session.append('llm/retry', {
|
||||
turn,
|
||||
step,
|
||||
retry,
|
||||
maxRetries: resolved.maxTransientRetries,
|
||||
delayMs,
|
||||
failure,
|
||||
})
|
||||
retries.set(agent, retry)
|
||||
const eventData = policy.mode === 'normal'
|
||||
? {
|
||||
turn,
|
||||
step,
|
||||
provider,
|
||||
mode: policy.mode,
|
||||
policyKey,
|
||||
retry,
|
||||
maxRetries: policy.maxRetries,
|
||||
delayMs,
|
||||
failure,
|
||||
}
|
||||
: {
|
||||
turn,
|
||||
step,
|
||||
provider,
|
||||
mode: policy.mode,
|
||||
policyKey,
|
||||
retry,
|
||||
delayMs,
|
||||
failure,
|
||||
}
|
||||
agent.session.append('llm/retry', eventData)
|
||||
if (!await cancellableDelay(delayMs, fusedSignal)) return
|
||||
return { kind: 'retry' }
|
||||
}
|
||||
|
||||
ctx.on('agent/settled', (agent) => {
|
||||
retries.delete(agent)
|
||||
})
|
||||
|
||||
// A completed model response ends the consecutive-failure sequence even
|
||||
// when its tool calls keep the turn running into another request.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
if (event.type !== 'assistant/message') return
|
||||
const agent = ctx.agents.get(session.id)
|
||||
if (agent?.session === session) retries.delete(agent)
|
||||
})
|
||||
|
||||
const disposeListener = ctx.on('agent/request-error', (
|
||||
async function recover(
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
_error: RequestError,
|
||||
failure: LlmFailure,
|
||||
priorFailures: readonly LlmFailure[],
|
||||
policy: ResolvedRetryPolicy | undefined,
|
||||
signal: AbortSignal,
|
||||
next: () => Promise<RequestErrorAction>,
|
||||
): Promise<RequestErrorAction> {
|
||||
if (policy === undefined) return next()
|
||||
// The call-local policy belongs to the registration that served this
|
||||
// failure. Recover only the durable provider identity from the header;
|
||||
// downstream recovery may append later state before an always fallback.
|
||||
const provider = providerForClosedStep(agent.session.events, turn, step)
|
||||
/* v8 ignore next 3 -- agent-loop closes only steps whose request header was recorded */
|
||||
if (provider === undefined) {
|
||||
throw new Error(`llm-retry: no request provider for closed turn ${turn}/step ${step}`)
|
||||
}
|
||||
if (policy.mode === 'always') {
|
||||
if (signal.aborted || lifetime.signal.aborted) return
|
||||
const fusedSignal = AbortSignal.any([signal, lifetime.signal])
|
||||
// The loop and plugin lifetime stay open until delegated recovery settles.
|
||||
// An abort then wins before the decision or fallback can mutate later state.
|
||||
const downstream = await settleDownstream(next)
|
||||
if (fusedSignal.aborted) return
|
||||
if (downstream.type === 'error') {
|
||||
ctx.logger.warn(
|
||||
`llm-retry: provider "${provider}" always policy ignored a downstream recovery failure: %o`,
|
||||
downstream.error,
|
||||
)
|
||||
}
|
||||
if (downstream.type === 'decision' && downstream.decision?.kind === 'retry') {
|
||||
return downstream.decision
|
||||
}
|
||||
} else if (!policy.retryableCodes.includes(failure.code)) {
|
||||
return next()
|
||||
}
|
||||
|
||||
const policyKey = retryPolicyKey(policy)
|
||||
const firstPriorTurn = turn - priorFailures.length
|
||||
const priorPolicyRetry = agent.session.events.findLast((event): event is SessionEvent<'llm/retry'> =>
|
||||
event.type === 'llm/retry'
|
||||
&& event.data.turn >= firstPriorTurn
|
||||
&& event.data.turn < turn
|
||||
&& event.data.provider === provider
|
||||
&& event.data.policyKey === policyKey,
|
||||
)
|
||||
const previousRetry = priorPolicyRetry?.data.retry ?? 0
|
||||
if (policy.mode === 'normal' && previousRetry >= policy.maxRetries) return next()
|
||||
const retry = previousRetry + 1
|
||||
let delayMs: number
|
||||
if (failure.providerRetryAfterMs !== undefined
|
||||
&& Number.isFinite(failure.providerRetryAfterMs)
|
||||
&& failure.providerRetryAfterMs > 0) {
|
||||
if (failure.providerRetryAfterMs > policy.maxDelayMs) {
|
||||
if (policy.mode === 'normal') return next()
|
||||
delayMs = localDelay(policy, retry, random)
|
||||
} else {
|
||||
delayMs = failure.providerRetryAfterMs
|
||||
}
|
||||
} else {
|
||||
delayMs = localDelay(policy, retry, random)
|
||||
}
|
||||
|
||||
return backoff(agent, turn, step, failure, provider, policy, policyKey, retry, delayMs, signal)
|
||||
}
|
||||
|
||||
const disposeListener = ctx.on('agent/request-error', (
|
||||
agent: Agent,
|
||||
turn: number,
|
||||
step: number,
|
||||
error: RequestError,
|
||||
failure: LlmFailure,
|
||||
priorFailures: readonly LlmFailure[],
|
||||
policy: ResolvedRetryPolicy | undefined,
|
||||
signal: AbortSignal,
|
||||
next: () => Promise<RequestErrorAction>,
|
||||
) => {
|
||||
@@ -197,30 +254,12 @@ export function apply(ctx: Context, config: Config = {}, internals: RetryInterna
|
||||
// removed. Lifetime cancellation must prevent that stale callback from
|
||||
// entering a downstream policy after disposal.
|
||||
if (lifetime.signal.aborted) return Promise.resolve<RequestErrorAction>(undefined)
|
||||
if (!resolved.retryableCodes.has(failure.code)) return next()
|
||||
const priorRetries = retries.get(agent) ?? 0
|
||||
if (priorRetries >= resolved.maxTransientRetries) return next()
|
||||
|
||||
const retry = priorRetries + 1
|
||||
let delayMs: number
|
||||
if (failure.providerRetryAfterMs !== undefined
|
||||
&& Number.isFinite(failure.providerRetryAfterMs)
|
||||
&& failure.providerRetryAfterMs > 0) {
|
||||
if (failure.providerRetryAfterMs > resolved.maxDelayMs) return next()
|
||||
delayMs = failure.providerRetryAfterMs
|
||||
} else {
|
||||
delayMs = localDelay(resolved, retry, random)
|
||||
}
|
||||
|
||||
const tracked = backoff(agent, turn, step, failure, retry, delayMs, signal)
|
||||
.finally(() => active.delete(tracked))
|
||||
active.add(tracked)
|
||||
return tracked
|
||||
return track(recover(agent, turn, step, error, failure, priorFailures, policy, signal, next))
|
||||
})
|
||||
|
||||
ctx.effect(() => async () => {
|
||||
disposeListener()
|
||||
lifetime.abort(new Error('llm-retry plugin disposed'))
|
||||
await Promise.allSettled([...active])
|
||||
}, 'llm-retry: abort and drain backoffs')
|
||||
}, 'llm-retry: abort and drain active recovery')
|
||||
}
|
||||
@@ -2,8 +2,10 @@
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { LlmFailure } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import type { InvariantFailure, InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
import { providerForClosedStep } from './history.ts'
|
||||
import type {} from './index.ts'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-llm-retry'
|
||||
@@ -13,6 +15,32 @@ export const name = 'llm-retry-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/** Validate the complete provider-neutral failure payload at the durable boundary. */
|
||||
function validateFailure(value: unknown, fail: InvariantFailure): asserts value is LlmFailure {
|
||||
if (typeof value !== 'object' || value === null) {
|
||||
fail('llm/retry failure must be an object')
|
||||
}
|
||||
const failure = value as Partial<LlmFailure>
|
||||
if (typeof failure.message !== 'string' || failure.message.length === 0) {
|
||||
fail('llm/retry failure.message must be a non-empty string')
|
||||
}
|
||||
if (typeof failure.code !== 'string' || failure.code.length === 0) {
|
||||
fail('llm/retry failure.code must be a non-empty string')
|
||||
}
|
||||
if (failure.status !== undefined
|
||||
&& (!Number.isInteger(failure.status) || failure.status < 100 || failure.status > 599)) {
|
||||
fail('llm/retry failure.status must be an integer from 100 through 599 when present')
|
||||
}
|
||||
if (failure.providerRetryAfterMs !== undefined
|
||||
&& (!Number.isFinite(failure.providerRetryAfterMs) || failure.providerRetryAfterMs <= 0)) {
|
||||
fail('llm/retry failure.providerRetryAfterMs must be a positive finite number when present')
|
||||
}
|
||||
if (failure.requestId !== undefined
|
||||
&& (typeof failure.requestId !== 'string' || failure.requestId.length === 0)) {
|
||||
fail('llm/retry failure.requestId must be a non-empty string when present')
|
||||
}
|
||||
}
|
||||
|
||||
/** Find the first turn in the structured-failure retry chain containing `turn`. */
|
||||
function retryChainStart(history: readonly SessionEvent[], turn: number): number {
|
||||
let startIndex = history.findLastIndex(
|
||||
@@ -47,15 +75,35 @@ function validateRetry(
|
||||
event: SessionEvent<'llm/retry'>,
|
||||
fail: InvariantFailure,
|
||||
): void {
|
||||
const { turn, step, retry, maxRetries, delayMs } = event.data
|
||||
const { turn, step, provider, mode, policyKey, retry, delayMs } = event.data
|
||||
const failure: unknown = event.data.failure
|
||||
validateFailure(failure, fail)
|
||||
if (!Number.isSafeInteger(retry) || retry < 1) {
|
||||
fail('llm/retry retry must be a positive safe integer')
|
||||
}
|
||||
if (!Number.isSafeInteger(maxRetries) || maxRetries < 1 || retry > maxRetries) {
|
||||
fail(`llm/retry retry ${retry} must not exceed a positive safe maxRetries ${maxRetries}`)
|
||||
if (typeof provider !== 'string' || provider.length === 0) {
|
||||
fail('llm/retry provider must be a non-empty string')
|
||||
}
|
||||
if (!(delayMs >= 0 && delayMs <= MAX_TIMER_DELAY_MS)) {
|
||||
fail(`llm/retry delayMs must be within 0..${MAX_TIMER_DELAY_MS}`)
|
||||
if (typeof policyKey !== 'string' || policyKey.length === 0) {
|
||||
fail('llm/retry policyKey must be a non-empty string')
|
||||
}
|
||||
switch (mode) {
|
||||
case 'normal': {
|
||||
const { maxRetries } = event.data
|
||||
if (!Number.isSafeInteger(maxRetries) || maxRetries < 1 || retry > maxRetries) {
|
||||
fail(`llm/retry retry ${retry} must not exceed a positive safe maxRetries ${maxRetries}`)
|
||||
}
|
||||
break
|
||||
}
|
||||
case 'always':
|
||||
if ('maxRetries' in event.data) fail('llm/retry always mode must omit maxRetries')
|
||||
break
|
||||
default:
|
||||
fail(`llm/retry mode must be normal or always, got ${String(mode)}`)
|
||||
}
|
||||
if (typeof delayMs !== 'number' || !Number.isFinite(delayMs)
|
||||
|| delayMs < 0 || delayMs > MAX_TIMER_DELAY_MS) {
|
||||
fail(`llm/retry delayMs must be a finite number within 0..${MAX_TIMER_DELAY_MS}`)
|
||||
}
|
||||
|
||||
const currentTurnEvents: SessionEvent[] = []
|
||||
@@ -86,6 +134,10 @@ function validateRetry(
|
||||
if (closedStep === undefined || step !== closedStep) {
|
||||
fail(`llm/retry names step ${step}, but the latest closed step is ${String(closedStep)}`)
|
||||
}
|
||||
const routedProvider = providerForClosedStep(history, turn, step)
|
||||
if (routedProvider !== provider) {
|
||||
fail(`llm/retry provider ${provider} does not match the failed request provider ${String(routedProvider)}`)
|
||||
}
|
||||
|
||||
const chainStart = retryChainStart(history, turn)
|
||||
const chain = history.slice(Math.max(chainStart, 0))
|
||||
@@ -95,9 +147,11 @@ function validateRetry(
|
||||
if (chainRetries.some(prior => prior.data.turn === turn && prior.data.step === step)) {
|
||||
fail(`llm/retry duplicates the retry record for turn ${turn}/step ${step}`)
|
||||
}
|
||||
const expectedRetry = chainRetries.length + 1
|
||||
const priorPolicyRetry = chainRetries.findLast(prior =>
|
||||
prior.data.provider === provider && prior.data.policyKey === policyKey)
|
||||
const expectedRetry = (priorPolicyRetry?.data.retry ?? 0) + 1
|
||||
if (retry !== expectedRetry) {
|
||||
fail(`llm/retry retry ${retry} must equal retry-chain position ${expectedRetry}`)
|
||||
fail(`llm/retry retry ${retry} must equal provider policy retry ${expectedRetry}`)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SessionStore, { SessionId, type Session } from '@deepseek-ai/dsh-session'
|
||||
import { ProviderRequestId } from '@deepseek-ai/dsh-llm'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import InvariantService from '@deepseek-ai/dsh-invariants'
|
||||
import * as RetryInvariant from '@deepseek-ai/dsh-llm-retry/invariant'
|
||||
import { providerForClosedStep } from '../src/history.ts'
|
||||
|
||||
async function setup(): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
@@ -13,207 +15,272 @@ async function setup(): Promise<Context> {
|
||||
return ctx
|
||||
}
|
||||
|
||||
const failure = { message: 'provider busy', code: 'RATE_LIMIT', status: 429 }
|
||||
|
||||
function closeStep(ctx: Context, id: string, turn = 1, step = 1) {
|
||||
const session = ctx.sessions.create(SessionId(id))
|
||||
session.append('turn/start', {
|
||||
turn,
|
||||
trigger: turn === 1
|
||||
? { kind: 'message', source: { kind: 'user' } }
|
||||
: { kind: 'retry' },
|
||||
})
|
||||
session.append('turn/start', { turn, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn, step })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'mock', model: 'mock' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('step/end', { turn, step })
|
||||
return session
|
||||
}
|
||||
|
||||
function appendRetryTurn(session: Session, turn: number) {
|
||||
session.append('turn/start', { turn, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn, step: 1 })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'mock', model: 'mock' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('step/end', { turn, step: 1 })
|
||||
session.append('llm/retry', { turn, step: 1, ...normal })
|
||||
}
|
||||
|
||||
const failure = { message: 'provider busy', code: 'RATE_LIMIT', status: 429 }
|
||||
const normal = {
|
||||
provider: 'mock',
|
||||
mode: 'normal' as const,
|
||||
policyKey: 'normal-policy',
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 1,
|
||||
failure,
|
||||
}
|
||||
const always = {
|
||||
provider: 'mock',
|
||||
mode: 'always' as const,
|
||||
policyKey: 'always-policy',
|
||||
retry: 1,
|
||||
delayMs: 1,
|
||||
failure,
|
||||
}
|
||||
|
||||
describe('llm-retry invariants', () => {
|
||||
it('accepts increasing retry schedules for successive failed turns', async () => {
|
||||
it('has no provider without the requested closed step or a route marker', () => {
|
||||
expect(providerForClosedStep([], 1, 1)).toBeUndefined()
|
||||
expect(providerForClosedStep([{
|
||||
type: 'step/end',
|
||||
data: { turn: 1, step: 1 },
|
||||
}] as never, 1, 1)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('accepts bounded and unbounded records after successive closed steps', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-valid')
|
||||
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 500, failure,
|
||||
})
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn: 2, step: 1 })
|
||||
session.append('step/end', { turn: 2, step: 1 })
|
||||
session.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 2, maxRetries: 2, delayMs: 0, failure,
|
||||
turn: 2, step: 1, ...normal, retry: 2, delayMs: 0,
|
||||
})
|
||||
const unbounded = closeStep(ctx, 'retry-invariant-always')
|
||||
unbounded.append('llm/retry', { turn: 1, step: 1, ...always })
|
||||
}).not.toThrow()
|
||||
expect(() => { ctx.emit('tools/change') }).not.toThrow()
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ retry: 0, maxRetries: 2, delayMs: 1 }, /positive safe integer/],
|
||||
[{ retry: 1.5, maxRetries: 2, delayMs: 1 }, /positive safe integer/],
|
||||
[{ retry: 1, maxRetries: 0, delayMs: 1 }, /positive safe maxRetries/],
|
||||
[{ retry: 1, maxRetries: 1.5, delayMs: 1 }, /positive safe maxRetries/],
|
||||
[{ retry: 3, maxRetries: 2, delayMs: 1 }, /must not exceed/],
|
||||
[{ retry: 1, maxRetries: 2, delayMs: -1 }, /delayMs/],
|
||||
[{ retry: 1, maxRetries: 2, delayMs: MAX_TIMER_DELAY_MS + 1 }, /delayMs/],
|
||||
])('rejects invalid retry bounds %#', async (data, message) => {
|
||||
it('validates the complete durable failure payload', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, `retry-invariant-bounds-${data.retry}-${data.maxRetries}-${data.delayMs}`)
|
||||
const complete = closeStep(ctx, 'retry-invariant-complete-failure')
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...data, failure })
|
||||
complete.append('llm/retry', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
...always,
|
||||
failure: {
|
||||
message: 'provider busy',
|
||||
code: 'RATE_LIMIT',
|
||||
status: 429,
|
||||
providerRetryAfterMs: 25,
|
||||
requestId: ProviderRequestId('request-1'),
|
||||
},
|
||||
})
|
||||
}).not.toThrow()
|
||||
|
||||
const invalidFailures: readonly [string, unknown, RegExp][] = [
|
||||
['null', null, /failure must be an object/],
|
||||
['message-type', { message: 1, code: 'RATE_LIMIT' }, /failure\.message/],
|
||||
['message-empty', { message: '', code: 'RATE_LIMIT' }, /failure\.message/],
|
||||
['code-type', { message: 'failed', code: 1 }, /failure\.code/],
|
||||
['code-empty', { message: 'failed', code: '' }, /failure\.code/],
|
||||
['status-type', { message: 'failed', code: 'RATE_LIMIT', status: 429.5 }, /failure\.status/],
|
||||
['status-low', { message: 'failed', code: 'RATE_LIMIT', status: 99 }, /failure\.status/],
|
||||
['status-high', { message: 'failed', code: 'RATE_LIMIT', status: 600 }, /failure\.status/],
|
||||
[
|
||||
'retry-after-type',
|
||||
{ message: 'failed', code: 'RATE_LIMIT', providerRetryAfterMs: '25' },
|
||||
/failure\.providerRetryAfterMs/,
|
||||
],
|
||||
[
|
||||
'retry-after-zero',
|
||||
{ message: 'failed', code: 'RATE_LIMIT', providerRetryAfterMs: 0 },
|
||||
/failure\.providerRetryAfterMs/,
|
||||
],
|
||||
['request-id-type', { message: 'failed', code: 'RATE_LIMIT', requestId: 1 }, /failure\.requestId/],
|
||||
['request-id-empty', { message: 'failed', code: 'RATE_LIMIT', requestId: '' }, /failure\.requestId/],
|
||||
]
|
||||
for (const [name, invalidFailure, message] of invalidFailures) {
|
||||
const session = closeStep(ctx, `retry-invariant-failure-${name}`)
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, ...always, failure: invalidFailure,
|
||||
} as never)
|
||||
}).toThrow(message)
|
||||
}
|
||||
})
|
||||
|
||||
it.each([
|
||||
['retry-zero', { ...normal, retry: 0 }, /positive safe integer/],
|
||||
['retry-fraction', { ...normal, retry: 1.5 }, /positive safe integer/],
|
||||
['max-zero', { ...normal, maxRetries: 0 }, /positive safe maxRetries/],
|
||||
['max-fraction', { ...normal, maxRetries: 1.5 }, /positive safe maxRetries/],
|
||||
['over-budget', { ...normal, retry: 3 }, /must not exceed/],
|
||||
['always-maximum', { ...always, maxRetries: 2 }, /always mode must omit maxRetries/],
|
||||
['unknown-mode', { ...always, mode: 'sometimes' }, /mode must be normal or always/],
|
||||
['empty-provider', { ...always, provider: '' }, /provider must be a non-empty string/],
|
||||
['empty-policy-key', { ...always, policyKey: '' }, /policyKey must be a non-empty string/],
|
||||
['delay-negative', { ...normal, delayMs: -1 }, /delayMs/],
|
||||
['delay-overflow', { ...normal, delayMs: MAX_TIMER_DELAY_MS + 1 }, /delayMs/],
|
||||
['delay-type', { ...normal, delayMs: '1' }, /delayMs/],
|
||||
])('rejects invalid retry data: %s', async (name, data, message) => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, `retry-invariant-${name}`)
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...data } as never)
|
||||
}).toThrow(message)
|
||||
})
|
||||
|
||||
it('rejects a retry record appended after its turn already closed', async () => {
|
||||
const ctx = await setup()
|
||||
const closed = closeStep(ctx, 'retry-invariant-closed-turn')
|
||||
closed.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
expect(() => {
|
||||
closed.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/inside an open turn/)
|
||||
})
|
||||
|
||||
it('starts a fresh chain when the turn before a retry trigger did not fail structurally', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-completed-predecessor')
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn: 2, step: 1 })
|
||||
session.append('step/end', { turn: 2, step: 1 })
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('walks the chain across non-boundary events and stops at an unmatched turn start', async () => {
|
||||
const ctx = await setup()
|
||||
// The failed predecessor's turn/start is outside this log prefix (e.g. a
|
||||
// truncated replay): the chain walk must stop rather than loop or throw.
|
||||
const session = ctx.sessions.create(SessionId('retry-invariant-unmatched-start'))
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
// A durable non-boundary record between the turns exercises the walk over
|
||||
// non-turn/end events.
|
||||
session.append('todo/write', { todos: [] })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn: 2, step: 1 })
|
||||
session.append('step/end', { turn: 2, step: 1 })
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('requires an open turn and its latest closed step', async () => {
|
||||
it('rejects records outside the latest closed step of an open turn', async () => {
|
||||
const ctx = await setup()
|
||||
const absent = ctx.sessions.create(SessionId('retry-invariant-no-turn'))
|
||||
expect(() => {
|
||||
absent.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
absent.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/inside an open turn/)
|
||||
|
||||
const wrongTurn = closeStep(ctx, 'retry-invariant-wrong-turn')
|
||||
expect(() => {
|
||||
wrongTurn.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
wrongTurn.append('llm/retry', { turn: 2, step: 1, ...normal })
|
||||
}).toThrow(/open turn is 1/)
|
||||
|
||||
const openStep = ctx.sessions.create(SessionId('retry-invariant-open-step'))
|
||||
openStep.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
openStep.append('step/start', { turn: 1, step: 1 })
|
||||
expect(() => {
|
||||
openStep.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
openStep.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/step 1 is still open/)
|
||||
|
||||
const noStep = ctx.sessions.create(SessionId('retry-invariant-no-step'))
|
||||
noStep.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
expect(() => {
|
||||
noStep.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/latest closed step is undefined/)
|
||||
|
||||
const wrongStep = closeStep(ctx, 'retry-invariant-wrong-step')
|
||||
expect(() => {
|
||||
wrongStep.append('llm/retry', {
|
||||
turn: 1, step: 2, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
wrongStep.append('llm/retry', { turn: 1, step: 2, ...normal })
|
||||
}).toThrow(/latest closed step is 1/)
|
||||
|
||||
const closedTurn = closeStep(ctx, 'retry-invariant-closed-turn')
|
||||
closedTurn.append('turn/end', { turn: 1, reason: { kind: 'aborted' } })
|
||||
expect(() => {
|
||||
closedTurn.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
}).toThrow(/inside an open turn/)
|
||||
})
|
||||
|
||||
it('rejects duplicate and out-of-sequence retry schedules', async () => {
|
||||
it('rejects a second retry record for the same step', async () => {
|
||||
const ctx = await setup()
|
||||
const duplicate = closeStep(ctx, 'retry-invariant-duplicate')
|
||||
duplicate.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
expect(() => {
|
||||
duplicate.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 2, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/duplicates/)
|
||||
const session = closeStep(ctx, 'retry-invariant-duplicate')
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
|
||||
const nonIncreasing = closeStep(ctx, 'retry-invariant-non-increasing')
|
||||
nonIncreasing.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
nonIncreasing.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
nonIncreasing.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
nonIncreasing.append('step/start', { turn: 2, step: 1 })
|
||||
nonIncreasing.append('step/end', { turn: 2, step: 1 })
|
||||
expect(() => {
|
||||
nonIncreasing.append('llm/retry', {
|
||||
turn: 2, step: 1, retry: 1, maxRetries: 3, delayMs: 1, failure,
|
||||
})
|
||||
}).toThrow(/retry-chain position 2/)
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal, retry: 2 })
|
||||
}).toThrow(/duplicates the retry record/)
|
||||
})
|
||||
|
||||
it('resets retry numbering after a completed chain', async () => {
|
||||
it('binds retry numbering to the provider policy and resets it after success', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-reset')
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
session.append('step/start', { turn: 2, step: 1 })
|
||||
session.append('step/end', { turn: 2, step: 1 })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
session.append('turn/start', {
|
||||
turn: 3,
|
||||
trigger: { kind: 'message', source: { kind: 'user' } },
|
||||
})
|
||||
session.append('step/start', { turn: 3, step: 1 })
|
||||
session.append('step/end', { turn: 3, step: 1 })
|
||||
|
||||
const mismatch = closeStep(ctx, 'retry-invariant-numbering')
|
||||
mismatch.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
mismatch.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
mismatch.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
mismatch.append('step/start', { turn: 2, step: 1 })
|
||||
mismatch.append('step/end', { turn: 2, step: 1 })
|
||||
expect(() => {
|
||||
session.append('llm/retry', {
|
||||
turn: 3, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
mismatch.append('llm/retry', { turn: 2, step: 1, ...normal, retry: 1 })
|
||||
}).toThrow(/must equal provider policy retry 2/)
|
||||
|
||||
const reset = closeStep(ctx, 'retry-invariant-reset')
|
||||
reset.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
reset.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, failure } })
|
||||
reset.append('turn/start', { turn: 2, trigger: { kind: 'retry' } })
|
||||
reset.append('step/start', { turn: 2, step: 1 })
|
||||
reset.append('assistant/message', {
|
||||
turn: 2,
|
||||
step: 1,
|
||||
content: [{ type: 'text', text: 'success' }],
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
}, { surfaceOp: 'append' })
|
||||
reset.append('step/end', { turn: 2, step: 1 })
|
||||
reset.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
reset.append('turn/start', { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reset.append('step/start', { turn: 3, step: 1 })
|
||||
reset.append('step/end', { turn: 3, step: 1 })
|
||||
expect(() => {
|
||||
reset.append('llm/retry', { turn: 3, step: 1, ...normal })
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('starts a fresh retry chain after incomplete predecessor boundaries', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
|
||||
const missingEnd = ctx.sessions.create(SessionId('retry-invariant-missing-end'))
|
||||
missingEnd.append('user/message', {
|
||||
content: [{ type: 'text', text: 'idle context' }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
appendRetryTurn(missingEnd, 2)
|
||||
|
||||
const nonFailureEnd = ctx.sessions.create(SessionId('retry-invariant-non-failure-end'))
|
||||
nonFailureEnd.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
nonFailureEnd.append('user/message', {
|
||||
content: [{ type: 'text', text: 'idle context' }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
appendRetryTurn(nonFailureEnd, 2)
|
||||
|
||||
const missingStart = ctx.sessions.create(SessionId('retry-invariant-missing-start'))
|
||||
missingStart.append('turn/end', {
|
||||
turn: 1,
|
||||
reason: { kind: 'error', step: 1, failure },
|
||||
})
|
||||
appendRetryTurn(missingStart, 2)
|
||||
|
||||
await ctx.plugin(InvariantService)
|
||||
await expect(ctx.plugin(RetryInvariant)).resolves.toBeDefined()
|
||||
})
|
||||
|
||||
it('rejects a provider that does not match the failed request route', async () => {
|
||||
const ctx = await setup()
|
||||
const session = closeStep(ctx, 'retry-invariant-provider')
|
||||
expect(() => {
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...always, provider: 'other' })
|
||||
}).toThrow(/does not match the failed request provider mock/)
|
||||
})
|
||||
|
||||
it('validates existing histories on late registration', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = ctx.sessions.create(SessionId('retry-invariant-late'))
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('llm/retry', { turn: 1, step: 1, ...normal })
|
||||
await ctx.plugin(InvariantService)
|
||||
await expect(ctx.plugin(RetryInvariant)).rejects.toThrow(/inside an open turn/)
|
||||
})
|
||||
|
||||
it('accepts a valid mixed pre-existing history on late registration', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = ctx.sessions.create(SessionId('retry-invariant-late-valid'))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('llm/retry', {
|
||||
turn: 1, step: 1, retry: 1, maxRetries: 2, delayMs: 1, failure,
|
||||
})
|
||||
await ctx.plugin(InvariantService)
|
||||
await expect(ctx.plugin(RetryInvariant)).resolves.toBeDefined()
|
||||
})
|
||||
})
|
||||
@@ -8,8 +8,8 @@ import Loader from '@cordisjs/plugin-loader'
|
||||
import Include from '@cordisjs/plugin-include'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import LlmService, { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { LlmAdapter, LlmError, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, ResolvedRetryPolicy, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
@@ -20,6 +20,16 @@ let context: Context | undefined
|
||||
|
||||
class TransientOnceAdapter extends LlmAdapter {
|
||||
requests = 0
|
||||
private readonly retryPolicy = resolveRetryPolicy({
|
||||
mode: 'normal',
|
||||
maxRetries: 1,
|
||||
retryableCodes: ['RATE_LIMIT', 'SERVER'],
|
||||
backoff: { initialDelayMs: 1, maxDelayMs: 1, jitterRatio: 0 },
|
||||
}, 'loader test provider retryPolicy')
|
||||
|
||||
override providerRetryPolicy(_provider: string): ResolvedRetryPolicy {
|
||||
return this.retryPolicy
|
||||
}
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.requests += 1
|
||||
@@ -75,7 +85,7 @@ describe('real Loader composition', () => {
|
||||
// Real-Loader composition resolves workspace packages through tsx at test
|
||||
// time; first resolution after the host/client program split is slow enough
|
||||
// to trip the default 5s budget on cold caches.
|
||||
it('loads the flat policy and records recovery through the shipping loop', { timeout: 60_000 }, async () => {
|
||||
it('loads provider-supplied policy and records recovery through the shipping loop', { timeout: 60_000 }, async () => {
|
||||
const loaded = await loadYaml([
|
||||
"- name: '@deepseek-ai/dsh-llm'",
|
||||
"- name: '@deepseek-ai/dsh-session'",
|
||||
@@ -83,12 +93,6 @@ describe('real Loader composition', () => {
|
||||
"- name: '@deepseek-ai/dsh-tools'",
|
||||
"- name: '@deepseek-ai/dsh-agent'",
|
||||
"- name: '@deepseek-ai/dsh-llm-retry'",
|
||||
' config:',
|
||||
' maxTransientRetries: 1',
|
||||
' initialDelayMs: 1',
|
||||
' maxDelayMs: 1',
|
||||
' jitterRatio: 0',
|
||||
' retryableCodes: [RATE_LIMIT, SERVER]',
|
||||
"- name: '@deepseek-ai/dsh-agent-loop'",
|
||||
])
|
||||
|
||||
|
||||
@@ -34,12 +34,18 @@ describe.each(['jsonl', 'sqlite'] as const)('%s retry-event persistence', (kind)
|
||||
const session = ctx.sessions.create(SessionId(`retry-${kind}`))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: 'mock', model: 'mock' } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
const event = session.append('llm/retry', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
provider: 'mock',
|
||||
mode: 'always',
|
||||
policyKey: '["always",500,10000,0.1]',
|
||||
retry: 1,
|
||||
maxRetries: 2,
|
||||
delayMs: 750,
|
||||
failure: { message: 'provider busy', code: 'RATE_LIMIT', status: 429 },
|
||||
})
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -39,13 +39,17 @@ async function harness(
|
||||
apiKey: 'mock-key',
|
||||
baseURL,
|
||||
streamIdleTimeoutMs: options.streamIdleTimeoutMs ?? 1_000,
|
||||
retryPolicy: {
|
||||
mode: 'normal',
|
||||
maxRetries: 2,
|
||||
backoff: {
|
||||
initialDelayMs: options.initialDelayMs ?? 10,
|
||||
maxDelayMs: options.initialDelayMs ?? 10,
|
||||
jitterRatio: 0,
|
||||
},
|
||||
},
|
||||
})
|
||||
await ctx.plugin(Retry, {
|
||||
maxTransientRetries: 2,
|
||||
initialDelayMs: options.initialDelayMs ?? 10,
|
||||
maxDelayMs: options.initialDelayMs ?? 10,
|
||||
jitterRatio: 0,
|
||||
})
|
||||
await ctx.plugin(Retry)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
return ctx
|
||||
}
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/** Build the package root and invariant companion as independent bundles. */
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/index.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/invariant.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
])
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: 3efb3ece3caadeaceaa3c504ba4b10ddb951127a
|
||||
README.zh.md: 4af8b8d08cc96ff0e36b10b15e1d86afd43004c9
|
||||
README.md: 2328188e420df6de60f024982a31d37a858a303e
|
||||
README.zh.md: 586767a9e790fa68d27673836700fd789ce6a180
|
||||
@@ -12,15 +12,16 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved.
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
`LlmService` preserves errors from final adapter selection, synchronous dispatch, iterator construction, and iteration, and binds their provenance to the exact stream handle returned for that model call. `isLlmAdapterFailure(stream, value)` reports only errors from that call's final adapter boundary; `llmFailureOf(stream, value)` returns the adjacent immutable `LlmFailure`; `llmRetryPolicyOf(stream)` returns the immutable policy of the exact registration selected at that boundary, even if the route is later disposed or replaced. A call that never reaches a final adapter has no serving policy. Nested model calls, `llm/stream` middleware, and downstream consumer failures remain unclassified for the outer call. Classification never replaces or mutates the adapter's original coded `Error`.
|
||||
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures the adapter's retry policy for each route, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
|
||||
|
||||
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
|
||||
|
||||
@@ -34,7 +35,7 @@ Reasoning identifiers are opaque adapter-owned strings rather than a core enum.
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` to supply provider-owned recovery configuration, `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use bounded normal retry policy, use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
@@ -76,7 +77,7 @@ Pass-through; the registry preserves the assembled request prefix, while the sel
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No default retry/caching/rate-limit policy ships in this service** — `llm/stream` remains a single-attempt call-wrapper seam; the agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure. `@deepseek-ai/dsh-llm-retry` is an optional policy plugin loaded by the shared example spine.
|
||||
- **No retry execution, caching, or rate limiting ships in this service** — provider registration stores retry policy, but `llm/stream` remains a single-attempt call-wrapper seam. The agent loop separately offers proven model-request failures to `agent/request-error`, whose default preserves the original failure; `@deepseek-ai/dsh-llm-retry` is the optional executor loaded by the shared example spine.
|
||||
- **`GenerateOptions` sampling is `temperature`/`maxTokens`/`stop` only** — no `tool_choice`, `top_p`, or penalty fields; the vocabulary grows when a producer lands ([dropped inert knobs](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md)).
|
||||
- **Producer-gated variants stay out until produced** — `prefill`, per-tool `strict`, block `cache` hints, and the `agent` message-source variant were pruned as producerless ([Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md)).
|
||||
- **`BlockAssembler` handles core block kinds only** — a plugin-added block type whose stream is never closed by `block-end` makes `blocks()` throw.
|
||||
|
||||
@@ -12,15 +12,16 @@
|
||||
|
||||
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber dispose。
|
||||
- `ctx.llm.listProviders(): LlmProviderInfo[]` 按注册顺序描述已注册提供方路由。
|
||||
- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方重试策略,并解析 normal 默认值。
|
||||
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` 发现某个已注册提供方当前公布的模型。
|
||||
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` 从拥有精确路由的适配器解析经校验的确切模型身份、可用上下文和推理(reasoning)元数据;异步适配器可选地支持取消。
|
||||
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` 校验显式推理强度,并填入适配器配置的默认值,但不自动调整。
|
||||
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` 解析配置并将其当前适配器注册捕获为一次可取消、一次性调用。
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` 将一次模型调用流式输出为原始 chunk(token 级 delta)。消费方使用 `BlockAssembler` 将 chunk 组装为块/消息。
|
||||
|
||||
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误,并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误;`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`。
|
||||
`LlmService` 保留来自最终适配器选择、同步 dispatch、iterator 构造与迭代的错误,并将其溯源绑定到该次模型调用返回的精确流句柄。`isLlmAdapterFailure(stream, value)` 只报告该调用最终适配器边界的错误;`llmFailureOf(stream, value)` 返回相邻的不可变 `LlmFailure`;`llmRetryPolicyOf(stream)` 返回在该边界选中的确切注册所对应的不可变策略,即使之后释放或替换路由也不变。未到达最终适配器的调用没有服务策略。嵌套模型调用、`llm/stream` middleware 和下游消费方失败对外层调用仍未分类。分类绝不替换或更改适配器的原始编码 `Error`。
|
||||
|
||||
提供方与模型元数据是发现表层,不是路由白名单。`registerAdapter()` 仍拥有提供方排他性,适配器则可以接受 `listModels()` 中不存在的模型 id;消费方禁止因模型未列出而拒绝请求。返回的元数据与输入脱离,无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。
|
||||
提供方与模型元数据是发现表层,不是路由白名单。`registerAdapter()` 仍拥有提供方排他性,并为每条路由捕获适配器的重试策略;适配器则可以接受 `listModels()` 中不存在的模型 id,消费方禁止因模型未列出而拒绝请求。返回的 selector 元数据与输入脱离,无效或重复适配器配置项会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。
|
||||
|
||||
确切模型元数据是独立的正确性查询,不是 catalog 装饰或全局 LLM 设置。`resolveModelInfo()` 会向拥有精确提供方/模型路由的适配器查询一次;适配器可以描述未列出的动态模型,缺少 `context` 或 `reasoning` 字段只表示相应能力不可用。无效的身份、上下文或推理元数据会以 `INVALID_MODEL_INFO`、`INVALID_MODEL_CONTEXT` 或 `INVALID_MODEL_REASONING` 失败。
|
||||
|
||||
@@ -34,7 +35,7 @@
|
||||
|
||||
### 扩展点
|
||||
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现将路由和模型 id 用作名称,不公布模型,也不返回容量或推理元数据。
|
||||
- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方持有的恢复配置,覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略,将路由和模型 id 用作名称,不公布模型,也不返回容量或推理元数据。
|
||||
- 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。发出 chunk 后重试的包装层没有持久尝试边界;因此已发布 agent 重试策略改用 `agent/request-error`。
|
||||
|
||||
### 内容块词汇(`types.ts`)
|
||||
@@ -76,7 +77,7 @@
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **本服务不内置默认重试/缓存/速率限制策略**:`llm/stream` 仍是单次尝试调用包装 seam;agent loop 会将已验证模型请求失败单独提供给 `agent/request-error`,其默认行为是保留原始失败。`@deepseek-ai/dsh-llm-retry` 是共享示例 spine 加载的可选策略插件。
|
||||
- **本服务不执行重试、缓存或速率限制**:提供方注册会存储重试策略,但 `llm/stream` 仍是单次尝试调用包装 seam。agent loop 会将已验证模型请求失败单独提供给 `agent/request-error`,其默认行为是保留原始失败;`@deepseek-ai/dsh-llm-retry` 是共享示例 spine 加载的可选执行器。
|
||||
- **`GenerateOptions` 采样只包含 `temperature`/`maxTokens`/`stop`**:没有 `tool_choice`、`top_p` 或 penalty 字段;有产生方落地时词汇才会增长(见 [已删除惰性旋钮](../../../.agents/notes/archived/simplification/2026-07-04-drop-inert-request-knobs.md))。
|
||||
- **由产生方调节的变体在实际产生前保持在外**:`prefill`、每工具 `strict`、块 `cache` 提示与 `agent` 消息源变体因没有产生方而被剪除(见 [Agent Note](../../../.agents/notes/archived/simplification/2026-07-04-prune-producerless-vocabulary-variants.md))。
|
||||
- **`BlockAssembler` 只处理核心块 kind**:如果插件添加块类型的流从未由 `block-end` 关闭,`blocks()` 会抛出异常。
|
||||
|
||||
@@ -38,11 +38,16 @@
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-timeout": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-timeout": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
@@ -6,9 +6,15 @@
|
||||
|
||||
import { HarnessError } from './error.ts'
|
||||
import type { LlmFailure, StreamChunk } from './types.ts'
|
||||
import type { ResolvedRetryPolicy } from './retry-policy.ts'
|
||||
|
||||
/** Errors and normalized facts proven to originate in one model call's final adapter boundary. */
|
||||
export type AdapterFailureScope = WeakMap<Error, LlmFailure>
|
||||
/** Call-local facts captured when one model call enters its final adapter boundary. */
|
||||
export interface AdapterFailureScope {
|
||||
/** Errors and normalized facts proven to originate in this call's final adapter boundary. */
|
||||
readonly failures: WeakMap<Error, LlmFailure>
|
||||
/** Immutable policy of the exact adapter registration selected for this call. */
|
||||
retryPolicy?: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
/** Call-local failure scopes keyed by the exact stream handle returned to a consumer. */
|
||||
const adapterFailureScopes = new WeakMap<AsyncIterable<StreamChunk>, AdapterFailureScope>()
|
||||
@@ -54,7 +60,7 @@ export function markLlmAdapterFailure(
|
||||
message: errorMessage(error),
|
||||
code: harnessErrorCode(error),
|
||||
})
|
||||
failures.set(error, failure)
|
||||
failures.failures.set(error, failure)
|
||||
return error
|
||||
}
|
||||
|
||||
@@ -136,7 +142,7 @@ export function isLlmAdapterFailure(
|
||||
value: unknown,
|
||||
): value is Error & { code?: string } {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error && failures !== undefined && failures.has(value)
|
||||
return value instanceof Error && failures !== undefined && failures.failures.has(value)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -151,5 +157,18 @@ export function llmFailureOf(
|
||||
value: unknown,
|
||||
): LlmFailure | undefined {
|
||||
const failures = adapterFailureScopes.get(stream)
|
||||
return value instanceof Error ? failures?.get(value) : undefined
|
||||
return value instanceof Error ? failures?.failures.get(value) : undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the retry policy of the exact registration selected at this call's
|
||||
* final adapter boundary. The policy remains available after that registration
|
||||
* is disposed or replaced; absence means no final adapter served the call.
|
||||
* @param stream - the exact stream returned by the model call.
|
||||
* @returns the immutable serving-registration policy, or `undefined`.
|
||||
*/
|
||||
export function llmRetryPolicyOf(
|
||||
stream: AsyncIterable<StreamChunk>,
|
||||
): ResolvedRetryPolicy | undefined {
|
||||
return adapterFailureScopes.get(stream)?.retryPolicy
|
||||
}
|
||||
@@ -16,6 +16,8 @@ import type {
|
||||
Message,
|
||||
StreamChunk,
|
||||
} from './types.ts'
|
||||
import { resolveRetryPolicy } from './retry-policy.ts'
|
||||
import type { ResolvedRetryPolicy } from './retry-policy.ts'
|
||||
import type { ProviderRequestId } from './brand.ts'
|
||||
import { callConfigEquals, deepFreeze } from './call-config.ts'
|
||||
import type { LlmCallConfig } from './call-config.ts'
|
||||
@@ -28,10 +30,11 @@ export * from './brand.ts'
|
||||
export * from './never.ts'
|
||||
export * from './error.ts'
|
||||
export * from './types.ts'
|
||||
export * from './retry-policy.ts'
|
||||
export { BlockAssembler } from './assembler.ts'
|
||||
export { callConfigEquals, deepFreeze, isAgentLoopRequest, markAgentLoopRequest } from './call-config.ts'
|
||||
export type { LlmCallConfig } from './call-config.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf } from './adapter-failure.ts'
|
||||
export { isLlmAdapterFailure, llmFailureOf, llmRetryPolicyOf } from './adapter-failure.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
@@ -134,6 +137,15 @@ export abstract class LlmAdapter {
|
||||
return { id: provider, name: provider }
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the provider-owned retry policy captured with this route.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @returns a resolved policy, or `undefined` to use the normal defaults.
|
||||
*/
|
||||
providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined {
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
@@ -204,7 +216,13 @@ export class LlmService extends Service {
|
||||
throw new LlmError(`adapter metadata for provider "${provider}" must preserve its id and have a non-empty name`, 'INVALID_ADAPTER')
|
||||
}
|
||||
unique.add(provider)
|
||||
registrations.push({ adapter, provider: { id: info.id, name: info.name } })
|
||||
const retryPolicy = adapter.providerRetryPolicy(provider)
|
||||
?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`)
|
||||
registrations.push({
|
||||
adapter,
|
||||
provider: { id: info.id, name: info.name },
|
||||
retryPolicy,
|
||||
})
|
||||
}
|
||||
for (const registration of registrations) this.adapters.set(registration.provider.id, registration)
|
||||
yield () => {
|
||||
@@ -224,6 +242,15 @@ export class LlmService extends Service {
|
||||
return [...this.adapters.values()].map(({ provider }) => ({ ...provider }))
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the retry policy captured when one provider route was registered.
|
||||
* @param provider - registered provider route to inspect.
|
||||
* @returns the provider-owned policy, with normal defaults already resolved.
|
||||
*/
|
||||
providerRetryPolicy(provider: string): ResolvedRetryPolicy {
|
||||
return this.registration(provider).retryPolicy
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover models advertised by one registered provider. Catalog membership
|
||||
* is advisory and never changes routing or request validation.
|
||||
@@ -459,6 +486,7 @@ export class LlmService extends Service {
|
||||
let iterator: AsyncIterator<StreamChunk>
|
||||
try {
|
||||
const registration = prepared?.registration ?? this.registration(options.provider)
|
||||
failures.retryPolicy = registration.retryPolicy
|
||||
const resolvedConfig = prepared === undefined
|
||||
? await this.resolveCallConfigFor(registration, options, options.signal)
|
||||
: prepared.config
|
||||
@@ -530,7 +558,7 @@ export class LlmService extends Service {
|
||||
options: GenerateOptions,
|
||||
prepared?: { registration: AdapterRegistration; config: LlmCallConfig },
|
||||
): AsyncIterable<StreamChunk> {
|
||||
const failures: AdapterFailureScope = new WeakMap<Error, LlmFailure>()
|
||||
const failures: AdapterFailureScope = { failures: new WeakMap<Error, LlmFailure>() }
|
||||
const stream = this.ctx.waterfall(
|
||||
this,
|
||||
'llm/stream',
|
||||
@@ -544,6 +572,7 @@ export class LlmService extends Service {
|
||||
interface AdapterRegistration {
|
||||
readonly adapter: LlmAdapter
|
||||
readonly provider: LlmProviderInfo
|
||||
readonly retryPolicy: ResolvedRetryPolicy
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user