refactor(llm): resolve model metadata together

This commit is contained in:
Yichen Jiang
2026-07-26 13:07:27 +08:00
parent baea5018e5
commit 73e7e27799
55 changed files with 607 additions and 460 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-20-routed-model-context-and-compaction-policy.md: b637ba24d4ba5fc25c8cdd515a821ee97883a326
2026-07-20-routed-model-context-and-compaction-policy.zh.md: 084e762ec29ddc0aecb0bf422c147b9d3122726b
2026-07-20-routed-model-context-and-compaction-policy.md: b00c744e4ac79983a1f492d710e7b1542278c4f4
2026-07-20-routed-model-context-and-compaction-policy.zh.md: 88e26aa7a99e8deb434d9a8fd80784be410a999a
@@ -14,9 +14,9 @@ Neither obvious configuration owner is sufficient. Compact-basic is optional and
### Adapters own exact-route capacity
`LlmAdapter.resolveModelContext(provider, model)` optionally returns `LlmModelContext` for one exact route. `LlmService.resolveModelContext()` selects the registered route owner, validates a positive integer `contextWindow`, and returns a detached value. The query is independent of `listModels()`: an unlisted dynamic model may have capacity metadata, and `undefined` means only that the adapter cannot describe capacity.
`LlmAdapter.resolveModel(provider, model, signal?)` returns aggregate metadata for one exact route, with optional `LlmModelContext` under its `context` field. `LlmService.resolveModelInfo()` selects the registered route owner, validates a positive integer `contextWindow`, and returns detached metadata. The query is independent of `listModels()`: an unlisted dynamic model may have capacity metadata, and an absent `context` means only that the adapter cannot describe capacity.
The hand-rolled DeepSeek adapter accepts optional `contextWindow` on each configured model plus an adapter-wide `defaultContextWindow`. Exact model capacity wins; an entry without capacity and an unlisted pass-through id inherit the adapter default, or return `undefined` when it is absent. The two built-in model entries each publish an exact 128,000-token capacity. The pi-ai adapter resolves capacity from the same catalog descriptor that authoritatively resolves the request model.
The hand-rolled DeepSeek adapter accepts optional `contextWindow` on each configured model plus an adapter-wide `defaultContextWindow`. Exact model capacity wins; an entry without capacity and an unlisted pass-through id inherit the adapter default, or omit `context` when it is absent. The two built-in model entries each publish an exact 128,000-token capacity. The pi-ai adapter resolves capacity from the same catalog descriptor that authoritatively resolves the request model.
### Token measurement remains model-agnostic
@@ -14,9 +14,9 @@ Status: implemented
### 适配器拥有精确路由容量
`LlmAdapter.resolveModelContext(provider, model)` 可以为一条精确路由返回 `LlmModelContext``LlmService.resolveModelContext()` 选择已注册的路由所属方,验证 `contextWindow` 为正整数,并返回分离。该查询独立于 `listModels()`:不在目录中的动态模型也可以拥有容量元数据,而 `undefined` 只表示适配器无法描述容量。
`LlmAdapter.resolveModel(provider, model, signal?)` 返回一条精确路由的聚合元数据,其中可选的 `LlmModelContext` 位于 `context` 字段下`LlmService.resolveModelInfo()` 选择已注册的路由所属方,验证 `contextWindow` 为正整数,并返回分离的元数据。该查询独立于 `listModels()`:不在目录中的动态模型也可以拥有容量元数据,而缺少 `context` 只表示适配器无法描述容量。
手写 DeepSeek 适配器允许每个已配置模型提供可选 `contextWindow`,并支持适配器级 `defaultContextWindow`。精确模型容量优先;未提供容量的模型项与未列出的透传 id 会继承适配器默认值,若默认值也不存在则返回 `undefined`。两个内置模型项都公开精确的 128,000 token 容量。pi-ai 适配器从同一个目录描述符解析容量,该描述符也用于权威解析请求模型。
手写 DeepSeek 适配器允许每个已配置模型提供可选 `contextWindow`,并支持适配器级 `defaultContextWindow`。精确模型容量优先;未提供容量的模型项与未列出的透传 id 会继承适配器默认值,若默认值也不存在则省略 `context`。两个内置模型项都公开精确的 128,000 token 容量。pi-ai 适配器从同一个目录描述符解析容量,该描述符也用于权威解析请求模型。
### Token 计量保持模型无关
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-24-adapter-owned-reasoning-effort-capabilities.md: 04aebf5fa61d896ac16b29578a39c9b0e6e38cf4
2026-07-24-adapter-owned-reasoning-effort-capabilities.zh.md: a1c89417562276fd70ac5e5c1be17c9e639eb3ed
2026-07-24-adapter-owned-reasoning-effort-capabilities.md: ffd633a3876c96f312fb274159f3f8422f8811ab
2026-07-24-adapter-owned-reasoning-effort-capabilities.zh.md: 05c9941ad9c0906ab5400b5a215b92381fba3119
@@ -10,11 +10,11 @@ Reasoning strength was adapter configuration only, so a conversation could not d
## Decision
`dsh-llm` represents a reasoning effort as the opaque branded `ReasoningEffortId`. An adapter's `resolveModelReasoning(provider, model)` returns a non-empty ordered list of ids with display metadata and may name one configured default. The core validates metadata, requires an explicit or configured effort to appear exactly in that list, and never clamps or aliases a value.
`dsh-llm` represents a reasoning effort as the opaque branded `ReasoningEffortId`. One adapter-owned `resolveModel(provider, model, signal?)` query returns `LlmResolvedModelInfo`: exact model identity plus optional context and reasoning metadata. `LlmService.resolveModelInfo()` validates and detaches that aggregate. When present, `reasoning.efforts` is a non-empty ordered list of ids with display metadata and may name one configured default. The core requires an explicit or configured effort to appear exactly in that list and never clamps or aliases a value.
`LlmCallConfig` and `GenerateOptions` carry the optional effort. The agent loop prepares the post-`agent/request` config under the active turn signal before writing `request/header`, so defaults and dynamic changes are model-visible only after becoming durable facts. The prepared call retains the exact adapter registration across asynchronous capability resolution, durable header logging, and dispatch; direct `LlmService.stream()` calls likewise capture their final registration before awaiting resolution. A route with no registered adapter retains its proposed config so an `llm/stream` middleware can own and short-circuit it; terminal dispatch still rejects an unhandled route. A resumed loop retains the logged effort only when its initial provider/model route is unchanged; a route change discards the previous model's opaque id.
`LlmCallConfig` and `GenerateOptions` carry the optional effort. The agent loop prepares the post-`agent/request` config under the active turn signal before writing `request/header`, so defaults and dynamic changes are model-visible only after becoming durable facts. The prepared call retains the exact adapter registration across asynchronous exact-model resolution, durable header logging, and dispatch; direct `LlmService.stream()` calls likewise capture their final registration before awaiting resolution. A route with no registered adapter retains its proposed config so an `llm/stream` middleware can own and short-circuit it; terminal dispatch still rejects an unhandled route. A resumed loop retains the logged effort only when its initial provider/model route is unchanged; a route change discards the previous model's opaque id.
The native DeepSeek adapter advertises `high` and `max`, defaults to configured effort or `high`, and exposes no effort capability while thinking is disabled. The pi-ai adapter derives each exact model's list from `getSupportedThinkingLevels()`, excludes `off`, preserves an absent profile default as a provider default, and leaves provider wire-value mapping inside pi-ai.
The native DeepSeek adapter advertises `high` and `max`, defaults to configured effort or `high`, and exposes no effort capability while thinking is disabled. The pi-ai adapter publishes each exact model's `getSupportedThinkingLevels()` result unchanged, including `off`, preserves an absent profile default as a provider default, and leaves provider wire-value mapping inside pi-ai. Its common stream options represent `off` by omitting `reasoning`, as required by pi-ai's own API.
## Alternatives considered
@@ -24,10 +24,10 @@ The native DeepSeek adapter advertises `high` and `max`, defaults to configured
**Clamp unsupported levels.** Rejected because a silent substitution makes the user's selected control differ from the logged request intent and hides stale deployment configuration.
**Include `off` as an effort.** Rejected because disabling reasoning is a mode capability with different request and output semantics, not a reasoning-strength level.
**Normalize every adapter to a core-owned level list or remove `off`.** Rejected because the selectable vocabulary belongs to the exact model capability. A client can render an adapter's `off` option without requiring every adapter to expose it.
## Consequences
Clients can query one exact route and render the adapter's order and names without knowing a global enum. Adapter configuration remains the deployment-default owner, while `agent/request` can replace the effective effort on each step. Invalid metadata fails with `INVALID_MODEL_REASONING`, and unsupported explicit or configured values fail with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
Clients can query one exact route once and render its identity, context capacity, and adapter-owned reasoning choices without knowing a global enum. Adapter configuration remains the deployment-default owner, while `agent/request` can replace the effective effort on each step. Invalid exact identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`; unsupported explicit or configured values fail with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
The capability query is asynchronous and exact-model resolution may fail for adapters backed by authoritative catalogs. Its optional signal is the caller's cancellation boundary; an asynchronous adapter must settle promptly after abort so loop disposal can reach quiescence. Keyless service, adapter, loop, session, and request-header tests pin validation, defaulting, dynamic changes, logging, resume behavior, HMR registration ownership, and cancellation; runnable snapshots pin the resolved effort in real assembled request headers, while key-gated adapter tests exercise provider serialization.
The aggregate exact-model query is asynchronous and may fail for adapters backed by authoritative catalogs. Its optional signal is the caller's cancellation boundary; an asynchronous adapter must settle promptly after abort so loop disposal can reach quiescence. Keyless service, adapter, loop, session, and request-header tests pin validation, defaulting, dynamic changes, logging, resume behavior, HMR registration ownership, and cancellation; runnable snapshots pin the resolved effort in real assembled request headers, while key-gated adapter tests exercise provider serialization.
@@ -10,11 +10,11 @@ Status: implemented
## 决策
`dsh-llm` 使用不透明的品牌类型 `ReasoningEffortId` 表示推理强度。适配器 `resolveModelReasoning(provider, model)` 返回非空有序 ID 列表及其展示元数据,并可指定一个由配置确定的默认值。核心会校验元数据,要求显式指定或配置指定的推理强度与列表中的某个 ID 完全一致,且绝不自动调整或为值提供别名。
`dsh-llm` 使用不透明的品牌类型 `ReasoningEffortId` 表示推理强度。适配器持有的单次 `resolveModel(provider, model, signal?)` 查询返回 `LlmResolvedModelInfo`,其中包含确切模型身份以及可选的上下文和推理元数据。`LlmService.resolveModelInfo()` 会校验该聚合结果并返回分离值。`reasoning.efforts` 存在时,是包含展示元数据的非空有序 ID 列表,并可指定一个由配置确定的默认值。核心要求显式指定或配置指定的推理强度与列表中的某个 ID 完全一致,且绝不自动调整或为值提供别名。
`LlmCallConfig``GenerateOptions` 携带可选的推理强度。agent loop(智能体循环)在活跃轮次信号的控制下准备 `agent/request` 处理完成后的配置,再写入 `request/header`,因此默认值和动态变更只有成为持久化事实后才对模型可见。准备完成的调用在异步能力解析、请求头持久记录和分派全程保留同一项确切的适配器注册;直接调用 `LlmService.stream()` 时,也会在等待解析前捕获最终的适配器注册。没有已注册适配器的路由会保留原定配置,使 `llm/stream` 中间件可以接管并短路该请求;若仍未得到处理,最终分发会拒绝该路由。恢复后的主循环仅在初始提供方/模型路由未变时保留日志中记录的推理强度;如果路由发生变化,则丢弃上一模型的不透明 ID。
`LlmCallConfig``GenerateOptions` 携带可选的推理强度。agent loop(智能体循环)在活跃轮次信号的控制下准备 `agent/request` 处理完成后的配置,再写入 `request/header`,因此默认值和动态变更只有成为持久化事实后才对模型可见。准备完成的调用在异步确切模型解析、请求头持久记录和分派全程保留同一项确切的适配器注册;直接调用 `LlmService.stream()` 时,也会在等待解析前捕获最终的适配器注册。没有已注册适配器的路由会保留原定配置,使 `llm/stream` 中间件可以接管并短路该请求;若仍未得到处理,最终分发会拒绝该路由。恢复后的主循环仅在初始提供方/模型路由未变时保留日志中记录的推理强度;如果路由发生变化,则丢弃上一模型的不透明 ID。
原生 DeepSeek 适配器声明 `high``max`,默认使用配置指定的推理强度,若未配置则使用 `high`;禁用思考时不暴露推理强度能力。pi-ai 适配器通过 `getSupportedThinkingLevels()` 按具体模型推导等级列表,排除 `off`,在 profile 未指定默认值时保留提供方默认行为,并将提供方协议值的映射留在 pi-ai 内部。
原生 DeepSeek 适配器声明 `high``max`,默认使用配置指定的推理强度,若未配置则使用 `high`;禁用思考时不暴露推理强度能力。pi-ai 适配器原样发布每个确切模型的 `getSupportedThinkingLevels()` 结果,其中包括 `off`profile 未指定默认值时保留提供方默认行为,并将提供方协议值的映射留在 pi-ai 内部。按照 pi-ai 自身 API 的要求,其通用流选项通过省略 `reasoning` 来表示 `off`
## 备选方案
@@ -24,10 +24,10 @@ Status: implemented
**自动调整不支持的等级。** 不予采纳:静默替换会导致用户选定的控制项与日志记录的请求意图不一致,还会掩盖陈旧的部署配置。
**将 `off` 列为推理强度。** 不予采纳:禁用推理属于具有不同请求和输出语义的模式能力,而不是推理强度等级
**将每个适配器规范化为核心持有的等级列表,或移除 `off`。** 不予采纳:可选值集合属于确切模型的能力。客户端可以渲染某个适配器的 `off` 选项,而无需要求所有适配器都暴露该选项
## 影响
客户端可以查询一确切路由,并按适配器给出的顺序和名称渲染等级,而无需了解全局枚举。适配器配置仍负责提供部署默认值,`agent/request` 则可以在每个步骤替换实际生效的推理强度。元数据无效时抛出 `INVALID_MODEL_REASONING`;显式指定或配置指定的值不受支持时,会在提供方 I/O 前抛出 `UNSUPPORTED_REASONING_EFFORT`
客户端只需查询一确切路由,即可渲染其身份、上下文容量和由适配器持有的推理选项,而无需了解全局枚举。适配器配置仍负责提供部署默认值,`agent/request` 则可以在每个步骤替换实际生效的推理强度。确切身份、上下文或推理元数据无效时,分别抛出 `INVALID_MODEL_INFO``INVALID_MODEL_CONTEXT` `INVALID_MODEL_REASONING`;显式指定或配置指定的值不受支持时,会在提供方 I/O 前抛出 `UNSUPPORTED_REASONING_EFFORT`
能力查询采用异步方式对于由权威目录支持的适配器,确切模型解析可能失败。可选信号构成调用方的取消边界;异步适配器必须在信号中止后迅速完成结算,使主循环的资源释放达到完全停稳。无密钥的服务、适配器、主循环、会话和请求头测试为校验、默认值解析、动态变更、日志记录、恢复行为、HMR(热模块替换)期间的注册所有权和取消提供回归保障;可运行快照锁定实际组装请求头中的已解析推理强度,仅在有密钥时运行的适配器测试则覆盖提供方序列化。
确切模型元数据的聚合查询采用异步方式,并且对于由权威目录支持的适配器可能失败。可选信号构成调用方的取消边界;异步适配器必须在信号中止后迅速完成结算,使主循环的资源释放达到完全停稳。无密钥的服务、适配器、主循环、会话和请求头测试为校验、默认值解析、动态变更、日志记录、恢复行为、HMR(热模块替换)期间的注册所有权和取消提供回归保障;可运行快照锁定实际组装请求头中的已解析推理强度,仅在有密钥时运行的适配器测试则覆盖提供方序列化。
+3 -3
View File
@@ -623,7 +623,7 @@ export interface PiAiProviderProfile {
/** Provider request headers; Harness attribution wins reserved names. */
headers?: Record<string, string>
/** Provider-neutral pi-ai reasoning level. */
reasoning?: ThinkingLevel
reasoning?: ModelThinkingLevel
/** Token budgets used by reasoning providers that support them. */
thinkingBudgets?: ThinkingBudgets
/** Prompt-cache retention preference. */
@@ -639,7 +639,7 @@ export interface PiAiProviderProfile {
}
```
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `ThinkingLevel` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
Depends on: `CacheRetention` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
Source: [`packages/llm/llm-pi-ai/src/config.ts:48`](../packages/llm/llm-pi-ai/src/config.ts)
@@ -687,7 +687,7 @@ export interface ReplayModelConfig {
}
```
Source: [`packages/support/llm-replay/src/index.ts:392`](../packages/support/llm-replay/src/index.ts)
Source: [`packages/support/llm-replay/src/index.ts:400`](../packages/support/llm-replay/src/index.ts)
## `@deepseek-ai/dsh-llm-retry`
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
adding-an-llm-adapter.md: 76bcfdd6638ef8b40977a5e1fe4f678648c043cd
adding-an-llm-adapter.zh.md: 13ae388bb0c6f29ab6269f364b849cb1694e2ef2
adding-an-llm-adapter.md: 255dce40e5c10a4c94e34b358ad71f0283c1227f
adding-an-llm-adapter.zh.md: 67d85f96326474f6acb1b604b2ed31eba18eab4f
+1 -1
View File
@@ -32,7 +32,7 @@ Registration is effect-based (HMR-safe); one adapter per provider route — dupl
- A `GenerateOptions` field your provider cannot honor (e.g. a `stop` list on a provider without stop sequences): throw `LlmError(..., 'UNSUPPORTED')` rather than silently dropping it.
- If the provider requires response ids, signatures, or other native metadata on follow-up calls, emit the minimal lossless-JSON projection as `finish.replayState`. Validate it when rebuilding history. `LlmService` passes it only when the historical provider route and target provider route are currently owned by the exact same adapter instance; your adapter decides whether same-model, cross-model, or cross-provider restoration is legal. Never infer native replay from provider/model names alone when state is absent.
Provider-specific thinking-mode toggles remain in the adapter's Config. Selectable reasoning strength uses the provider-neutral capability seam: return ordered opaque ids from `resolveModelReasoning()`, declare a configured `defaultEffort` only when one exists, honor the resolver's optional `AbortSignal`, and map `GenerateOptions.reasoningEffort` to the provider wire value. Do not expose provider wire spellings, clamp unsupported values, or include an `off` mode as an effort.
Provider-specific thinking-mode toggles remain in the adapter's Config. Exact model metadata uses one provider-neutral capability seam: implement `resolveModel()` with provider/model identity and optional `context` and `reasoning` fields, declare a configured `defaultEffort` only when one exists, and honor the resolver's optional `AbortSignal`. Reasoning efforts are ordered opaque ids mapped to provider requests by the adapter. Preserve the adapter's authoritative selectable list—including `off` when its upstream capability API returns it—without exposing final wire spellings or clamping unsupported values.
## Structure that worked
+1 -1
View File
@@ -32,7 +32,7 @@ export function apply(ctx: Context, config: Config) {
- 如果 `GenerateOptions` 中某个字段你的提供方无法支持(例如提供方不支持 stop sequences 时收到 `stop` 列表):抛出 `LlmError(..., 'UNSUPPORTED')`,而非静默丢弃。
- 如果提供方在后续调用中需要响应 ID、签名或其他原生元数据,请将其最小无损 JSON 投影作为 `finish.replayState` 发出。重建历史时验证该状态。只有历史提供方路由和目标提供方路由当前由完全相同的适配器实例拥有时,`LlmService` 才会传递该状态;由适配器决定同模型、跨模型或跨提供方恢复是否合法。状态缺失时,切勿仅根据提供方/模型名称推断原生回放。
提供方特有的 thinking 模式开关仍放在适配器的 Config 中。可选的推理强度使用提供方无关的能力 seam`resolveModelReasoning()` 返回有序的不透明 ID;仅当存在配置指定的默认值时才声明 `defaultEffort`;响应传给解析器的可选 `AbortSignal`;并将 `GenerateOptions.reasoningEffort` 映射为提供方协议值。不得暴露提供方协议值的具体拼写自动调整不支持的值,也不得把 `off` 模式列为推理强度
提供方特有的 thinking 模式开关仍放在适配器的 Config 中。确切模型元数据使用一处提供方无关的能力 seam实现 `resolveModel()`,返回提供方/模型身份以及可选的 `context` 和 `reasoning` 字段;仅当存在配置指定的默认值时才声明 `defaultEffort`;响应传给解析器的可选 `AbortSignal`。推理强度是由适配器映射到提供方请求的有序不透明 ID。请保留适配器给出的权威可选列表,包括其上游能力 API 返回的 `off`,但不要暴露最终协议值的具体拼写,也不要自动调整不支持的值。
## 经验证有效的结构
+1 -1
View File
@@ -616,7 +616,7 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmService](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
Source: [`packages/llm/llm/src/index.ts:54`](../../packages/llm/llm/src/index.ts)
Source: [`packages/llm/llm/src/index.ts:53`](../../packages/llm/llm/src/index.ts)
## `session/*`
+8 -18
View File
@@ -730,25 +730,15 @@ listProviders(): LlmProviderInfo[]
async listModels(provider: string): Promise<LlmModelInfo[]>
/**
* Resolve context capacity from the adapter that owns one exact route.
* This query is independent of the advisory model catalog: an unlisted model
* may return metadata, while `undefined` never rejects later routing.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @returns detached context metadata, or `undefined` when the adapter has none.
*/
async resolveModelContext( provider: string, model: string, ): Promise<LlmModelContext | undefined>
/**
* Resolve selectable reasoning efforts from the adapter that owns one exact
* route. Metadata is validated and detached; an absent result means an
* effort selector is unsupported for that model.
* Resolve and validate all metadata from the adapter that owns one exact
* route. The result is detached from adapter-owned objects; catalog
* membership remains advisory and does not control request routing.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
* @returns detached reasoning metadata, or `undefined` when unsupported.
* @returns exact model identity plus available context and reasoning metadata.
*/
async resolveModelReasoning( provider: string, model: string, signal?: AbortSignal, ): Promise<LlmModelReasoningInfo | undefined>
async resolveModelInfo( provider: string, model: string, signal?: AbortSignal, ): Promise<LlmResolvedModelInfo>
/**
* Validate a conversation call config against its exact model capability and
@@ -777,7 +767,7 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<Prepared
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
* `options.provider`. Replay state is retained only when the same adapter
* instance owns its historical provider and the target provider. Final
* adapter selection remains fixed through asynchronous reasoning resolution
* adapter selection remains fixed through asynchronous exact-model resolution
* and dispatch. Selection, dispatch, and iteration failures retain their
* original Error identity and are tagged in a call-local scope for narrow
* agent-loop request recovery; middleware and nested-call failures remain
@@ -788,9 +778,9 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<Prepared
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
```
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelContext](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmModelReasoningInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
Types: [GenerateOptions](../core-data-structures/core.md) · [LlmAdapter](../core-data-structures/llm-streaming.md) · [LlmCallConfig](../core-data-structures/core.md) · [LlmModelInfo](../core-data-structures/core.md) · [LlmProviderInfo](../core-data-structures/core.md) · [LlmResolvedModelInfo](../core-data-structures/core.md) · [PreparedLlmCall](../core-data-structures/llm-streaming.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
Source: [`packages/llm/llm/src/index.ts:192`](../../packages/llm/llm/src/index.ts)
Source: [`packages/llm/llm/src/index.ts:177`](../../packages/llm/llm/src/index.ts)
## `ctx.permission` — `PermissionService`
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
core.md: d60268a163580558b09ae0af7b15b78232436a4b
core.zh.md: b81c6a90ab7f74580b0f6599bd78868771f3e339
core.md: 86aea325f5401d728a9b4aa147d78db9163a32c1
core.zh.md: d09fb48419c14959eefb5a1df1c593b36c188cf5
+11 -1
View File
@@ -199,7 +199,7 @@ interface LlmModelInfo {
}
```
Correctness-sensitive model capacity is queried separately from the advisory catalog and is owned by the adapter serving the exact route.
Correctness-sensitive metadata is resolved separately from the advisory catalog and is owned by the adapter serving the exact route. Context capacity and reasoning choices share one exact-model result so consumers do not repeat authoritative model resolution.
```ts type-equiv
/** Provider-owned context capacity for one exact provider/model route. */
@@ -241,6 +241,16 @@ interface LlmModelReasoningInfo {
}
```
```ts type-equiv
/** Exact-route model metadata resolved by its owning adapter. */
interface LlmResolvedModelInfo extends LlmModelInfo {
/** Provider-owned context capacity when known. */
context?: LlmModelContext
/** Adapter-owned selectable reasoning levels when exposed. */
reasoning?: LlmModelReasoningInfo
}
```
```ts type-equiv
/** A single model request, fully assembled. */
interface GenerateOptions {
+11 -1
View File
@@ -205,7 +205,7 @@ interface LlmModelInfo {
}
```
对正确性敏感的模型容量与参考目录分开查询,并归服务该确切路由的适配器所有。
对正确性敏感的元数据与参考目录分开解析,并归服务该确切路由的适配器所有。上下文容量和推理选项共用同一个确切模型结果,消费方因而无需重复执行权威模型解析。
```ts type-equiv
/** Provider-owned context capacity for one exact provider/model route. */
@@ -247,6 +247,16 @@ interface LlmModelReasoningInfo {
}
```
```ts type-equiv
/** Exact-route model metadata resolved by its owning adapter. */
interface LlmResolvedModelInfo extends LlmModelInfo {
/** Provider-owned context capacity when known. */
context?: LlmModelContext
/** Adapter-owned selectable reasoning levels when exposed. */
reasoning?: LlmModelReasoningInfo
}
```
```ts type-equiv
/** A single model request, fully assembled. */
interface GenerateOptions {
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
llm-streaming.md: 5155e680bfcc253d06742c7d3adcddc633a01c52
llm-streaming.zh.md: 9b928253065da99c74f749b905375d790fab46c9
llm-streaming.md: ffe520eca12af0fa2d483b9f231c943107d74649
llm-streaming.zh.md: d4d41040ceaa7e2b2ca08c47ae83fefbaba3f7d4
+12 -23
View File
@@ -157,7 +157,7 @@ declare class BlockAssembler {
## The seam
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerInfo()` and asynchronous `listModels()` methods feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. The separate `resolveModelContext()` query exposes correctness-sensitive capacity, while `resolveModelReasoning()` exposes ordered model-owned effort ids and an optional deployment default; absence from either query means unavailable metadata or capability, not invalid catalog membership. A reasoning resolver receives optional cancellation and must settle promptly after abort. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across capability resolution, durable header logging, and dispatch. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
`LlmAdapter` is the provider seam: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerInfo()` and asynchronous `listModels()` methods feed `LlmService.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or capability, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmService.resolveModelInfo()` validates and detaches the aggregate. The service validates and materializes reasoning through `resolveCallConfig()` at the final adapter boundary, so direct calls cannot bypass unsupported-effort rejection; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § Content blocks and streaming](../architecture.md#content-blocks-and-streaming-dsh-llm).
```ts type-equiv
/** One model call whose config and adapter registration were resolved together. */
@@ -198,30 +198,19 @@ declare abstract class LlmAdapter {
*/
listModels(_provider: string): Promise<readonly LlmModelInfo[]>;
/**
* Resolve context capacity for one model accepted by this adapter. Absence
* means the adapter does not know the capacity, not that routing is invalid.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @returns provider-owned context metadata, or `undefined` when unavailable.
* Resolve all metadata available for one exact model. This query is
* independent of the advisory catalog and does not validate request routing.
* @param provider - one provider route owned by this adapter.
* @param model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; asynchronous
* implementations must settle promptly after it aborts.
* @returns provider/model identity plus any context and reasoning metadata.
*/
resolveModelContext(
_provider: string,
_model: string,
): Promise<LlmModelContext | undefined>;
/**
* Resolve selectable reasoning efforts for one exact model. Absence means
* the model has no selectable reasoning-effort capability.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; implementations
* must settle promptly after it aborts.
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
*/
resolveModelReasoning(
_provider: string,
_model: string,
resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined>;
): Promise<LlmResolvedModelInfo>;
/**
* Stream one model call as raw chunks. The only required method.
* @param options - the fully-assembled request; implementations must honor `options.signal`.
+12 -23
View File
@@ -157,7 +157,7 @@ declare class BlockAssembler {
## seam
`LlmAdapter` 是提供方 seam:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerInfo()` 与异步 `listModels()` 方法为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单独的 `resolveModelContext()` 查询暴露对正确性敏感的容量信息,`resolveModelReasoning()` 则暴露由模型持有的有序推理强度 ID 和可选的部署默认值;任一查询返回缺失表示元数据或能力不可用,而不表示目录成员关系无效。推理能力解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值,因此直接调用也无法绕过对不支持推理强度的拒绝;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使能力解析、请求头持久记录和分派全程使用同一项注册。适配器查找发生在 `llm/stream` waterfall(瀑布式事件)的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface(`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
`LlmAdapter` 是提供方 seam:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerInfo()` 与异步 `listModels()` 方法为 `LlmService.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、由模型持有的有序推理强度 ID 和部署默认值;字段缺失表示元数据或能力不可用,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmService.resolveModelInfo()` 会校验聚合结果并返回分离值。服务通过最终适配器边界的 `resolveCallConfig()` 校验推理强度并填入默认值,因此直接调用也无法绕过对不支持推理强度的拒绝;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册。适配器查找发生在 `llm/stream` waterfall(瀑布式事件)的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。消费方 surface(`ctx.llm.stream()`)与 `llm/stream` waterfall 见 [architecture.md § 内容块与流式传输](../architecture.md#content-blocks-and-streaming-dsh-llm)。
```ts type-equiv
/** One model call whose config and adapter registration were resolved together. */
@@ -198,30 +198,19 @@ declare abstract class LlmAdapter {
*/
listModels(_provider: string): Promise<readonly LlmModelInfo[]>;
/**
* Resolve context capacity for one model accepted by this adapter. Absence
* means the adapter does not know the capacity, not that routing is invalid.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @returns provider-owned context metadata, or `undefined` when unavailable.
* Resolve all metadata available for one exact model. This query is
* independent of the advisory catalog and does not validate request routing.
* @param provider - one provider route owned by this adapter.
* @param model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; asynchronous
* implementations must settle promptly after it aborts.
* @returns provider/model identity plus any context and reasoning metadata.
*/
resolveModelContext(
_provider: string,
_model: string,
): Promise<LlmModelContext | undefined>;
/**
* Resolve selectable reasoning efforts for one exact model. Absence means
* the model has no selectable reasoning-effort capability.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; implementations
* must settle promptly after it aborts.
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
*/
resolveModelReasoning(
_provider: string,
_model: string,
resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined>;
): Promise<LlmResolvedModelInfo>;
/**
* Stream one model call as raw chunks. The only required method.
* @param options - the fully-assembled request; implementations must honor `options.signal`.
+1 -1
View File
@@ -33,7 +33,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac
| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:71`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) |
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:54`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
| `goal/changed` | `emit` | [`packages/goal/goal/src/types.ts:167`](../packages/goal/goal/src/types.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-session`](../packages/goal/goal-session) |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:54`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/support/llm-replay), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`session-title`](../packages/session-title/session-title) |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:53`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/support/llm-replay), [`session-checkpoint-policy`](../packages/session-persistence/session-checkpoint-policy), [`session-title`](../packages/session-title/session-title) |
| `session/created` | `emit` | [`packages/core/session/src/index.ts:79`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | `apiproxy`, [`compact`](../packages/compact/compact), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`jsonrpc`](../packages/ui/jsonrpc), [`llm-retry`](../packages/llm/llm-retry), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`user-approval`](../packages/ui/user-approval) |
| `session/disposed` | `emit` | [`packages/core/session/src/index.ts:89`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `apiproxy`, [`session-persistence`](../packages/session-persistence/session-persistence), [`session-title`](../packages/session-title/session-title) |
| `session/event` | `emit` | [`packages/core/session/src/index.ts:101`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`acp`](../packages/acp/acp), `apiproxy`, [`cli-demo`](../packages/examples/cli-demo), [`compact`](../packages/compact/compact), [`goal`](../packages/goal/goal), [`goal-session`](../packages/goal/goal-session), [`hook-protocol`](../packages/hooks/hook-protocol), [`jsonrpc`](../packages/ui/jsonrpc), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-title`](../packages/session-title/session-title), [`token-meter`](../packages/llm/token-meter), [`tui`](../packages/ui/tui), [`user-approval`](../packages/ui/user-approval), [`workspace`](../packages/workspace/workspace), [`workspace-context`](../packages/context/workspace-context) |
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
llm-adapter.md: 8b8201099e2b78385301daf14605be53436353c7
llm-adapter.zh.md: 735b7a4babf3d31652374e91cbce794cdd580725
llm-adapter.md: 3bf005647d3e7908f1d1553374266c1d6c12c2e6
llm-adapter.zh.md: b967d112ed6879b11486ab7aedf64653089d93de
+1 -1
View File
@@ -112,7 +112,7 @@ async function* exampleChunks(): AsyncIterable<StreamChunk> {
`stream()` receives the exported `GenerateOptions` type. It includes the model, adapter-owned reasoning-effort id, conversation history, system prompt, tool schemas, generation parameters, stop sequences, and abort signal; treat the TypeScript type exported by `@deepseek-ai/dsh-llm` as authoritative. Map supported fields to the provider API. If the provider cannot honor a field, throw `LlmError` with a stable code instead of silently dropping it.
Override `resolveModelReasoning(provider, model, signal?)` when an exact model exposes selectable reasoning strengths. Return ordered opaque ids and display names plus an optional configured default; do not promote provider names into a core enum. Honor the optional signal for asynchronous lookup so cancellation and disposal reach quiescence. The service validates the metadata and rejects unsupported explicit values before `stream()`. Returning `undefined` means that model has no selectable reasoning-effort capability.
Override `resolveModel(provider, model, signal?)` to return exact provider/model identity plus optional `context` and `reasoning` metadata in one lookup. Reasoning metadata contains ordered opaque ids and display names plus an optional configured default; preserve the adapter's authoritative selectable list, including `off` when its upstream capability API returns it, instead of promoting those values into a core enum. Honor the optional signal for asynchronous lookup so cancellation and disposal reach quiescence. The service validates the aggregate and rejects unsupported explicit efforts before `stream()`; omitting `reasoning` means that model has no selectable reasoning-effort capability.
## Register an adapter
+1 -1
View File
@@ -112,7 +112,7 @@ async function* exampleChunks(): AsyncIterable<StreamChunk> {
`stream()` 接收仓库导出的 `GenerateOptions`。它包含模型名、由适配器持有的推理强度 ID、对话历史、系统提示词、tool schema、生成参数、停止序列和中止信号;完整字段以 `@deepseek-ai/dsh-llm` 导出的 TypeScript 类型为准。适配器必须将支持的字段映射到具体 API;无法支持的字段应抛出带稳定 code 的 `LlmError`,不能静默丢弃。
当某个具体模型提供可选推理强度时,请覆写 `resolveModelReasoning(provider, model, signal?)`。返回有序的不透明 ID、展示名称,以及可选的配置默认值;不要将提供方使用的等级名称提升为核心枚举。异步查询必须响应这个可选信号,让取消和资源释放都能达到完全停稳。服务会校验元数据,并在调用 `stream()` 前拒绝显式指定但不受支持的值。返回 `undefined` 表示该模型没有可选的推理强度能力。
请覆写 `resolveModel(provider, model, signal?)`,在一次查询中返回确切的提供方/模型身份以及可选的 `context``reasoning` 元数据。推理元数据包含有序的不透明 ID、展示名称,以及可选的配置默认值;请保留适配器给出的权威可选列表,包括其上游能力 API 返回的 `off`,而不要将这些值提升为核心枚举。异步查询必须响应这个可选信号,让取消和资源释放都能达到完全停稳。服务会校验聚合结果,并在调用 `stream()` 前拒绝显式指定但不受支持的推理强度;省略 `reasoning` 表示该模型没有可选的推理强度能力。
## 注册适配器
+14 -9
View File
@@ -4,22 +4,27 @@ import {
LlmAdapter,
ReasoningEffortId,
type GenerateOptions,
type LlmModelReasoningInfo,
type LlmResolvedModelInfo,
type StreamChunk,
} from '@deepseek-ai/dsh-llm'
const HIGH = ReasoningEffortId('high')
const MAX = ReasoningEffortId('max')
const OFF = ReasoningEffortId('off')
/** Keyless headless-agent adapter: one real bash call followed by a final answer. */
class CliMockAdapter extends LlmAdapter {
override async resolveModelReasoning(): Promise<LlmModelReasoningInfo> {
override async resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
return {
efforts: [
{ id: HIGH, name: 'High' },
{ id: MAX, name: 'Max' },
],
defaultEffort: HIGH,
provider,
id: model,
name: model,
reasoning: {
efforts: [
{ id: OFF, name: 'Off' },
{ id: HIGH, name: 'High' },
],
defaultEffort: HIGH,
},
}
}
@@ -56,6 +61,6 @@ export function apply(ctx: Context): void {
ctx.llm.registerAdapter(['cli-mock'], new CliMockAdapter())
ctx.on('agent/request', async (_agent, _turn, step, _config, _signal, next) => {
const config = await next()
return step === 2 ? { ...config, reasoningEffort: MAX } : config
return step === 2 ? { ...config, reasoningEffort: OFF } : config
})
}
@@ -159,7 +159,7 @@ describe('headless stream-json snapshots', () => {
{
"model": "cli-mock",
"provider": "cli-mock",
"reasoningEffort": "max",
"reasoningEffort": "off",
},
]
`)
+20 -15
View File
@@ -1,9 +1,8 @@
import type { Context } from 'cordis'
import type {
GenerateOptions,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmResolvedModelInfo,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import { CallId, LlmAdapter, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
@@ -41,21 +40,27 @@ class ScriptedTuiAdapter extends LlmAdapter {
])
}
override resolveModelContext(_provider: string, _model: string): Promise<LlmModelContext> {
return Promise.resolve({ contextWindow: 128_000 })
}
override resolveModelReasoning(
_provider: string,
override resolveModel(
provider: string,
model: string,
): Promise<LlmModelReasoningInfo | undefined> {
if (model !== 'tui-scripted-model-pro') return Promise.resolve(undefined)
): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
provider,
id: model,
name: model === 'tui-scripted-model-pro' ? 'Scripted Pro' : 'Scripted Base',
context: { contextWindow: 128_000 },
...model !== 'tui-scripted-model-pro'
? {}
: {
reasoning: {
efforts: [
{ id: ReasoningEffortId('off'), name: 'Off' },
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
},
},
})
}
+1 -1
View File
@@ -272,7 +272,7 @@ export class BasicCompactService extends CompactService {
return this.compactRegion(range.start, range.end, agent, signal)
}
const context = await this.ctx.llm.resolveModelContext(target.provider, target.model)
const context = (await this.ctx.llm.resolveModelInfo(target.provider, target.model)).context
const targetKey = `${target.provider}/${target.model}`
if (context === undefined) {
throw new TargetPressureConfigError(
@@ -16,7 +16,7 @@ import type {
ContentBlock,
GenerateOptions,
LlmFailure,
LlmModelContext,
LlmResolvedModelInfo,
Message,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
@@ -33,8 +33,13 @@ class ContextAdapter extends LlmAdapter {
super()
}
override resolveModelContext(): Promise<LlmModelContext> {
return Promise.resolve({ contextWindow: this.contextWindow })
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
provider,
id: model,
name: model,
context: { contextWindow: this.contextWindow },
})
}
override async * stream(): AsyncIterable<StreamChunk> {
@@ -47,9 +52,14 @@ class RoutedContextAdapter extends LlmAdapter {
super()
}
override resolveModelContext(provider: string): Promise<LlmModelContext | undefined> {
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
const contextWindow = this.windows[provider]
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
return Promise.resolve({
provider,
id: model,
name: model,
...contextWindow === undefined ? {} : { context: { contextWindow } },
})
}
override async * stream(): AsyncIterable<StreamChunk> {
@@ -485,7 +495,11 @@ describe('pressure measurement and retention', () => {
void new LlmService(ctx)
void new TokenMeterService(ctx)
ctx.llm.registerAdapter(['unknown-context'], new ContextAdapter(1_000))
vi.spyOn(ctx.llm, 'resolveModelContext').mockResolvedValue(undefined)
vi.spyOn(ctx.llm, 'resolveModelInfo').mockImplementation((provider, model) => Promise.resolve({
provider,
id: model,
name: model,
}))
const compact = service(compactConfig, ctx)
const session = conversation(4)
session.append('request/header', {
@@ -1337,7 +1351,11 @@ describe('automatic listener and loader composition', () => {
const ctx = createContext()
const warnings: string[] = []
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
vi.spyOn(ctx.llm, 'resolveModelContext').mockResolvedValue(undefined)
vi.spyOn(ctx.llm, 'resolveModelInfo').mockImplementation((provider, model) => Promise.resolve({
provider,
id: model,
name: model,
}))
void new TestCompactService(ctx, {
thresholdRatio: 0.5,
retainTokens: 180,
@@ -2,7 +2,7 @@ import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
import { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
import { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
import type { Agent } from '@deepseek-ai/dsh-agent'
@@ -41,8 +41,13 @@ class StepwiseToolAdapter extends LlmAdapter {
super()
}
override resolveModelContext(): Promise<{ contextWindow: number }> {
return Promise.resolve({ contextWindow: 400 })
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
provider,
id: model,
name: model,
context: { contextWindow: 400 },
})
}
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
@@ -76,8 +81,13 @@ class OverflowRecoveryAdapter extends LlmAdapter {
super()
}
override resolveModelContext(): Promise<{ contextWindow: number }> {
return Promise.resolve({ contextWindow: 128 })
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
provider,
id: model,
name: model,
context: { contextWindow: 128 },
})
}
override async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
@@ -381,12 +381,8 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
jsDoc: '/**\n * Discover models advertised by one registered provider. Catalog membership\n * is advisory and never changes routing or request validation.\n * @param provider - registered provider route to inspect.\n * @returns detached model metadata in adapter-preferred order.\n */',
},
{
signature: 'async resolveModelContext( provider: string, model: string, ): Promise<LlmModelContext | undefined>',
jsDoc: '/**\n * Resolve context capacity from the adapter that owns one exact route.\n * This query is independent of the advisory model catalog: an unlisted model\n * may return metadata, while `undefined` never rejects later routing.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @returns detached context metadata, or `undefined` when the adapter has none.\n */',
},
{
signature: 'async resolveModelReasoning( provider: string, model: string, signal?: AbortSignal, ): Promise<LlmModelReasoningInfo | undefined>',
jsDoc: '/**\n * Resolve selectable reasoning efforts from the adapter that owns one exact\n * route. Metadata is validated and detached; an absent result means an\n * effort selector is unsupported for that model.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @param signal - optional cancellation for adapter-owned asynchronous lookup.\n * @returns detached reasoning metadata, or `undefined` when unsupported.\n */',
signature: 'async resolveModelInfo( provider: string, model: string, signal?: AbortSignal, ): Promise<LlmResolvedModelInfo>',
jsDoc: '/**\n * Resolve and validate all metadata from the adapter that owns one exact\n * route. The result is detached from adapter-owned objects; catalog\n * membership remains advisory and does not control request routing.\n * @param provider - registered provider route to inspect.\n * @param model - exact model id passed to the adapter.\n * @param signal - optional cancellation for adapter-owned asynchronous lookup.\n * @returns exact model identity plus available context and reasoning metadata.\n */',
},
{
signature: 'async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>',
@@ -398,7 +394,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
},
{
signature: 'stream(options: GenerateOptions): AsyncIterable<StreamChunk>',
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection remains fixed through asynchronous reasoning resolution\n * and dispatch. Selection, dispatch, and iteration failures retain their\n * original Error identity and are tagged in a call-local scope for narrow\n * agent-loop request recovery; middleware and nested-call failures remain\n * untagged for the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
jsDoc: '/**\n * Stream one model call as raw chunks (token-level deltas). Throws\n * `LlmError` with code `NO_ADAPTER` if no adapter is registered for\n * `options.provider`. Replay state is retained only when the same adapter\n * instance owns its historical provider and the target provider. Final\n * adapter selection remains fixed through asynchronous exact-model resolution\n * and dispatch. Selection, dispatch, and iteration failures retain their\n * original Error identity and are tagged in a call-local scope for narrow\n * agent-loop request recovery; middleware and nested-call failures remain\n * untagged for the outer call.\n * @param options - the full request; `options.provider` selects the adapter.\n * @returns the chunk stream, possibly wrapped by `llm/stream` listeners.\n */',
},
],
},
@@ -1706,7 +1702,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'LlmAdapter',
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModelContext(_provider: string, _model: string): Promise<LlmModelContext | undefined>;\n resolveModelReasoning(_provider: string, _model: string, _signal?: AbortSignal): Promise<LlmModelReasoningInfo | undefined>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
},
{
name: 'LlmCallConfig',
@@ -1736,6 +1732,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'LlmReasoningEffortInfo',
declaration: 'export interface LlmReasoningEffortInfo {\n id: ReasoningEffortId;\n name: string;\n description?: string;\n}',
},
{
name: 'LlmResolvedModelInfo',
declaration: 'export interface LlmResolvedModelInfo extends LlmModelInfo {\n context?: LlmModelContext;\n reasoning?: LlmModelReasoningInfo;\n}',
},
{
name: 'Message',
declaration: 'export interface Message {\n role: \'system\' | \'user\' | \'assistant\';\n content: ContentBlock[];\n provenance?: AssistantProvenance;\n}',
+11 -6
View File
@@ -1,4 +1,4 @@
import type { GenerateOptions, LlmModelReasoningInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelReasoningInfo, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
/** Helpers to write scripted responses tersely. */
@@ -71,11 +71,16 @@ export class MockAdapter extends LlmAdapter {
super()
}
override resolveModelReasoning(
_provider: string,
_model: string,
): Promise<LlmModelReasoningInfo | undefined> {
return Promise.resolve(this.reasoning)
override resolveModel(
provider: string,
model: string,
): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
provider,
id: model,
name: model,
...this.reasoning === undefined ? {} : { reasoning: this.reasoning },
})
}
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
@@ -8,7 +8,7 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import LlmService, { LlmError, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelReasoningInfo } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelReasoningInfo, LlmResolvedModelInfo } from '@deepseek-ai/dsh-llm'
import SessionStore, { Session, SessionId, foldRequestHeader } from '@deepseek-ai/dsh-session'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry, { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
@@ -152,7 +152,7 @@ describe('request stability across the loop', () => {
expect(resumedHeaders.at(-1)?.data.reason).toBe('resume')
})
it('keeps reasoning resolution, request logging, and dispatch on one adapter registration', async () => {
it('keeps exact-model resolution, request logging, and dispatch on one adapter registration', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(SessionStore)
@@ -163,13 +163,18 @@ describe('request stability across the loop', () => {
const started = Promise.withResolvers<undefined>()
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
const first = new class extends MockAdapter {
override resolveModelReasoning(
_provider: string,
_model: string,
override async resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): typeof reasoning.promise {
): Promise<LlmResolvedModelInfo> {
started.resolve(undefined)
return reasoning.promise
return {
provider,
id: model,
name: model,
reasoning: await reasoning.promise,
}
}
}([textResponse('first')])
const second = new MockAdapter([textResponse('second')], {
@@ -200,7 +205,7 @@ describe('request stability across the loop', () => {
it('aborts a blocked reasoning lookup before quiescent disposal completes', async () => {
const started = Promise.withResolvers<AbortSignal>()
const adapter = new class extends MockAdapter {
override resolveModelReasoning(
override resolveModel(
_provider: string,
_model: string,
signal?: AbortSignal,
@@ -235,13 +240,13 @@ describe('request stability across the loop', () => {
})
it.each(['plain error', 'LLM error'] as const)(
'does not swallow a %s from reasoning resolution',
'does not swallow a %s from exact-model resolution',
async (kind) => {
const failure = kind === 'plain error'
? new Error('reasoning metadata failed')
: new LlmError('unsupported effort', 'UNSUPPORTED_REASONING_EFFORT')
const adapter = new class extends MockAdapter {
override resolveModelReasoning(): Promise<never> {
override resolveModel(): Promise<never> {
return Promise.reject(failure)
}
}([])
+2 -2
View File
@@ -28,9 +28,9 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for UI selectors and deployment introspection, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelContext('deepseek', model)` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. When neither value exists it returns `undefined` without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. When neither value exists, `context` is absent without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
`ctx.llm.resolveModelReasoning('deepseek', model)` returns the ordered `high` and `max` efforts for every pass-through model while thinking is enabled. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header` and serialized as the official top-level `reasoning_effort` field. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O.
The same exact-model result exposes ordered `high` and `max` efforts under `reasoning` for every pass-through model while thinking is enabled. `reasoningEffort` selects the deployment default and falls back to `high` when omitted. `agent/request` can replace it on each conversation step; the resolved value is logged in `request/header` and serialized as the official top-level `reasoning_effort` field. An unsupported value fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O.
`thinking: disabled` removes the reasoning capability and omits `reasoning_effort`; combining it with a configured default fails plugin loading, and a per-request effort fails as unsupported. A request with `GenerateOptions.purpose: 'session-title'` also forces thinking disabled and omits the already-resolved effort, reserving its bounded output for visible title text without changing conversation or compaction defaults.
+31 -25
View File
@@ -8,10 +8,9 @@
import { attributionHeaders, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
import type {
GenerateOptions,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
LlmResolvedModelInfo,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import { idleWatchdog, MAX_TIMER_DELAY_MS, timeoutOf } from '@deepseek-ai/dsh-timeout'
@@ -59,6 +58,15 @@ const REASONING_EFFORTS = [
{ id: MAX_REASONING_EFFORT, name: 'Max' },
] as const
function modelInfo(provider: string, model: DeepSeekCatalogModel): LlmModelInfo {
return {
provider,
id: model.id,
name: model.name ?? model.id,
...model.description === undefined ? {} : { description: model.description },
}
}
function providerRetryAfterMs(value: string | null): number | undefined {
if (value === null) return undefined
if (/^\d+$/.test(value)) {
@@ -127,34 +135,32 @@ export class DeepSeekAdapter extends LlmAdapter {
}
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
return Promise.resolve((this.options.models ?? []).map(model => ({
provider,
id: model.id,
name: model.name ?? model.id,
...model.description === undefined ? {} : { description: model.description },
})))
return Promise.resolve((this.options.models ?? []).map(model => modelInfo(provider, model)))
}
override resolveModelContext(
_provider: string,
override resolveModel(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const contextWindow = this.options.models?.find(entry => entry.id === model)?.contextWindow
?? this.options.defaultContextWindow
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
}
override resolveModelReasoning(
_provider: string,
_model: string,
_signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined> {
if (this.options.defaults?.thinking === 'disabled') return Promise.resolve(undefined)
): Promise<LlmResolvedModelInfo> {
const configured = this.options.models?.find(entry => entry.id === model)
const contextWindow = configured?.contextWindow
?? this.options.defaultContextWindow
return Promise.resolve({
efforts: REASONING_EFFORTS,
defaultEffort: this.options.defaults?.reasoningEffort === 'max'
? MAX_REASONING_EFFORT
: HIGH_REASONING_EFFORT,
...configured === undefined
? { provider, id: model, name: model }
: modelInfo(provider, configured),
...contextWindow === undefined ? {} : { context: { contextWindow } },
...this.options.defaults?.thinking === 'disabled'
? {}
: {
reasoning: {
efforts: REASONING_EFFORTS,
defaultEffort: this.options.defaults?.reasoningEffort === 'max'
? MAX_REASONING_EFFORT
: HIGH_REASONING_EFFORT,
},
},
})
}
+30 -21
View File
@@ -213,8 +213,8 @@ describe('DeepSeekAdapter against a mock server', () => {
thinking: { type: 'disabled' },
})
expect(server.requests[0]).not.toHaveProperty('reasoning_effort')
await expect(ctx.llm.resolveModelReasoning('deepseek', 'deepseek-v4-flash'))
.resolves.toBeUndefined()
await expect(ctx.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
.resolves.not.toHaveProperty('reasoning')
})
it('rejects a per-request effort before I/O when thinking is disabled', async () => {
@@ -573,15 +573,19 @@ describe('plugin registration and config', () => {
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'deepseek-v4-flash' },
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'deepseek-v4-pro' },
])
await expect(ctx.llm.resolveModelContext('deepseek', 'deepseek-v4-flash'))
.resolves.toEqual({ contextWindow: 128_000 })
await expect(ctx.llm.resolveModelReasoning('deepseek', 'deepseek-v4-flash'))
.resolves.toEqual({
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
await expect(ctx.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
.resolves.toMatchObject({
provider: 'deepseek',
id: 'deepseek-v4-flash',
name: 'deepseek-v4-flash',
context: { contextWindow: 128_000 },
reasoning: {
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
},
})
})
@@ -635,10 +639,15 @@ describe('plugin registration and config', () => {
{ provider: 'deepseek', id: 'private-fast', name: 'private-fast' },
{ provider: 'deepseek', id: 'private-reasoner', name: 'Private Reasoner', description: 'Higher reasoning budget' },
])
await expect(ctx.llm.resolveModelContext('deepseek', 'private-fast'))
.resolves.toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelContext('deepseek', 'arbitrary-unlisted'))
.resolves.toBeUndefined()
await expect(ctx.llm.resolveModelInfo('deepseek', 'private-fast'))
.resolves.toMatchObject({ context: { contextWindow: 32_000 } })
await expect(ctx.llm.resolveModelInfo('deepseek', 'private-reasoner'))
.resolves.toMatchObject({
name: 'Private Reasoner',
description: 'Higher reasoning budget',
})
await expect(ctx.llm.resolveModelInfo('deepseek', 'arbitrary-unlisted'))
.resolves.not.toHaveProperty('context')
})
it('uses exact model capacity before the adapter-wide default', async () => {
@@ -654,12 +663,12 @@ describe('plugin registration and config', () => {
],
})
await expect(ctx.llm.resolveModelContext('deepseek', 'inherits-default'))
.resolves.toEqual({ contextWindow: 256_000 })
await expect(ctx.llm.resolveModelContext('deepseek', 'exact-override'))
.resolves.toEqual({ contextWindow: 64_000 })
await expect(ctx.llm.resolveModelContext('deepseek', 'unlisted-pass-through'))
.resolves.toEqual({ contextWindow: 256_000 })
await expect(ctx.llm.resolveModelInfo('deepseek', 'inherits-default'))
.resolves.toMatchObject({ context: { contextWindow: 256_000 } })
await expect(ctx.llm.resolveModelInfo('deepseek', 'exact-override'))
.resolves.toMatchObject({ context: { contextWindow: 64_000 } })
await expect(ctx.llm.resolveModelInfo('deepseek', 'unlisted-pass-through'))
.resolves.toMatchObject({ context: { contextWindow: 256_000 } })
})
it('allows an explicit empty model catalog', async () => {
+3 -3
View File
@@ -28,9 +28,9 @@ Configure credentials and deployment-specific transport settings per provider. O
Each provider name must exist in pi-ai's installed catalog and may appear only once in this plugin instance. Registration with `ctx.llm` is atomic: a collision with any provider route already owned by another adapter fails plugin loading without registering the remaining routes. Model ids are not lifecycle config; an unknown model fails before any provider request with `LlmError('UNKNOWN_MODEL')`.
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelContext(provider, model)` performs the same exact descriptor lookup and returns its context window, keeping capacity metadata on the route-owning adapter rather than a consuming plugin.
The adapter exposes each configured provider's installed pi-ai models through `ctx.llm.listModels(provider)`. This is provider-neutral selector metadata derived from `getModels(provider)`; request-time resolution still performs the authoritative catalog lookup, so discovery does not create a second model registry. `ctx.llm.resolveModelInfo(provider, model)` performs that exact descriptor lookup once and returns its identity, context window, and selectable thinking levels, keeping authoritative metadata on the route-owning adapter rather than its consumers.
`ctx.llm.resolveModelReasoning(provider, model)` uses pi-ai's `getSupportedThinkingLevels(model)` and returns that model's ordered levels after filtering the separate `off` control. The Harness exposes the canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`. A non-reasoning model returns `undefined`. The profile `reasoning` value is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and any explicit value absent from the exact model capability fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped.
The `reasoning.efforts` list is pi-ai's ordered `getSupportedThinkingLevels(model)` result without filtering or normalization, including `off` and the model-specific availability of `xhigh` or `max`. The Harness exposes each canonical pi-ai level as an opaque ID; provider/model wire spellings remain inside pi-ai's `thinkingLevelMap`. A non-reasoning model therefore exposes pi-ai's `off` choice. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and any explicit value absent from the exact model capability fails with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. pi-ai's common stream options represent `off` by omitting `reasoning`.
Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, and `streamIdleTimeoutMs`. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
@@ -49,7 +49,7 @@ If a listener rewrites assembled assistant content, the loop drops replay state
- pi-ai tool-call arguments are parsed objects; the harness stores raw JSON strings. The adapter parses input and re-stringifies output.
- pi-ai reports failures as in-stream error events; these map to `finish {kind:'error'|'aborted', failure}` chunks. Provider-specific error text distinguishes terminal `QUOTA` from transient `RATE_LIMIT`, while text and usage signals evaluated against the resolved model's context window normalize overflow to `CONTEXT_WINDOW_EXCEEDED`. A terminal `stop` whose message carries no content blocks maps to a `finish {kind:'error'}` with code `EMPTY_RESPONSE` (retried by default policy) instead of a successful empty message.
- pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.
- pi-ai may internally support an `off` thinking level, but the Harness reasoning-effort capability deliberately excludes mode changes.
- pi-ai's `off` thinking level crosses the Harness capability seam unchanged and becomes an omitted pi-ai common `reasoning` option at dispatch.
- `GenerateOptions.stop` is rejected with `UNSUPPORTED_OPTION` because pi-ai's common streaming surface cannot guarantee it across providers.
## App attribution
+28 -46
View File
@@ -11,6 +11,7 @@ import { getSupportedThinkingLevels } from '@earendil-works/pi-ai'
import type {
Api,
Model,
ModelThinkingLevel,
SimpleStreamOptions,
ThinkingLevel,
} from '@earendil-works/pi-ai'
@@ -22,9 +23,8 @@ import {
} from '@deepseek-ai/dsh-llm'
import type {
GenerateOptions,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmResolvedModelInfo,
ReasoningEffortId as ReasoningEffortIdType,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
@@ -44,7 +44,7 @@ export interface PiAiAdapterOptions {
* Resolve a catalog model dynamically and apply only the configured endpoint
* override, preserving the catalog's API/capability/compatibility metadata.
*/
function resolveModel(profile: PiAiProviderProfile, modelId: string): Model<Api> {
function resolvePiModel(profile: PiAiProviderProfile, modelId: string): Model<Api> {
const model = getBuiltinModels(profile.provider as BuiltinProvider).find(candidate => candidate.id === modelId) as Model<Api> | undefined
if (model === undefined) {
throw new LlmError(`pi-ai provider "${profile.provider}" has no catalog model "${modelId}"`, 'UNKNOWN_MODEL')
@@ -55,11 +55,12 @@ function resolveModel(profile: PiAiProviderProfile, modelId: string): Model<Api>
/** Copy profile stream knobs into pi-ai's common option vocabulary. */
function profileOptions(
profile: PiAiProviderProfile,
reasoning: ThinkingLevel | undefined,
reasoning: ModelThinkingLevel | undefined,
): SimpleStreamOptions {
const enabledReasoning: ThinkingLevel | undefined = reasoning === 'off' ? undefined : reasoning
return {
...profile.apiKey === undefined ? {} : { apiKey: profile.apiKey },
...reasoning === undefined ? {} : { reasoning },
...enabledReasoning === undefined ? {} : { reasoning: enabledReasoning },
...profile.thinkingBudgets === undefined ? {} : { thinkingBudgets: profile.thinkingBudgets },
...profile.cacheRetention === undefined ? {} : { cacheRetention: profile.cacheRetention },
...profile.transport === undefined ? {} : { transport: profile.transport },
@@ -70,19 +71,14 @@ function profileOptions(
}
}
/** Selectable pi-ai levels exclude the separate on/off control. */
function supportedReasoningLevels(model: Model<Api>): ThinkingLevel[] {
return getSupportedThinkingLevels(model).filter((level): level is ThinkingLevel => level !== 'off')
}
/** Validate an explicit Harness/profile effort without invoking pi-ai's clamp. */
function resolveReasoningLevel(
model: Model<Api>,
effort: ReasoningEffortIdType | ThinkingLevel | undefined,
): ThinkingLevel | undefined {
effort: ReasoningEffortIdType | ModelThinkingLevel | undefined,
): ModelThinkingLevel | undefined {
if (effort === undefined) return undefined
const supported = supportedReasoningLevels(model)
if (supported.some(level => level === effort)) return effort as ThinkingLevel
const supported = getSupportedThinkingLevels(model)
if (supported.some(level => level === effort)) return effort as ModelThinkingLevel
throw new LlmError(
`pi-ai provider "${model.provider}" model "${model.id}" does not support reasoning effort "${effort}"`,
'UNSUPPORTED_REASONING_EFFORT',
@@ -123,27 +119,11 @@ export class PiAiAdapter extends LlmAdapter {
})))
}
override resolveModelContext(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const profile = this.profiles.get(provider)
if (profile === undefined) {
return Promise.reject(new LlmError(
`pi-ai adapter does not own provider "${provider}"`,
'NO_ADAPTER',
))
}
return Promise.resolve().then(() => ({
contextWindow: resolveModel(profile, model).contextWindow,
}))
}
override resolveModelReasoning(
override resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined> {
): Promise<LlmResolvedModelInfo> {
const profile = this.profiles.get(provider)
if (profile === undefined) {
return Promise.reject(new LlmError(
@@ -152,21 +132,23 @@ export class PiAiAdapter extends LlmAdapter {
))
}
return Promise.resolve().then(() => {
const resolvedModel = resolveModel(profile, model)
const levels = supportedReasoningLevels(resolvedModel)
if (levels.length === 0) {
resolveReasoningLevel(resolvedModel, profile.reasoning)
return undefined
}
const resolvedModel = resolvePiModel(profile, model)
const levels = getSupportedThinkingLevels(resolvedModel)
const defaultLevel = resolveReasoningLevel(resolvedModel, profile.reasoning)
return {
efforts: levels.map(level => ({
id: ReasoningEffortId(level),
name: `${level.charAt(0).toUpperCase()}${level.slice(1)}`,
})),
...defaultLevel === undefined
? {}
: { defaultEffort: ReasoningEffortId(defaultLevel) },
provider,
id: model,
name: resolvedModel.name,
context: { contextWindow: resolvedModel.contextWindow },
reasoning: {
efforts: levels.map(level => ({
id: ReasoningEffortId(level),
name: `${level.charAt(0).toUpperCase()}${level.slice(1)}`,
})),
...defaultLevel === undefined
? {}
: { defaultEffort: ReasoningEffortId(defaultLevel) },
},
}
})
}
@@ -179,7 +161,7 @@ export class PiAiAdapter extends LlmAdapter {
if (profile === undefined) {
throw new LlmError(`pi-ai adapter does not own provider "${options.provider}"`, 'NO_ADAPTER')
}
const model = resolveModel(profile, options.model)
const model = resolvePiModel(profile, options.model)
const reasoning = resolveReasoningLevel(
model,
options.reasoningEffort ?? profile.reasoning,
+3 -3
View File
@@ -5,7 +5,7 @@
*/
import { getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
import type { CacheRetention, ThinkingBudgets, ThinkingLevel, Transport } from '@earendil-works/pi-ai'
import type { CacheRetention, ModelThinkingLevel, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
import z from 'schemastery'
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
@@ -23,7 +23,7 @@ export interface PiAiProviderProfile {
/** Provider request headers; Harness attribution wins reserved names. */
headers?: Record<string, string>
/** Provider-neutral pi-ai reasoning level. */
reasoning?: ThinkingLevel
reasoning?: ModelThinkingLevel
/** Token budgets used by reasoning providers that support them. */
thinkingBudgets?: ThinkingBudgets
/** Prompt-cache retention preference. */
@@ -62,7 +62,7 @@ const profile = z.object({
apiKey: z.string(),
baseURL: z.string(),
headers: z.dict(z.string()),
reasoning: z.union(['minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
reasoning: z.union(['off', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
thinkingBudgets,
cacheRetention: z.union(['none', 'short', 'long']),
transport: z.union(['sse', 'websocket', 'websocket-cached', 'auto']),
+14 -1
View File
@@ -9,7 +9,7 @@ import { assemble, type AssembledResult } from './assemble.ts'
/**
* Real-API e2e for the pi-ai-backed adapter: V4 Flash + V4 Pro with provider
* defaults and representative high/max reasoning. Mirrors the native
* defaults and representative off/high/max reasoning. Mirrors the native
* adapter's StreamChunk contract and exercises a replayed tool follow-up.
* Key-gated.
*/
@@ -74,6 +74,19 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
expect(textOf(result).toLowerCase()).toContain('pong')
})
it('flash + reasoning off: plain text without reasoning blocks', async () => {
const ctx = await harness(FLASH)
const result = await assemble(ctx,{
model: FLASH,
reasoningEffort: ReasoningEffortId('off'),
messages: ask('Reply with exactly the word: pong'),
maxTokens: 50,
})
expect(result.finish.kind).toBe('stop')
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(false)
expect(textOf(result).toLowerCase()).toContain('pong')
})
it.each([FLASH, PRO])('%s + reasoning high: reasoning blocks present', async (model) => {
const ctx = await harness(model)
const result = await assemble(ctx,{
+44 -23
View File
@@ -149,7 +149,7 @@ describe('PiAiAdapter provider routing', () => {
})
it('uses a dynamic request effort and rejects unsupported efforts before network I/O', async () => {
const server = await mockServer([{ events: textEvents }])
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
const ctx = await harness(server.url, { reasoning: 'max' })
await assemble(ctx, {
@@ -159,12 +159,20 @@ describe('PiAiAdapter provider routing', () => {
})
expect(server.requests[0]).toMatchObject({ reasoning_effort: 'high' })
await assemble(ctx, {
model: 'deepseek-v4-flash',
reasoningEffort: ReasoningEffortId('off'),
messages: [],
})
expect(server.requests[1]).toMatchObject({ thinking: { type: 'disabled' } })
expect(server.requests[1]).not.toHaveProperty('reasoning_effort')
await expect(assemble(ctx, {
model: 'deepseek-v4-flash',
reasoningEffort: ReasoningEffortId('xhigh'),
messages: [],
})).rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
expect(server.requests).toHaveLength(1)
expect(server.requests).toHaveLength(2)
})
it('preserves omitted profile options when constructing the adapter directly', async () => {
@@ -341,27 +349,30 @@ describe('provider profile lifecycle', () => {
provider: 'openai', id: 'gpt-4.1', name: 'GPT-4.1',
})
expect(models.every(model => model.provider === 'openai')).toBe(true)
const context = await ctx.llm.resolveModelContext('openai', 'gpt-4.1')
expect(context).toBeDefined()
expect(typeof context?.contextWindow).toBe('number')
const info = await ctx.llm.resolveModelInfo('openai', 'gpt-4.1')
expect(typeof info.context?.contextWindow).toBe('number')
})
it('exposes model-specific reasoning levels without off or an invented provider default', async () => {
it('exposes pi-ai model thinking levels verbatim without inventing a provider default', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
await ctx.plugin(LlmPiAi, {
providers: [{ provider: 'deepseek' }, { provider: 'openai' }],
})
await expect(ctx.llm.resolveModelReasoning('deepseek', 'deepseek-v4-flash'))
.resolves.toEqual({
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
await expect(ctx.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
.resolves.toMatchObject({
reasoning: {
efforts: [
{ id: ReasoningEffortId('off'), name: 'Off' },
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
},
})
const extended = await ctx.llm.resolveModelReasoning('openai', 'gpt-5.6-sol')
expect(extended?.efforts.map(effort => effort.id)).toEqual([
const extended = await ctx.llm.resolveModelInfo('openai', 'gpt-5.6-sol')
expect(extended.reasoning?.efforts.map(effort => effort.id)).toEqual([
ReasoningEffortId('off'),
ReasoningEffortId('minimal'),
ReasoningEffortId('low'),
ReasoningEffortId('medium'),
@@ -369,8 +380,12 @@ describe('provider profile lifecycle', () => {
ReasoningEffortId('xhigh'),
ReasoningEffortId('max'),
])
await expect(ctx.llm.resolveModelReasoning('openai', 'gpt-4.1'))
.resolves.toBeUndefined()
await expect(ctx.llm.resolveModelInfo('openai', 'gpt-4.1'))
.resolves.toMatchObject({
reasoning: {
efforts: [{ id: ReasoningEffortId('off'), name: 'Off' }],
},
})
})
it('uses a supported profile reasoning value as the model default and rejects an unsupported one', async () => {
@@ -379,16 +394,24 @@ describe('provider profile lifecycle', () => {
await supported.plugin(LlmPiAi, {
providers: [{ provider: 'deepseek', reasoning: 'max' }],
})
await expect(supported.llm.resolveModelReasoning('deepseek', 'deepseek-v4-flash'))
.resolves.toMatchObject({ defaultEffort: ReasoningEffortId('max') })
await expect(supported.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
.resolves.toMatchObject({ reasoning: { defaultEffort: ReasoningEffortId('max') } })
const unsupported = new Context()
await unsupported.plugin(LlmService)
await unsupported.plugin(LlmPiAi, {
providers: [{ provider: 'deepseek', reasoning: 'medium' }],
})
await expect(unsupported.llm.resolveModelReasoning('deepseek', 'deepseek-v4-flash'))
await expect(unsupported.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
.rejects.toMatchObject({ code: 'UNSUPPORTED_REASONING_EFFORT' })
const disabled = new Context()
await disabled.plugin(LlmService)
await disabled.plugin(LlmPiAi, {
providers: [{ provider: 'deepseek', reasoning: 'off' }],
})
await expect(disabled.llm.resolveModelInfo('deepseek', 'deepseek-v4-flash'))
.resolves.toMatchObject({ reasoning: { defaultEffort: ReasoningEffortId('off') } })
})
it('accepts absent credentials for pi-ai ambient authentication', async () => {
@@ -440,11 +463,9 @@ describe('provider profile lifecycle', () => {
it('constructs the adapter directly and rejects routes it does not own', async () => {
const adapter = new PiAiAdapter({ profiles: [{ provider: 'openai' }] })
await expect(adapter.listModels('anthropic')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(adapter.resolveModelContext('anthropic', 'claude-sonnet-4'))
await expect(adapter.resolveModel('anthropic', 'claude-sonnet-4'))
.rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(adapter.resolveModelReasoning('anthropic', 'claude-sonnet-4'))
.rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(adapter.resolveModelContext('openai', 'not-a-catalog-model'))
await expect(adapter.resolveModel('openai', 'not-a-catalog-model'))
.rejects.toMatchObject({ code: 'UNKNOWN_MODEL' })
await expect((async () => {
for await (const _chunk of adapter.stream({ provider: 'anthropic', model: 'claude-sonnet-4', messages: [] })) { /* drain */ }
+4 -5
View File
@@ -11,8 +11,7 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
- `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): () => void` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber.
- `ctx.llm.listProviders(): LlmProviderInfo[]` Describe registered provider routes in registration order.
- `ctx.llm.listModels(provider: string): Promise<LlmModelInfo[]>` Discover the models one registered provider currently advertises.
- `ctx.llm.resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined>` Resolve authoritative context capacity for one exact route from its owning adapter.
- `ctx.llm.resolveModelReasoning(provider: string, model: string, signal?: AbortSignal): Promise<LlmModelReasoningInfo | undefined>` Resolve ordered adapter-owned reasoning efforts and an optional deployment default for one exact route, with optional cancellation for asynchronous adapters.
- `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>` Resolve validated exact-model identity plus available context and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters.
- `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>` Validate an explicit effort and materialize an adapter-configured default without clamping.
- `ctx.llm.prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>` Resolve a config and capture its current adapter registration as one cancellable, one-shot call.
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
@@ -21,9 +20,9 @@ An adapter registry plus a single streaming call surface, interceptable via a wa
Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`.
Context capacity is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelContext()` asks the adapter that owns the exact provider/model route; an adapter can describe an unlisted dynamic model, and `undefined` means only that capacity is unavailable. Invalid returned capacity fails with `INVALID_MODEL_CONTEXT`.
Exact-model metadata is a separate correctness query, not a catalog decoration or global LLM setting. `resolveModelInfo()` asks the adapter that owns the exact provider/model route once; an adapter can describe an unlisted dynamic model, and absent `context` or `reasoning` fields mean only that those capabilities are unavailable. Invalid identity, context, or reasoning metadata fails with `INVALID_MODEL_INFO`, `INVALID_MODEL_CONTEXT`, or `INVALID_MODEL_REASONING`.
Reasoning effort is also an exact-route capability, but its identifiers are opaque adapter-owned strings rather than a core enum. `resolveModelReasoning()` validates and detaches the ordered display metadata; `undefined` means the model has no selectable effort. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. Invalid capability metadata fails with `INVALID_MODEL_REASONING`; an unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
Reasoning identifiers are opaque adapter-owned strings rather than a core enum. An adapter publishes its ordered selectable list, including an `off` id when that model's capability API exposes one. `resolveCallConfig()` accepts only an exact advertised identifier, materializes `defaultEffort` when present, and otherwise preserves the provider default. Asynchronous model resolvers receive the caller's signal and must settle promptly after cancellation. `prepareCall()` additionally retains the exact adapter registration through header logging and terminal dispatch, so HMR cannot combine one adapter's capability result with another adapter's request; reusing its one-shot handle or changing its call-config fields fails with `INVALID_PREPARED_CALL`. An unsupported explicit or configured effort fails with `UNSUPPORTED_REASONING_EFFORT` before provider I/O.
### Events
@@ -33,7 +32,7 @@ Reasoning effort is also an exact-route capability, but its identifiers are opaq
### Extension points
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, `resolveModelContext()` when exact capacity is known, and `resolveModelReasoning()` when a model exposes selectable efforts; an asynchronous reasoning resolver must honor its optional cancellation signal. The defaults use the route id as its name, advertise no models, and return neither capacity nor reasoning metadata.
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use the route and model ids as names, advertise no models, and return no capacity or reasoning metadata.
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead.
### Content-block vocabulary (`types.ts`)
+63 -69
View File
@@ -10,9 +10,8 @@ import { Context, Service } from 'cordis'
import type {
GenerateOptions,
LlmFailure,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmResolvedModelInfo,
LlmProviderInfo,
Message,
StreamChunk,
@@ -147,34 +146,20 @@ export abstract class LlmAdapter {
}
/**
* Resolve context capacity for one model accepted by this adapter. Absence
* means the adapter does not know the capacity, not that routing is invalid.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @returns provider-owned context metadata, or `undefined` when unavailable.
* Resolve all metadata available for one exact model. This query is
* independent of the advisory catalog and does not validate request routing.
* @param provider - one provider route owned by this adapter.
* @param model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; asynchronous
* implementations must settle promptly after it aborts.
* @returns provider/model identity plus any context and reasoning metadata.
*/
resolveModelContext(
_provider: string,
_model: string,
): Promise<LlmModelContext | undefined> {
return Promise.resolve(undefined)
}
/**
* Resolve selectable reasoning efforts for one exact model. Absence means
* the model has no selectable reasoning-effort capability.
* @param _provider - one provider route owned by this adapter.
* @param _model - exact model id passed to {@link GenerateOptions.model}.
* @param _signal - cancellation for this exact-model lookup; implementations
* must settle promptly after it aborts.
* @returns adapter-owned effort metadata, or `undefined` when unsupported.
*/
resolveModelReasoning(
_provider: string,
_model: string,
resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined> {
return Promise.resolve(undefined)
): Promise<LlmResolvedModelInfo> {
return Promise.resolve({ provider, id: model, name: model })
}
/**
@@ -273,53 +258,59 @@ export class LlmService extends Service {
}
/**
* Resolve context capacity from the adapter that owns one exact route.
* This query is independent of the advisory model catalog: an unlisted model
* may return metadata, while `undefined` never rejects later routing.
* Resolve and validate all metadata from the adapter that owns one exact
* route. The result is detached from adapter-owned objects; catalog
* membership remains advisory and does not control request routing.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @returns detached context metadata, or `undefined` when the adapter has none.
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
* @returns exact model identity plus available context and reasoning metadata.
*/
async resolveModelContext(
async resolveModelInfo(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
const context = await this.registration(provider).adapter.resolveModelContext(provider, model)
if (context === undefined) return undefined
if (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0) {
signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
return this.resolveModelInfoFor(this.registration(provider), model, signal)
}
private async resolveModelInfoFor(
registration: AdapterRegistration,
model: string,
signal?: AbortSignal,
): Promise<LlmResolvedModelInfo> {
const provider = registration.provider.id
const resolved = await registration.adapter.resolveModel(provider, model, signal)
if (
typeof resolved.provider !== 'string'
|| resolved.provider !== provider
|| typeof resolved.id !== 'string'
|| resolved.id !== model
|| typeof resolved.name !== 'string'
|| resolved.name.length === 0
|| (resolved.description !== undefined && typeof resolved.description !== 'string')
) {
throw new LlmError(
`adapter returned invalid exact model metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_INFO',
)
}
const context = resolved.context
if (context !== undefined && (!Number.isInteger(context.contextWindow) || context.contextWindow <= 0)) {
throw new LlmError(
`adapter returned invalid context metadata for provider "${provider}" model "${model}"`,
'INVALID_MODEL_CONTEXT',
)
}
return { contextWindow: context.contextWindow }
}
/**
* Resolve selectable reasoning efforts from the adapter that owns one exact
* route. Metadata is validated and detached; an absent result means an
* effort selector is unsupported for that model.
* @param provider - registered provider route to inspect.
* @param model - exact model id passed to the adapter.
* @param signal - optional cancellation for adapter-owned asynchronous lookup.
* @returns detached reasoning metadata, or `undefined` when unsupported.
*/
async resolveModelReasoning(
provider: string,
model: string,
signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined> {
return this.resolveModelReasoningFor(this.registration(provider), model, signal)
}
private async resolveModelReasoningFor(
registration: AdapterRegistration,
model: string,
signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined> {
const provider = registration.provider.id
const reasoning = await registration.adapter.resolveModelReasoning(provider, model, signal)
if (reasoning === undefined) return undefined
const info: LlmResolvedModelInfo = {
provider,
id: model,
name: resolved.name,
...resolved.description === undefined ? {} : { description: resolved.description },
...context === undefined ? {} : { context: { contextWindow: context.contextWindow } },
}
const reasoning = resolved.reasoning
if (reasoning === undefined) return info
if (reasoning.efforts.length === 0) {
throw new LlmError(
`adapter returned invalid reasoning metadata for provider "${provider}" model "${model}"`,
@@ -355,8 +346,11 @@ export class LlmService extends Service {
)
}
return {
efforts,
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
...info,
reasoning: {
efforts,
...reasoning.defaultEffort === undefined ? {} : { defaultEffort: reasoning.defaultEffort },
},
}
}
@@ -379,7 +373,7 @@ export class LlmService extends Service {
config: LlmCallConfig,
signal?: AbortSignal,
): Promise<LlmCallConfig> {
const reasoning = await this.resolveModelReasoningFor(registration, config.model, signal)
const reasoning = (await this.resolveModelInfoFor(registration, config.model, signal)).reasoning
const requested = config.reasoningEffort
if (reasoning === undefined) {
if (requested !== undefined) {
@@ -520,7 +514,7 @@ export class LlmService extends Service {
* `LlmError` with code `NO_ADAPTER` if no adapter is registered for
* `options.provider`. Replay state is retained only when the same adapter
* instance owns its historical provider and the target provider. Final
* adapter selection remains fixed through asynchronous reasoning resolution
* adapter selection remains fixed through asynchronous exact-model resolution
* and dispatch. Selection, dispatch, and iteration failures retain their
* original Error identity and are tagged in a call-local scope for narrow
* agent-loop request recovery; middleware and nested-call failures remain
+8
View File
@@ -182,6 +182,14 @@ export interface LlmModelReasoningInfo {
defaultEffort?: ReasoningEffortId
}
/** Exact-route model metadata resolved by its owning adapter. */
export interface LlmResolvedModelInfo extends LlmModelInfo {
/** Provider-owned context capacity when known. */
context?: LlmModelContext
/** Adapter-owned selectable reasoning levels when exposed. */
reasoning?: LlmModelReasoningInfo
}
/**
* Raw streaming protocol emitted by adapters.
* Block indexes correlate interleaved deltas, and `block-end` carries the
+71 -37
View File
@@ -19,6 +19,7 @@ import type {
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
LlmResolvedModelInfo,
} from '@deepseek-ai/dsh-llm'
class ScriptedAdapter extends LlmAdapter {
@@ -68,18 +69,17 @@ class CatalogAdapter extends ScriptedAdapter {
return Promise.resolve(this.models)
}
override resolveModelContext(
_provider: string,
override resolveModel(
provider: string,
model: string,
): Promise<LlmModelContext | undefined> {
return Promise.resolve(this.contexts[model])
}
override resolveModelReasoning(
_provider: string,
model: string,
): Promise<LlmModelReasoningInfo | undefined> {
return Promise.resolve(this.reasoning[model])
): Promise<LlmResolvedModelInfo> {
return Promise.resolve({
provider,
id: model,
name: model,
...this.contexts[model] === undefined ? {} : { context: this.contexts[model] },
...this.reasoning[model] === undefined ? {} : { reasoning: this.reasoning[model] },
})
}
}
@@ -667,8 +667,32 @@ describe('LlmService', () => {
expect(ctx.llm.listProviders()).toEqual([{ id: 'plain', name: 'plain' }])
await expect(ctx.llm.listModels('plain')).resolves.toEqual([])
await expect(ctx.llm.listModels('missing')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(ctx.llm.resolveModelContext('plain', 'unlisted')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
await expect(ctx.llm.resolveModelInfo('plain', 'unlisted')).resolves.toEqual({
provider: 'plain', id: 'unlisted', name: 'unlisted',
})
await expect(ctx.llm.resolveModelInfo('missing', 'm')).rejects.toMatchObject({ code: 'NO_ADAPTER' })
})
it.each([
[{ provider: 1, id: 'model', name: 'Model' }, 'non-string provider'],
[{ provider: 'other', id: 'model', name: 'Model' }, 'mismatched provider'],
[{ provider: 'route', id: 1, name: 'Model' }, 'non-string id'],
[{ provider: 'route', id: 'other', name: 'Model' }, 'mismatched id'],
[{ provider: 'route', id: 'model', name: 1 }, 'non-string name'],
[{ provider: 'route', id: 'model', name: '' }, 'empty name'],
[{ provider: 'route', id: 'model', name: 'Model', description: 1 }, 'non-string description'],
] as const)('rejects invalid exact model metadata (%s: %s)', async (metadata, _label) => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new class extends ScriptedAdapter {
override resolveModel(): Promise<LlmResolvedModelInfo> {
return Promise.resolve(metadata as unknown as LlmResolvedModelInfo)
}
}(SCRIPT)
ctx.llm.registerAdapter(['route'], adapter)
await expect(ctx.llm.resolveModelInfo('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_INFO' })
})
it('resolves detached model context independently of advisory catalog membership', async () => {
@@ -681,11 +705,13 @@ describe('LlmService', () => {
{ unlisted: source },
))
const resolved = await ctx.llm.resolveModelContext('route', 'unlisted')
expect(resolved).toEqual({ contextWindow: 32_000 })
const resolved = await ctx.llm.resolveModelInfo('route', 'unlisted')
expect(resolved.context).toEqual({ contextWindow: 32_000 })
source.contextWindow = 64_000
expect(resolved).toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelContext('route', 'other')).resolves.toBeUndefined()
expect(resolved.context).toEqual({ contextWindow: 32_000 })
await expect(ctx.llm.resolveModelInfo('route', 'other')).resolves.toEqual({
provider: 'route', id: 'other', name: 'other',
})
})
it('resolves detached adapter-owned reasoning metadata and materializes its default', async () => {
@@ -705,10 +731,10 @@ describe('LlmService', () => {
{ model: source },
))
const resolved = await ctx.llm.resolveModelReasoning('route', 'model')
expect(resolved).toEqual(source)
const resolved = await ctx.llm.resolveModelInfo('route', 'model')
expect(resolved.reasoning).toEqual(source)
source.efforts[0]!.name = 'mutated'
expect(resolved?.efforts[0]?.name).toBe('Standard')
expect(resolved.reasoning?.efforts[0]?.name).toBe('Standard')
await expect(ctx.llm.resolveCallConfig({ provider: 'route', model: 'model' })).resolves.toEqual({
provider: 'route',
model: 'model',
@@ -734,7 +760,7 @@ describe('LlmService', () => {
{},
{ model: metadata as unknown as LlmModelReasoningInfo },
))
await expect(ctx.llm.resolveModelReasoning('route', 'model'))
await expect(ctx.llm.resolveModelInfo('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_REASONING' })
})
@@ -764,13 +790,16 @@ describe('LlmService', () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const adapter = new class extends RecordingAdapter {
override resolveModelReasoning(
_provider: string,
_model: string,
): Promise<LlmModelReasoningInfo> {
return Promise.resolve({
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
const reasoning: LlmModelReasoningInfo = {
efforts: [{ id: ReasoningEffortId('standard'), name: 'Standard' }],
defaultEffort: ReasoningEffortId('standard'),
}
return Promise.resolve({
provider,
id: model,
name: model,
reasoning,
})
}
}(SCRIPT)
@@ -799,19 +828,24 @@ describe('LlmService', () => {
expect(Object.isFrozen(adapter.lastOptions)).toBe(true)
})
it('pins one adapter registration across asynchronous reasoning resolution and dispatch', async () => {
it('pins one adapter registration across asynchronous exact-model resolution and dispatch', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const started = Promise.withResolvers<undefined>()
const reasoning = Promise.withResolvers<LlmModelReasoningInfo>()
const first = new class extends RecordingAdapter {
override resolveModelReasoning(
_provider: string,
_model: string,
override async resolveModel(
provider: string,
model: string,
_signal?: AbortSignal,
): Promise<LlmModelReasoningInfo> {
): Promise<LlmResolvedModelInfo> {
started.resolve(undefined)
return reasoning.promise
return {
provider,
id: model,
name: model,
reasoning: await reasoning.promise,
}
}
}(SCRIPT)
const disposeFirst = ctx.llm.registerAdapter(['route'], first)
@@ -869,18 +903,18 @@ describe('LlmService', () => {
})).toThrow(expect.objectContaining({ code: 'INVALID_PREPARED_CALL' }))
})
it('passes cancellation through reasoning capability resolution', async () => {
it('passes cancellation through exact-model resolution', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
const started = Promise.withResolvers<undefined>()
const adapter = new class extends ScriptedAdapter {
override resolveModelReasoning(
override resolveModel(
_provider: string,
_model: string,
signal?: AbortSignal,
): Promise<LlmModelReasoningInfo | undefined> {
): Promise<LlmResolvedModelInfo> {
started.resolve(undefined)
return new Promise((_resolve, reject) => {
return new Promise<LlmResolvedModelInfo>((_resolve, reject) => {
if (signal === undefined) {
reject(new Error('missing reasoning signal'))
return
@@ -918,7 +952,7 @@ describe('LlmService', () => {
[],
{ model: { contextWindow } },
))
await expect(ctx.llm.resolveModelContext('route', 'model'))
await expect(ctx.llm.resolveModelInfo('route', 'model'))
.rejects.toMatchObject({ code: 'INVALID_MODEL_CONTEXT' })
},
)
+1 -1
View File
@@ -4,7 +4,7 @@ Replay-aware token measurement through the singleton `ctx.tokenMeter` service. I
## Configuration
The estimator has no settings. It intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. Any key is rejected, including the obsolete global `contextWindow`; model capacity belongs to the adapter that owns an exact provider/model route and is available through `ctx.llm.resolveModelContext()`.
The estimator has no settings. It intentionally uses one fixed heuristic: four characters per token plus structural overhead for roles, blocks, and request-envelope fields. Any key is rejected, including the obsolete global `contextWindow`; model capacity belongs to the adapter that owns an exact provider/model route and is available through `ctx.llm.resolveModelInfo().context`.
## Measurement contract
+13 -5
View File
@@ -11,7 +11,7 @@ import { delimiter as pathDelimiter } from 'node:path'
import type { Context } from 'cordis'
import { decodeStorageRecord } from '@deepseek-ai/dsh-session'
import type { SessionEvent } from '@deepseek-ai/dsh-session'
import type { GenerateOptions, LlmModelContext, LlmModelInfo, LlmProviderInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmProviderInfo, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import { LlmAdapter, LlmError, assertNever } from '@deepseek-ai/dsh-llm'
/**
@@ -268,12 +268,20 @@ class ReplayAdapter extends LlmAdapter {
})))
}
override resolveModelContext(provider: string, model: string): Promise<LlmModelContext | undefined> {
override resolveModel(provider: string, model: string): Promise<LlmResolvedModelInfo> {
const configured = this.providers.get(provider)
/* v8 ignore next -- LlmService only asks about routes registered from this same map. */
if (configured === undefined) return Promise.resolve(undefined)
const contextWindow = configured.models?.find(candidate => candidate.id === model)?.contextWindow
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
if (configured === undefined) return Promise.resolve({ provider, id: model, name: model })
const configuredModel = configured.models?.find(candidate => candidate.id === model)
return Promise.resolve({
provider,
id: model,
name: configuredModel?.name ?? model,
...configuredModel?.description === undefined ? {} : { description: configuredModel.description },
...configuredModel?.contextWindow === undefined
? {}
: { context: { contextWindow: configuredModel.contextWindow } },
})
}
override stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
@@ -258,10 +258,12 @@ describe('installLlmReplay (through the real LlmService)', () => {
{ provider: 'deepseek', id: 'pro', name: 'Pro', description: 'Larger model' },
])
await expect(ctx.llm.listModels('empty')).resolves.toEqual([])
await expect(ctx.llm.resolveModelContext('deepseek', 'flash')).resolves.toEqual({ contextWindow: 128_000 })
await expect(ctx.llm.resolveModelContext('deepseek', 'pro')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('deepseek', 'unlisted')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelContext('empty', 'unlisted')).resolves.toBeUndefined()
await expect(ctx.llm.resolveModelInfo('deepseek', 'flash')).resolves.toMatchObject({
context: { contextWindow: 128_000 },
})
await expect(ctx.llm.resolveModelInfo('deepseek', 'pro')).resolves.not.toHaveProperty('context')
await expect(ctx.llm.resolveModelInfo('deepseek', 'unlisted')).resolves.not.toHaveProperty('context')
await expect(ctx.llm.resolveModelInfo('empty', 'unlisted')).resolves.not.toHaveProperty('context')
expect(await drain(ctx.llm.stream({ provider: 'deepseek', model: 'pro', messages: [] }))).toEqual(TEXT_CHUNKS)
dispose()
+3 -3
View File
@@ -10,7 +10,7 @@ This package owns interactive terminal presentation and input only. It injects `
After terminal startup succeeds, the package provides the terminal-local `ctx.tui` extension service. A plugin that injects it can call `openOverlay()` with a component factory and constrained layout options; the host exposes the viewport, semantic theme, display-text escaping, redraw, close, and a lifetime signal, but not the pi-tui tree, terminal, focus controller, or overlay handle. Plugin overlays, the model selector, and user questions share one FIFO modal queue. Each request is an effect of the calling plugin fiber, so unload removes queued work or closes visible work before cleanup settles; terminal shutdown unloads dependents before stopping pi-tui. Overlay state is not logged or replayed. Component code is trusted and may render ANSI styling, but must pass untrusted text through `host.display()`. The [interactive-extension Agent Note](../../../.agents/notes/implemented/architecture/2026-07-22-tui-interactive-extension-service.md) owns the boundary and rejected alternatives.
The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions in a wide bottom-left keyboard panel with progress, numbered options, and aligned descriptions. The latest logged session title becomes the header subtitle, with `welcome` before a title exists, and the terminal window title becomes `<session title> — <configured title>`. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Its idle view compares token-meter pressure with `ctx.llm.resolveModelContext()` for the current route, displays `context unknown` when the adapter has no capacity metadata, and also shows tool-card mode plus the current model and any explicitly selected reasoning effort; while the agent runs, an elapsed working indicator and `esc interrupt` replace that summary. Surface replacement events rebuild the transcript so compacted history does not reappear.
The TUI rebuilds resumed history from the active session surface, renders Markdown responses and reasoning, applies each tool's `presentCall` / `presentResult` intent to terminal, diff, or generic cards, keeps the latest `todo/write` plan above the editor, and presents `ctx.userInteraction` questions in a wide bottom-left keyboard panel with progress, numbered options, and aligned descriptions. The latest logged session title becomes the header subtitle, with `welcome` before a title exists, and the terminal window title becomes `<session title> — <configured title>`. A durable `llm/retry` event retracts the failed step's live chunks and renders the scheduled retry count, delay, and failure in the transcript; success, exhaustion, and cancellation then settle through ordinary session events. The footer totals each logged model step's usage once, including failed attempts, while treating committed-message usage as a fallback for logs without a usage chunk. Its idle view compares token-meter pressure with `ctx.llm.resolveModelInfo()` context for the current route, displays `context unknown` when the adapter has no capacity metadata, and also shows tool-card mode plus the current model and any explicitly selected reasoning effort; while the agent runs, an elapsed working indicator and `esc interrupt` replace that summary. Surface replacement events rebuild the transcript so compacted history does not reappear.
An embedding may provide `TuiRuntime.formatCwd` when its logical workspace label differs from the session's host directory. The override changes only the footer label; tools continue to use the session `cwd`.
@@ -22,11 +22,11 @@ When optional `ctx.sessionReferences` is mounted, the same `@` menu also offers
While the agent is running, ordinary editor submissions call `agent.steer()`; otherwise they call `agent.followup()`. A slash at the start of the submitted line enters `ctx.commands` instead: known commands execute directly, unknown commands produce a warning, and neither path automatically reaches the model. A command producer may explicitly schedule agent work; [`dsh-plan-mode`](../../plan/plan-mode/README.md#model-and-human-surfaces) uses that contract for `/plan [message]`. The TUI registers `/help`, `/model`, `/clear`, `/reasoning`, `/tools`, `/redraw`, `/reload`, `/resume`, `/status`, and `/exit` as agent-scoped definitions; every other effective command joins autocomplete and `/help` dynamically, as do `/skill:` completions. A status line above the editor reports the turn phase the TUI derives from session events — waiting for the first token, thinking, responding, or executing tools — with the elapsed time in that phase and the running step total, refreshed each second, and ends with the `Enter sends steering, Esc cancels` hint; while steering messages wait to reach the model it inserts a `N queued ·` badge before the hint that clears as each drains. Ctrl+C or Escape cancels a running turn. Tool cards collapse long bodies into a configurable head/tail preview; Ctrl+O toggles every card between its preview and full output. Ctrl+R toggles reasoning, Ctrl+L redraws, and Ctrl+D exits while idle.
`/model` opens the advisory `ctx.llm` catalog as a keyboard selector: Up/Down moves, Shift+Tab cycles the focused model's adapter-advertised reasoning efforts in display order, Enter selects the model and effort, and Escape closes it. Models without selectable effort metadata ignore Shift+Tab; the selector does not synthesize `off`, clamp a value, or transfer an effort between models. `/model <model>` still selects an unambiguous model id directly, while `/model <provider>/<model>` selects an exact target and uses its adapter default when one exists. The configured target or latest logged request header initializes the selector, and an unlisted current model remains visible because catalogs are advisory. Selection is local to this TUI session. Prompt assembly snapshots the target for one step, replaces `{{provider}}` and `{{model}}`, and applies the same provider/model/reasoning-effort target through `agent/request`; a switch during assembly therefore starts with a later step. The request header durably records targets that reach the model, while an unused selection remains process-local.
`/model` opens the advisory `ctx.llm` catalog as a keyboard selector: Up/Down moves, Shift+Tab cycles the focused model's adapter-advertised reasoning efforts in display order, Enter selects the model and effort, and Escape closes it. Models without selectable effort metadata ignore Shift+Tab; the selector renders the exact advertised list—including `off` when present—and does not synthesize, clamp, or transfer an effort between models. `/model <model>` still selects an unambiguous model id directly, while `/model <provider>/<model>` selects an exact target and uses its adapter default when one exists. The configured target or latest logged request header initializes the selector, and an unlisted current model remains visible because catalogs are advisory. Selection is local to this TUI session. Prompt assembly snapshots the target for one step, replaces `{{provider}}` and `{{model}}`, and applies the same provider/model/reasoning-effort target through `agent/request`; a switch during assembly therefore starts with a later step. The request header durably records targets that reach the model, while an unused selection remains process-local.
`/reload` (EXPERIMENTAL, dev-only) re-reads every file-backed loader config tree and applies the diff to the running app — the HMR watcher's config path, invoked manually; it needs the cordis Loader in the context and degrades to a warning without one, runs only while the agent is idle, and refuses re-entry while a reload is in flight. Module-source hot reload remains watcher-owned. When a `skills` service is mounted, `/skill:<name> [instructions]` loads that skill's instructions into the conversation as a user turn; autocomplete lists the model-invocable skills, and any skill (including a model-disabled one) is loadable by its exact name.
The footer sums the session's reported usage as `↑<uncached input> ↓<output>`, followed by `cache <rate>%` once any input has been billed — the share of billed prompt tokens (uncached input plus cache reads and writes) served from the provider cache, rounded to a percent. It also compares token-meter pressure with `ctx.llm.resolveModelContext()` for the current route (omitting the context share when the adapter has no capacity metadata) and shows the current model and tool-card mode; the right side clips first when the footer is narrow.
The footer sums the session's reported usage as `↑<uncached input> ↓<output>`, followed by `cache <rate>%` once any input has been billed — the share of billed prompt tokens (uncached input plus cache reads and writes) served from the provider cache, rounded to a percent. It also compares token-meter pressure with `ctx.llm.resolveModelInfo()` context for the current route (omitting the context share when the adapter has no capacity metadata) and shows the current model and tool-card mode; the right side clips first when the footer is narrow.
`/status` adds a point-in-time diagnostics card to the transcript and remains available while the agent runs. It reports the session id, title, working directory, selected provider/model, selected reasoning effort or default behavior, reasoning-block visibility, agent state, event/turn/step/tool-call counts, exact input/output/cache token buckets, KV-cache hit rate, token-meter context use and capacity, creation time, and latest event time. Missing titles, models, cache input, or context capacity are labeled instead of inferred. The card is terminal-only and does not duplicate the compact footer.
+3 -3
View File
@@ -630,7 +630,7 @@ async function readModelChoices(
models.push({ provider: provider.id, id: current.model, name: current.model })
}
return Promise.all(models.map(async (model): Promise<ModelChoice> => {
const reasoning = await ctx.llm.resolveModelReasoning(provider.id, model.id)
const reasoning = (await ctx.llm.resolveModelInfo(provider.id, model.id)).reasoning
return {
provider: provider.id,
model: model.id,
@@ -2101,8 +2101,8 @@ export function createTuiChat(
contextWindow = undefined
const resolution = selected === undefined
? Promise.resolve({ kind: 'resolved', contextWindow: undefined } as const)
: ctx.llm.resolveModelContext(selected.provider, selected.model).then(
context => ({ kind: 'resolved', contextWindow: context?.contextWindow } as const),
: ctx.llm.resolveModelInfo(selected.provider, selected.model).then(
info => ({ kind: 'resolved', contextWindow: info.context?.contextWindow } as const),
(error: unknown) => ({ kind: 'error', error } as const),
)
contextResolution = resolution
+17 -11
View File
@@ -10,10 +10,9 @@ import AgentRegistry, {
} from '@deepseek-ai/dsh-agent'
import type {
ContentBlock,
LlmModelContext,
LlmModelInfo,
LlmModelReasoningInfo,
LlmProviderInfo,
LlmResolvedModelInfo,
} from '@deepseek-ai/dsh-llm'
import CommandService from '@deepseek-ai/dsh-commands'
import SessionStore, { SessionId, type Session, type SessionHeader } from '@deepseek-ai/dsh-session'
@@ -53,11 +52,10 @@ export interface TuiHarnessOptions {
providers: LlmProviderInfo[]
models: LlmModelInfo[]
listModels?: (provider: string) => Promise<LlmModelInfo[]>
resolveModelContext?: (provider: string, model: string) => Promise<LlmModelContext | undefined>
resolveModelReasoning?: (
resolveModelInfo?: (
provider: string,
model: string,
) => Promise<LlmModelReasoningInfo | undefined>
) => Promise<Pick<LlmResolvedModelInfo, 'context' | 'reasoning'>>
}
/** Provide a fake `sessionPersistence` service so resume surfaces can list sessions. */
sessionPersistence?: {
@@ -128,12 +126,20 @@ export async function createTuiTestHarness<TerminalType extends Terminal, Exit e
return catalog.listModels?.(provider)
?? Promise.resolve(catalog.models.filter(model => model.provider === provider).map(model => ({ ...model })))
},
resolveModelContext(provider: string, model: string) {
return catalog.resolveModelContext?.(provider, model)
?? Promise.resolve({ contextWindow: options.contextWindow ?? 128_000 })
},
resolveModelReasoning(provider: string, model: string) {
return catalog.resolveModelReasoning?.(provider, model) ?? Promise.resolve(undefined)
async resolveModelInfo(provider: string, model: string) {
const advertised = catalog.models.find(candidate =>
candidate.provider === provider && candidate.id === model)
const capabilities = await (catalog.resolveModelInfo?.(provider, model)
?? Promise.resolve({
context: { contextWindow: options.contextWindow ?? 128_000 },
}))
return {
provider,
id: model,
name: advertised?.name ?? model,
...advertised?.description === undefined ? {} : { description: advertised.description },
...capabilities,
}
},
} as never)
}
@@ -26,7 +26,7 @@ buffer
style 8-8 fg=bright-blue
style 38-77 fg=bright-black
style 83-83 fg=bright-blue
15| " │ → deepseek/deepseek-v4-pro DeepSeek V4 Pro — Max │ "
15| " │ → deepseek/deepseek-v4-pro DeepSeek V4 Pro — Off │ "
style 8-8 fg=bright-blue
style 10-64 fg=bright-blue inverse
style 83-83 fg=bright-blue
@@ -8,10 +8,10 @@ buffer
style 10-16 bold
1| " Snapshot agent ready."
style 1-21 fg=bright-black
2| " deepseek-v4-pro max • main-session"
2| " deepseek-v4-pro off • main-session"
style 1-36 dim
3| <blank>
4| " Model selected: deepseek/deepseek-v4-pro. Reasoning effort: Max. New steps will use it. "
4| " Model selected: deepseek/deepseek-v4-pro. Reasoning effort: Off. New steps will use it. "
style 1-87 fg=bright-black
5| "────────────────────────────────────────────────────────────────────────────────────────────"
style 0-91 dim
@@ -19,7 +19,7 @@ buffer
style 1-1 inverse
7| "────────────────────────────────────────────────────────────────────────────────────────────"
style 0-91 dim
8| "deepseek-v4-pro max /workspace/project ↑0 ↓0 0% context tools:collapsed"
8| "deepseek-v4-pro off /workspace/project ↑0 ↓0 0% context tools:collapsed"
style 0-45 dim
style 65-91 dim
9-31| <blank>
+11 -6
View File
@@ -640,12 +640,16 @@ describe('TUI terminal-state snapshots', () => {
{ provider: 'deepseek', id: 'deepseek-v4-flash', name: 'DeepSeek V4 Flash' },
{ provider: 'deepseek', id: 'deepseek-v4-pro', name: 'DeepSeek V4 Pro' },
],
resolveModelReasoning: () => Promise.resolve({
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
resolveModelInfo: () => Promise.resolve({
context: { contextWindow: 128_000 },
reasoning: {
efforts: [
{ id: ReasoningEffortId('off'), name: 'Off' },
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
},
}),
},
}, { columns: 92, rows: 32 })
@@ -657,6 +661,7 @@ describe('TUI terminal-state snapshots', () => {
await renderAfter(harness, () => {
harness.terminal.send('\x1b[B')
harness.terminal.send('\x1b[Z')
harness.terminal.send('\x1b[Z')
})
await checkpoint('model-effort-switching', harness.terminal, { includeScrollback: true })
await renderAfter(harness, () => {
+36 -23
View File
@@ -5,7 +5,11 @@ import { describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import { CombinedAutocompleteProvider, type Terminal } from '@earendil-works/pi-tui'
import AgentRegistry, { agentEvents, assembleContextFor, AgentMessageId, type Agent } from '@deepseek-ai/dsh-agent'
import { ReasoningEffortId, type LlmCallConfig } from '@deepseek-ai/dsh-llm'
import {
ReasoningEffortId,
type LlmCallConfig,
type LlmModelReasoningInfo,
} from '@deepseek-ai/dsh-llm'
import { GOAL_CHANGE_VERSION, GoalId, renderGoalChange, type GoalSnapshotChangeMeta } from '@deepseek-ai/dsh-goal'
import CommandService, { type CommandInvocation } from '@deepseek-ai/dsh-commands'
import SessionStore, { SessionId, type JsonValue, type SessionEvent, type SessionHeader, type TurnEndReason } from '@deepseek-ai/dsh-session'
@@ -143,7 +147,11 @@ function provideLlmCatalog(ctx: Context): void {
ctx.provide('llm', {
listProviders: () => [],
listModels: () => Promise.resolve([]),
resolveModelContext: () => Promise.resolve(undefined),
resolveModelInfo: (provider: string, model: string) => Promise.resolve({
provider,
id: model,
name: model,
}),
} as never)
}
@@ -1155,7 +1163,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
expect(result.terminal.output).toContain('restored answer')
expect(result.terminal.output).toContain('write tests')
expect(result.terminal.output).toContain('↑1.3k ↓42')
// Context resolution is async (resolveModelContext); settle before reading.
// Exact model resolution is async; settle before reading.
await tick()
expect(result.terminal.output).toContain('42% context tools:collapsed')
// Narrow terminals clip the right-hand context/tools segment first; the
@@ -1741,7 +1749,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [],
models: [],
resolveModelContext: () => Promise.resolve(undefined),
resolveModelInfo: () => Promise.resolve({}),
},
})
result.terminal.send('/status')
@@ -2288,6 +2296,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
it('opens a keyboard selector and switches the session model without sending slash text to the agent', async () => {
const initialContext = Promise.withResolvers<{ contextWindow: number }>()
let deferInitialContext = true
const result = await setup({
agentOptions: { provider: 'alpha', model: 'a1' },
contextTokens: 50,
@@ -2299,37 +2308,41 @@ describe('pi-tui chat lifecycle and transcript', () => {
{ provider: 'beta', id: 'b1', name: 'Beta One' },
{ provider: 'beta', id: 'shared', name: 'Beta Shared' },
],
resolveModelContext: (provider, model) => provider === 'alpha' && model === 'a1'
? initialContext.promise
: Promise.resolve({ contextWindow: 200 }),
resolveModelReasoning: (provider, model) => {
async resolveModelInfo(provider, model) {
const shouldDeferContext = provider === 'alpha' && model === 'a1' && deferInitialContext
if (shouldDeferContext) deferInitialContext = false
const context = shouldDeferContext
? await initialContext.promise
: { contextWindow: 200 }
let reasoning: LlmModelReasoningInfo | undefined
if (model === 'a1') {
return Promise.resolve({
reasoning = {
efforts: [
{ id: ReasoningEffortId('low'), name: 'Low' },
{ id: ReasoningEffortId('high'), name: 'High' },
],
defaultEffort: ReasoningEffortId('low'),
})
}
if (model === 'b1') {
return Promise.resolve({
}
} else if (model === 'b1') {
reasoning = {
efforts: [
{ id: ReasoningEffortId('high'), name: 'High' },
{ id: ReasoningEffortId('max'), name: 'Max' },
],
defaultEffort: ReasoningEffortId('high'),
})
}
if (provider === 'alpha' && model === 'shared') {
return Promise.resolve({
}
} else if (provider === 'alpha' && model === 'shared') {
reasoning = {
efforts: [
{ id: ReasoningEffortId('standard'), name: 'Standard' },
{ id: ReasoningEffortId('ultra'), name: 'Ultra' },
],
})
}
}
return {
context,
...reasoning === undefined ? {} : { reasoning },
}
return Promise.resolve(undefined)
},
},
})
@@ -2522,7 +2535,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'alpha', name: 'Alpha' }],
models: [{ provider: 'alpha', id: 'a1', name: 'Alpha One' }],
resolveModelContext: () => Promise.resolve(undefined),
resolveModelInfo: () => Promise.resolve({}),
},
})
unset.terminal.send('/model')
@@ -2554,7 +2567,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [],
listModels: () => Promise.reject(new Error('catalog offline')),
resolveModelContext: () => Promise.reject(new Error('capacity offline')),
resolveModelInfo: () => Promise.reject(new Error('capacity offline')),
},
})
failed.terminal.send('/model')
@@ -2569,7 +2582,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [{ provider: 'deepseek', id: 'model-1', name: 'Model One' }],
resolveModelReasoning: () => Promise.reject(new Error('reasoning metadata offline')),
resolveModelInfo: () => Promise.reject(new Error('reasoning metadata offline')),
},
})
reasoningFailed.terminal.send('/model')
@@ -2619,7 +2632,7 @@ describe('pi-tui chat lifecycle and transcript', () => {
catalog: {
providers: [{ id: 'deepseek', name: 'DeepSeek' }],
models: [],
resolveModelContext: () => context.promise,
resolveModelInfo: () => context.promise.then(value => ({ context: value })),
},
})
await contextResult.controller.dispose()
+1
View File
@@ -41,6 +41,7 @@ export const LINK_MAP: Record<string, string> = {
LlmCallConfig: 'core.md',
LlmModelContext: 'core.md',
LlmModelReasoningInfo: 'core.md',
LlmResolvedModelInfo: 'core.md',
LlmFailure: 'llm-streaming.md',
LlmModelInfo: 'core.md',
LlmProviderInfo: 'core.md',
+10
View File
@@ -61,6 +61,11 @@
"symbol": "LlmModelReasoningInfo",
"source": "packages/llm/llm/src/types.ts"
},
{
"doc": "docs/core-data-structures/core.md",
"symbol": "LlmResolvedModelInfo",
"source": "packages/llm/llm/src/types.ts"
},
{
"doc": "docs/core-data-structures/core.md",
"symbol": "GenerateOptions",
@@ -1309,6 +1314,11 @@
"symbol": "LlmModelReasoningInfo",
"source": "packages/llm/llm/src/types.ts"
},
{
"doc": "docs/core-data-structures/core.zh.md",
"symbol": "LlmResolvedModelInfo",
"source": "packages/llm/llm/src/types.ts"
},
{
"doc": "docs/core-data-structures/core.zh.md",
"symbol": "GenerateOptions",