Merge pull request #540 from deepseek-harness/worktree/llm-default-context-window
feat(llm): add default context window fallback
This commit is contained in:
Tianyi Cui
GitHub
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-20-routed-model-context-and-compaction-policy.md: f0b9288d3d864bfcc2964862b1ff294406daa345
|
||||
2026-07-20-routed-model-context-and-compaction-policy.zh.md: cda740a5671a3ef8a5bb415e5cc45ca8397c1c59
|
||||
2026-07-20-routed-model-context-and-compaction-policy.md: b637ba24d4ba5fc25c8cdd515a821ee97883a326
|
||||
2026-07-20-routed-model-context-and-compaction-policy.zh.md: 084e762ec29ddc0aecb0bf422c147b9d3122726b
|
||||
+3
-3
@@ -16,7 +16,7 @@ Neither obvious configuration owner is sufficient. Compact-basic is optional and
|
||||
|
||||
`LlmAdapter.resolveModelContext(provider, model)` optionally returns `LlmModelContext` for one exact route. `LlmService.resolveModelContext()` selects the registered route owner, validates a positive integer `contextWindow`, and returns a detached value. The query is independent of `listModels()`: an unlisted dynamic model may have capacity metadata, and `undefined` means only that the adapter cannot describe capacity.
|
||||
|
||||
The hand-rolled DeepSeek adapter accepts optional `contextWindow` on each configured model. Its two default model entries publish 128,000 tokens; an explicit entry without capacity and an unlisted pass-through id return `undefined`. The pi-ai adapter resolves capacity from the same catalog descriptor that authoritatively resolves the request model.
|
||||
The hand-rolled DeepSeek adapter accepts optional `contextWindow` on each configured model plus an adapter-wide `defaultContextWindow`. Exact model capacity wins; an entry without capacity and an unlisted pass-through id inherit the adapter default, or return `undefined` when it is absent. The two built-in model entries each publish an exact 128,000-token capacity. The pi-ai adapter resolves capacity from the same catalog descriptor that authoritatively resolves the request model.
|
||||
|
||||
### Token measurement remains model-agnostic
|
||||
|
||||
@@ -36,7 +36,7 @@ An adapter that lacks capacity metadata remains a valid LLM route. Manual proact
|
||||
|
||||
## Testing
|
||||
|
||||
Service tests cover detached context metadata, invalid adapter output, catalog independence, and default absence. Adapter tests cover DeepSeek configured/default/unlisted behavior and pi-ai exact descriptor resolution. Compact tests cover ratio scaling, exact provider/model overrides, load-time rejection of invalid merged ratios, runtime absolute-budget validation, same-model-id provider switches, target-specific warning suppression, and capacity-independent overflow recovery. Loader fixtures reject the removed token-meter capacity setting, and examples configure capacity on adapters.
|
||||
Service tests cover detached context metadata, invalid adapter output, catalog independence, and default absence. Adapter tests cover DeepSeek exact/default/unlisted resolution, invalid capacities, and pi-ai exact descriptor resolution. Compact tests cover ratio scaling, exact provider/model overrides, load-time rejection of invalid merged ratios, runtime absolute-budget validation, same-model-id provider switches, target-specific warning suppression, and capacity-independent overflow recovery. Loader fixtures reject the removed token-meter capacity setting, and examples configure capacity on adapters.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -51,7 +51,7 @@ Service tests cover detached context metadata, invalid adapter output, catalog i
|
||||
- Capacity has one authoritative owner at the provider seam, while compaction policy stays in the optional consuming plugin.
|
||||
- The same compact-basic instance safely handles different windows, provider switches, and identical model ids under different providers without consulting discovery metadata.
|
||||
- LLM-only and meter-only compositions remain valid; loading compact-basic adds no reverse dependency from adapters.
|
||||
- Deployments using explicit DeepSeek model lists must provide `contextWindow` for proactive pressure on those entries. Missing metadata is visible instead of silently applying a wrong global fallback.
|
||||
- DeepSeek deployments may set exact per-model capacities, or use `defaultContextWindow` for entries without capacity and unlisted pass-through ids.
|
||||
- Ratio defaults scale naturally across models, while exact-target absolute retention remains available for deployment-specific behavior.
|
||||
|
||||
This note supersedes the global-capacity and no-model-policy parts of the [replay token meter service Agent Note](2026-07-15-replay-token-meter-service.md). Its single-fold measurement decision remains unchanged.
|
||||
+3
-3
@@ -16,7 +16,7 @@ Status: implemented
|
||||
|
||||
`LlmAdapter.resolveModelContext(provider, model)` 可以为一条精确路由返回 `LlmModelContext`。`LlmService.resolveModelContext()` 选择已注册的路由所属方，验证 `contextWindow` 为正整数，并返回分离值。该查询独立于 `listModels()`：不在目录中的动态模型也可以拥有容量元数据，而 `undefined` 只表示适配器无法描述容量。
|
||||
|
||||
手写 DeepSeek 适配器允许每个已配置模型提供可选 `contextWindow`。两个默认模型项都公开 128,000 token；未提供容量的显式模型项与未列出的透传 id 返回 `undefined`。pi-ai 适配器从同一个目录描述符解析容量，该描述符也用于权威解析请求模型。
|
||||
手写 DeepSeek 适配器允许每个已配置模型提供可选 `contextWindow`，并支持适配器级 `defaultContextWindow`。精确模型容量优先；未提供容量的模型项与未列出的透传 id 会继承适配器默认值，若默认值也不存在则返回 `undefined`。两个内置模型项都公开精确的 128,000 token 容量。pi-ai 适配器从同一个目录描述符解析容量，该描述符也用于权威解析请求模型。
|
||||
|
||||
### Token 计量保持模型无关
|
||||
|
||||
@@ -36,7 +36,7 @@ Compact-basic 拥有消费方策略。顶层字段定义默认值；`modelPolici
|
||||
|
||||
## 测试
|
||||
|
||||
服务测试覆盖分离上下文元数据、无效适配器输出、目录独立性与默认缺失行为。适配器测试覆盖 DeepSeek 的配置值、默认值与未列出行为，以及 pi-ai 的精确描述符解析。压缩测试覆盖比例缩放、精确提供方/模型覆盖、加载期拒绝无效合并比例、运行时校验绝对预算、相同模型 id 的提供方切换、目标专用警告抑制与不依赖容量的溢出恢复。Loader fixture 会拒绝已经移除的 token-meter 容量设置，示例则在适配器上配置容量。
|
||||
服务测试覆盖分离上下文元数据、无效适配器输出、目录独立性与默认缺失行为。适配器测试覆盖 DeepSeek 的精确容量、默认容量、未列出模型解析及无效容量，以及 pi-ai 的精确描述符解析。压缩测试覆盖比例缩放、精确提供方/模型覆盖、加载期拒绝无效合并比例、运行时校验绝对预算、相同模型 id 的提供方切换、目标专用警告抑制与不依赖容量的溢出恢复。Loader fixture 会拒绝已经移除的 token-meter 容量设置，示例则在适配器上配置容量。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
@@ -51,7 +51,7 @@ Compact-basic 拥有消费方策略。顶层字段定义默认值；`modelPolici
|
||||
- 容量在提供方 seam 上拥有唯一权威归属方，而压缩策略留在可选消费插件中。
|
||||
- 同一个 compact-basic 实例无需查询发现元数据，就能安全处理不同窗口、提供方切换，以及不同提供方下的相同模型 id。
|
||||
- 仅 LLM 与仅 meter 的组合仍然有效；加载 compact-basic 不会让适配器产生反向依赖。
|
||||
- 使用显式 DeepSeek 模型列表的部署必须为需要主动压力检查的条目提供 `contextWindow`。系统会暴露缺失元数据，而不是静默应用错误的全局回退值。
|
||||
- DeepSeek 部署可以设置精确的逐模型容量，也可以让未提供容量的模型项与未列出的透传 id 使用 `defaultContextWindow`。
|
||||
- 比例默认值会随模型自然缩放，同时仍可按精确目标使用绝对保留值，以满足部署专用行为。
|
||||
|
||||
本记录取代[回放式 token 计量服务 Agent Note](2026-07-15-replay-token-meter-service.md) 中的全局容量与无模型策略部分，单折叠计量决策保持不变。
|
||||
@@ -532,6 +532,8 @@ export interface Config {
|
||||
thinking?: 'enabled' | 'disabled'
|
||||
/** Thinking effort (only meaningful with thinking enabled). */
|
||||
reasoningEffort?: 'high' | 'max'
|
||||
/** Positive context capacity used when the selected model has no exact value. */
|
||||
defaultContextWindow?: number
|
||||
/** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */
|
||||
models?: DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
|
||||
|
||||
@@ -12,11 +12,10 @@
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL
|
||||
thinking: enabled
|
||||
reasoningEffort: max
|
||||
defaultContextWindow: 256000
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
contextWindow: 256000
|
||||
- id: deepseek-v4-pro
|
||||
contextWindow: 256000
|
||||
|
||||
# The default composition confines bash AND the filesystem tools to the
|
||||
# workspace and asks before a wider retry. Snapshot runs select
|
||||
|
||||
@@ -17,10 +17,10 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; high | max — omitted ⇒ not sent
|
||||
streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default
|
||||
defaultContextWindow: 256000 # optional positive-integer fallback for models without an exact value
|
||||
models: # optional; defaults to V4 Flash and V4 Pro
|
||||
- id: deepseek-v4-flash
|
||||
name: DeepSeek V4 Flash
|
||||
contextWindow: 128000
|
||||
- id: private-reasoner
|
||||
description: Company-hosted reasoning model
|
||||
contextWindow: 64000
|
||||
@@ -28,7 +28,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
|
||||
The plugin registers the single provider route `deepseek`. A request selects it with `provider: deepseek`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` and `deepseek-v4-pro`, each with a 128,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek')` for clients such as ACP editors, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id.
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelContext('deepseek', model)` returns it only for an exact configured id; omission or an unlisted pass-through model returns `undefined` without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelContext('deepseek', model)` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. When neither value exists it returns `undefined` without invalidating routing. Pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort` wire field is not sent and the server applies its own default for the model. The only accepted values are `high` and `max` (DeepSeek's official effort levels). It is meaningful only with thinking enabled (the provider default).
|
||||
|
||||
|
||||
@@ -40,6 +40,8 @@ export interface DeepSeekAdapterOptions {
|
||||
baseURL: string
|
||||
/** Request defaults applied to every call (thinking mode, effort). */
|
||||
defaults?: RequestDefaults
|
||||
/** Positive context capacity used when the selected model has no exact value. */
|
||||
defaultContextWindow?: number
|
||||
/** Advisory models exposed to discovery consumers; requests remain unrestricted. */
|
||||
models?: readonly DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
@@ -96,6 +98,10 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
|
||||
constructor(private readonly options: DeepSeekAdapterOptions) {
|
||||
super()
|
||||
if (options.defaultContextWindow !== undefined
|
||||
&& (!Number.isInteger(options.defaultContextWindow) || options.defaultContextWindow <= 0)) {
|
||||
throw new Error('llm-deepseek: defaultContextWindow must be a positive integer')
|
||||
}
|
||||
this.streamIdleTimeoutMs = options.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS
|
||||
if (!Number.isFinite(this.streamIdleTimeoutMs)
|
||||
|| this.streamIdleTimeoutMs <= 0
|
||||
@@ -124,6 +130,7 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
model: string,
|
||||
): Promise<LlmModelContext | undefined> {
|
||||
const contextWindow = this.options.models?.find(entry => entry.id === model)?.contextWindow
|
||||
?? this.options.defaultContextWindow
|
||||
return Promise.resolve(contextWindow === undefined ? undefined : { contextWindow })
|
||||
}
|
||||
|
||||
|
||||
@@ -40,6 +40,8 @@ export interface Config {
|
||||
thinking?: 'enabled' | 'disabled'
|
||||
/** Thinking effort (only meaningful with thinking enabled). */
|
||||
reasoningEffort?: 'high' | 'max'
|
||||
/** Positive context capacity used when the selected model has no exact value. */
|
||||
defaultContextWindow?: number
|
||||
/** Advisory models shown by discovery consumers; defaults to V4 Flash and V4 Pro. */
|
||||
models?: DeepSeekCatalogModel[]
|
||||
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
|
||||
@@ -58,6 +60,7 @@ export const Config: z<Config> = z.object({
|
||||
baseURL: z.string(),
|
||||
thinking: z.union(['enabled', 'disabled']),
|
||||
reasoningEffort: z.union(['high', 'max']),
|
||||
defaultContextWindow: z.number().step(1).min(1),
|
||||
models: z.array(catalogModel).default(DEFAULT_MODELS),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
})
|
||||
@@ -103,6 +106,9 @@ export function apply(ctx: Context, config: Config): void {
|
||||
thinking: config.thinking,
|
||||
reasoningEffort: config.reasoningEffort,
|
||||
},
|
||||
...config.defaultContextWindow === undefined
|
||||
? {}
|
||||
: { defaultContextWindow: config.defaultContextWindow },
|
||||
models: resolveModels(config.models),
|
||||
streamIdleTimeoutMs: config.streamIdleTimeoutMs ?? DEFAULT_STREAM_IDLE_TIMEOUT_MS,
|
||||
}))
|
||||
|
||||
@@ -571,6 +571,27 @@ describe('plugin registration and config', () => {
|
||||
.resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('uses exact model capacity before the adapter-wide default', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
defaultContextWindow: 256_000,
|
||||
models: [
|
||||
{ id: 'inherits-default' },
|
||||
{ id: 'exact-override', contextWindow: 64_000 },
|
||||
],
|
||||
})
|
||||
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'inherits-default'))
|
||||
.resolves.toEqual({ contextWindow: 256_000 })
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'exact-override'))
|
||||
.resolves.toEqual({ contextWindow: 64_000 })
|
||||
await expect(ctx.llm.resolveModelContext('deepseek', 'unlisted-pass-through'))
|
||||
.resolves.toEqual({ contextWindow: 256_000 })
|
||||
})
|
||||
|
||||
it('allows an explicit empty model catalog', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
@@ -612,6 +633,26 @@ describe('plugin registration and config', () => {
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
})
|
||||
|
||||
it.each([0, 1.5])(
|
||||
'rejects invalid adapter-wide default context capacity %s',
|
||||
async (defaultContextWindow) => {
|
||||
expect(() => new DeepSeekAdapter({
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
defaultContextWindow,
|
||||
})).toThrow(/defaultContextWindow must be a positive integer/)
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: 'http://127.0.0.1:1',
|
||||
defaultContextWindow,
|
||||
})).rejects.toThrow(/defaultContextWindow/)
|
||||
expect(ctx.llm.listProviders()).toEqual([])
|
||||
},
|
||||
)
|
||||
|
||||
it('falls back to DEEPSEEK_API_KEY and DEEPSEEK_BASE_URL env vars', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'env-key')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', 'http://127.0.0.1:1')
|
||||
|
||||
Reference in New Issue
Block a user