Merge remote-tracking branch 'origin/master' into worktree/routed-model-compaction-policy
# Conflicts: # .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.i18n.yaml # .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md # .agents/notes/implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md # docs/cordis-catalog/events.md # docs/cordis-catalog/services.md # docs/event-producer-consumer.md # examples/headless-agent/tests/harness.ts # examples/repl-agent/cordis.yml # packages/compact/compact-basic/README.md # packages/compact/compact-basic/src/index.ts # packages/compact/compact-basic/tests/compact-basic.spec.ts # packages/compact/compact-basic/tests/loader-composition.spec.ts # packages/cordis/tool-cordis/src/api-catalog.ts # packages/llm/README.md # packages/llm/llm-deepseek/src/adapter.ts # packages/llm/llm-pi-ai/src/adapter.ts # packages/llm/llm/README.md # packages/llm/llm/src/index.ts # scripts/gen-cordis-catalog.ts # website/zh-CN/api/harness/events.md # website/zh-CN/api/harness/llm.md # website/zh-CN/api/harness/token-meter.md # website/zh-CN/guide/config.md
This commit is contained in:
851 files changed
+32175
-13361
No files matched your search
@@ -10,13 +10,14 @@ This backend owns the compaction policy:
|
||||
|
||||
- **Measurement** — the singleton `ctx.tokenMeter` prices the latest canonical logged envelope and current surface at one consumed-log revision. Post-step pressure therefore includes the actual system prompt, tools, prefix, routing, assistant completion, tool results, buffered context, and steering.
|
||||
- **Routed policy** — proactive pressure resolves capacity from the adapter that owns the latest durable provider/model route, then scales the default policy plus an optional exact-target override into concrete token budgets. Model discovery remains advisory and is not consulted.
|
||||
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope.
|
||||
- **Model-free pruning** — after pressure or canonical overflow qualifies, the optional [`ctx.toolResultPrune`](../compact-tool-result-prune/README.md) service rewrites oversized tool results before range selection. Compact-basic remeasures through `ctx.tokenMeter`, skips summarization when pressure becomes safe, and otherwise summarizes the pruned surface. Below-pressure post-step checks never prune.
|
||||
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes. The optional pruner can repair an oversized closed tool unit when its text-bearing result is the removable bulk; indivisible non-tool units and non-prunable tool remainders remain out of scope.
|
||||
- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
|
||||
- **Summarization** — a direct `llm/stream` call uses the configured provider/model pair and cap, falling back to the latest logged request target and then the agent target, without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
|
||||
- **Framing** — the replacement user message marks established checkpoint context with `<compacted-summary>` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint.
|
||||
- **Lifecycle** — `compactRegion()` mutates `agent.session` and records its start, summary, replacement, and end. The serial `agent/post-step` listener checks pressure after successful output and tool work are durable but before `step/end`. Canonical provider overflow is handled through `agent/request-error` after the failed step closes.
|
||||
- **Overflow recovery** — provider-confirmed overflow does not require capacity metadata: it bypasses normal pressure and retention and attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized only when `surface.replaceGeneration` advances; no range, no replacement, recovery failure, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
|
||||
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. A region failure records an error end and leaves the surface unchanged. Operational post-step failures warn and continue; overflow-recovery failure preserves the original provider error.
|
||||
- **Overflow recovery** — provider-confirmed overflow needs no capacity metadata: it bypasses normal pressure and retention, prunes, then attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized whenever `surface.replaceGeneration` advances, including when pruning lands before later summary work throws. No replacement, an exhausted target-specific cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
|
||||
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no summary replacement landed. A region failure records an error end; the surface remains unchanged unless pruning already landed. Operational post-step failures warn and continue, while overflow-recovery failure preserves the original provider error only when no earlier replacement advanced the surface. Cancellation remains authoritative after any progress.
|
||||
|
||||
The protected `summarize()` method is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on `ctx.tokenMeter`. The hook returns the summary blocks together with the call envelope it used (`{ summary, provider, model, maxTokens? }`), which is logged on `compact/summary`.
|
||||
|
||||
@@ -57,7 +58,7 @@ export function apply(ctx: Context): void {
|
||||
}
|
||||
```
|
||||
|
||||
Loading the plugin registers `ctx.compact`. With `auto: true` (the default) it compacts automatically under token pressure; a consumer (a future `/compact` tool) can also call `ctx.compact.compactIfNeeded(...)` or `ctx.compact.compactRegion(...)` directly.
|
||||
Loading the plugin registers `ctx.compact`. Add [`dsh-compact-tool-result-prune`](../compact-tool-result-prune/README.md) as a sibling before this plugin to enable the optional model-free pass. With `auto: true` (the default) it compacts automatically under token pressure; a consumer (a future `/compact` tool) can also call `ctx.compact.compactIfNeeded(...)` or `ctx.compact.compactRegion(...)` directly.
|
||||
|
||||
For example, the same compact plugin can safely serve models with different capacities and one target-specific policy:
|
||||
|
||||
@@ -79,7 +80,7 @@ For example, the same compact plugin can safely serve models with different capa
|
||||
|
||||
#### What the model sees
|
||||
|
||||
After a successful step crosses the threshold, the next request receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. Overflow recovery rebuilds the immediate retry from that replacement. This one checkpoint replaces the selected older range and is followed by the retained recent units.
|
||||
After a successful step crosses the threshold, oversized tool results are first rewritten when the optional pruner is loaded. If summarization remains necessary, the next request receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. Overflow recovery rebuilds the immediate retry from whatever replacement advanced the surface. A checkpoint replaces the selected older range and is followed by the retained recent units.
|
||||
|
||||
##### Conversation checkpoint preamble
|
||||
|
||||
@@ -89,7 +90,7 @@ This is an automatically generated checkpoint condensing an earlier span of the
|
||||
|
||||
#### Token effect
|
||||
|
||||
The replacement reduces future input history rather than appending a second copy. The summary remains until a later compaction replaces it; one oversized indivisible unit can still exceed the budget.
|
||||
Model-free pruning can avoid the auxiliary call entirely; otherwise it reduces that call's transcript before the summary replaces an older range. The replacement reduces future input history rather than appending a second copy. A summary remains until a later compaction replaces it, while an indivisible non-tool unit can still exceed the budget.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
@@ -165,7 +166,7 @@ Prefix-stable for auxiliary calls while this instruction and the summarizer rout
|
||||
|
||||
- **Meter accuracy follows the fixed heuristic** — missing reusable provider usage falls back to character count plus structural overhead rather than exact tokenization.
|
||||
- **Overflow classification is adapter-maintained** — provider wording can change; both DeepSeek adapters normalize currently recognized context-limit failures to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- **Single-unit and envelope-only overflow remain outside surface compaction** — recovery cannot split one indivisible message/tool unit or shrink system/tools/prefix.
|
||||
- **Some indivisible-unit and envelope-only overflow remains outside surface compaction** — recovery cannot shrink system/tools/prefix, split an indivisible non-tool node, or repair a tool unit whose non-prunable remainder still exceeds the window. The optional pruner can shrink text-bearing tool-result bulk inside an otherwise indivisible pair.
|
||||
- **`compactRegion` requires an open turn** — a manual call on a fully-closed session throws ("no open turn") rather than compacting.
|
||||
- **Summarization failure fails closed with full, over-budget history** — including truncation at the summarization `maxTokens`, which hidden reasoning tokens can consume; the auto path logs a warning and proceeds.
|
||||
- **Summarization failure preserves the latest durable surface** — before any replacement, the auto path logs a warning and proceeds with full over-budget history. If pruning already landed, a later summarization failure proceeds from that durable pruned surface. Summarization truncation at `maxTokens`, which hidden reasoning tokens can consume, follows the same rule.
|
||||
- **The summarization call has no transcript-snapshot coverage** — `dsh-llm-replay` derives calls from `assistant/chunk` events, so this chunk-less direct `ctx.llm.stream()` call cannot replay (named deferred replay infrastructure in [the seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-18-compaction-capability-seam.md)).
|
||||
@@ -27,8 +27,14 @@
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-token-meter": "^0.0.1",
|
||||
"@deepseek-ai/dsh-compact-tool-result-prune": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@deepseek-ai/dsh-compact-tool-result-prune": {
|
||||
"optional": true
|
||||
}
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
@@ -41,8 +47,10 @@
|
||||
"@deepseek-ai/dsh-compact": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm-retry": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-token-meter": "workspace:^",
|
||||
"@deepseek-ai/dsh-compact-tool-result-prune": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
|
||||
@@ -12,6 +12,8 @@ import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import { CONTEXT_WINDOW_EXCEEDED_CODE, assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, LlmCallConfig } from '@deepseek-ai/dsh-llm'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
// Type-only: makes the optional sibling service available to `ctx.get()`.
|
||||
import type {} from '@deepseek-ai/dsh-compact-tool-result-prune'
|
||||
import {
|
||||
resolveCompactSpec,
|
||||
resolveConfig,
|
||||
@@ -150,29 +152,54 @@ export class BasicCompactService extends CompactService {
|
||||
}
|
||||
})
|
||||
|
||||
ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => {
|
||||
if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE || signal.aborted) return next()
|
||||
ctx.on('agent/request-error', async (
|
||||
agent,
|
||||
_turn,
|
||||
_step,
|
||||
_error,
|
||||
failure,
|
||||
priorFailures,
|
||||
signal,
|
||||
next,
|
||||
) => {
|
||||
const priorOverflowFailures = priorFailures.filter(
|
||||
item => item.code === CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
).length
|
||||
if (failure.code !== CONTEXT_WINDOW_EXCEEDED_CODE || signal.aborted) return next()
|
||||
const target = routedTarget(agent.session)
|
||||
if (target === undefined) return next()
|
||||
const policy = resolveTargetPolicy(this.config, target)
|
||||
if (retryAttempt >= policy.maxOverflowRetries) return next()
|
||||
if (priorOverflowFailures >= policy.maxOverflowRetries) return next()
|
||||
|
||||
let generation: number
|
||||
const generation = agent.session.surface.replaceGeneration
|
||||
let result: CompactionResult | null
|
||||
try {
|
||||
generation = agent.session.surface.replaceGeneration
|
||||
result = await this.compactIfNeeded(agent, 'context-overflow', signal)
|
||||
} catch (recoveryError: unknown) {
|
||||
const message = recoveryError instanceof Error ? recoveryError.message : String(recoveryError)
|
||||
// A model-free prune can land before later summary work fails. That
|
||||
// durable reduction is sufficient retry proof; do not discard it just
|
||||
// because the optional second phase threw. Cancellation still wins.
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while recovery is awaited.
|
||||
if (!signal.aborted && agent.session.surface.replaceGeneration > generation) {
|
||||
ctx.logger.warn(
|
||||
`context-overflow compaction failed after durable surface progress: ${message}; `
|
||||
+ 'retrying from the replacement surface',
|
||||
)
|
||||
return { action: 'retry' }
|
||||
}
|
||||
ctx.logger.warn(
|
||||
`context-overflow compaction failed: ${message}; preserving the original request error`,
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while recovery is awaited.
|
||||
`context-overflow compaction failed: ${message}; ${signal.aborted
|
||||
? 'cancellation prevents retry'
|
||||
: 'preserving the original request error'}`,
|
||||
)
|
||||
return next()
|
||||
}
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited.
|
||||
if (signal.aborted || result === null
|
||||
if (signal.aborted
|
||||
|| agent.session.surface.replaceGeneration <= generation) return next()
|
||||
logResult(result, 'context overflow recovery')
|
||||
if (result !== null) logResult(result, 'context overflow recovery')
|
||||
return { action: 'retry' }
|
||||
})
|
||||
}
|
||||
@@ -205,7 +232,7 @@ export class BasicCompactService extends CompactService {
|
||||
* @param agent - agent whose latest durable routed request is measured.
|
||||
* @param trigger - normal post-step pressure or context-overflow recovery.
|
||||
* @param signal - live turn cancellation signal forwarded to summarization.
|
||||
* @returns the latest compaction result, or `null` when no check/work applies.
|
||||
* @returns the latest summary compaction result, or `null` when no summary ran.
|
||||
*/
|
||||
override async compactIfNeeded(
|
||||
agent: Agent,
|
||||
@@ -216,13 +243,10 @@ export class BasicCompactService extends CompactService {
|
||||
if (target === undefined) return null
|
||||
const policy = resolveTargetPolicy(this.config, target)
|
||||
const meter = this.ctx.tokenMeter
|
||||
let measurement = meter.measure(agent.session)
|
||||
switch (trigger) {
|
||||
case 'context-overflow': {
|
||||
const measurement = meter.measure(agent.session)
|
||||
const range = selectCompactableRange(agent.session, measurement, 0)
|
||||
if (range === null) return null
|
||||
return this.compactRegion(range.start, range.end, agent, signal)
|
||||
}
|
||||
case 'context-overflow':
|
||||
break
|
||||
case 'pressure':
|
||||
break
|
||||
/* v8 ignore next -- closed-union exhaustiveness guard */
|
||||
@@ -230,6 +254,21 @@ export class BasicCompactService extends CompactService {
|
||||
assertNever(trigger, 'compaction trigger')
|
||||
}
|
||||
|
||||
// Pruning is optional so compact-basic remains independently composable.
|
||||
// Overflow always qualifies; pressure first resolves the routed model's
|
||||
// capacity and checks its target-specific threshold.
|
||||
const prune = this.ctx.get('toolResultPrune')
|
||||
|
||||
if (trigger === 'context-overflow') {
|
||||
if (prune !== undefined) {
|
||||
prune.pruneSession(agent.session)
|
||||
measurement = meter.measure(agent.session)
|
||||
}
|
||||
const range = selectCompactableRange(agent.session, measurement, 0)
|
||||
if (range === null) return null
|
||||
return this.compactRegion(range.start, range.end, agent, signal)
|
||||
}
|
||||
|
||||
const context = await this.ctx.llm.resolveModelContext(target.provider, target.model)
|
||||
const targetKey = `${target.provider}/${target.model}`
|
||||
if (context === undefined) {
|
||||
@@ -240,7 +279,14 @@ export class BasicCompactService extends CompactService {
|
||||
)
|
||||
}
|
||||
const spec = resolveCompactSpec(policy, context.contextWindow)
|
||||
let measurement = meter.measure(agent.session)
|
||||
if (measurement.totalTokens < spec.thresholdTokens) return null
|
||||
|
||||
// Once pressure qualifies, land the model-free pass before choosing a
|
||||
// summary range, then remeasure through the singleton replay fold.
|
||||
if (prune !== undefined) {
|
||||
prune.pruneSession(agent.session)
|
||||
measurement = meter.measure(agent.session)
|
||||
}
|
||||
if (measurement.totalTokens < spec.thresholdTokens) return null
|
||||
|
||||
let result: CompactionResult | null = null
|
||||
|
||||
@@ -146,14 +146,10 @@ export function frameSummary(summary: readonly ContentBlock[]): ContentBlock[] {
|
||||
/** Map a terminal summarization finish to its fail-closed error. */
|
||||
function finishError(finish: FinishReason): Error | undefined {
|
||||
switch (finish.kind) {
|
||||
case 'error': {
|
||||
const error = new Error(finish.message) as Error & { code?: string }
|
||||
if (finish.code !== undefined) error.code = finish.code
|
||||
return error
|
||||
}
|
||||
case 'error':
|
||||
case 'aborted': {
|
||||
const error = new Error('summarization stream aborted') as Error & { code?: string }
|
||||
error.code = 'ABORTED'
|
||||
const error = new Error(finish.failure.message) as Error & { code?: string }
|
||||
error.code = finish.failure.code
|
||||
return error
|
||||
}
|
||||
case 'max-tokens': {
|
||||
|
||||
@@ -11,9 +11,16 @@ import {
|
||||
} from '@deepseek-ai/dsh-compact-basic/src/config.ts'
|
||||
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
|
||||
import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, LlmModelContext, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
ContentBlock,
|
||||
GenerateOptions,
|
||||
LlmFailure,
|
||||
LlmModelContext,
|
||||
StreamChunk,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import ToolResultPruneService from '@deepseek-ai/dsh-compact-tool-result-prune'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
const SIGNAL = new AbortController().signal
|
||||
@@ -132,6 +139,43 @@ function toolConversation(): Session {
|
||||
return session
|
||||
}
|
||||
|
||||
/** One closed routed tool step followed by an open turn for rewrite events. */
|
||||
function oversizedToolResult(chars = 3_000, withCompactablePrompt = false): Session {
|
||||
const session = new Session(SessionId(`oversized-tool-${chars}`))
|
||||
const callId = CallId('oversized')
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
if (withCompactablePrompt) {
|
||||
session.append('user/message', {
|
||||
content: [{ type: 'text', text: 'older history '.repeat(200) }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
}
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL } },
|
||||
reason: 'initial',
|
||||
})
|
||||
session.append('assistant/message', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
content: [{ type: 'tool-call', id: callId, name: 'bash', arguments: '{}' }],
|
||||
provenance: { provider: MODEL, model: MODEL },
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('tool/call', { turn: 1, step: 1, callId, name: 'bash', arguments: '{}' })
|
||||
session.append('tool/result', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
callId,
|
||||
content: [{ type: 'text', text: 'X'.repeat(chars) }],
|
||||
isError: false,
|
||||
meta: { presentation: 'preserved' },
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
return session
|
||||
}
|
||||
|
||||
class TestCompactService extends BasicCompactService {
|
||||
summary: ContentBlock[] = [{ type: 'text', text: 'small checkpoint' }]
|
||||
summaryProvider = 'summary-provider'
|
||||
@@ -579,6 +623,78 @@ describe('pressure measurement and retention', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('optional model-free tool-result pruning', () => {
|
||||
const pruneConfig = { thresholdChars: 100, headChars: 20, tailChars: 10 }
|
||||
|
||||
it('does not prune a below-pressure session opportunistically', async () => {
|
||||
const ctx = createContext(10_000)
|
||||
const prune = new ToolResultPruneService(ctx, pruneConfig)
|
||||
const compact = new TestCompactService(ctx, {
|
||||
auto: false,
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 100,
|
||||
})
|
||||
const session = oversizedToolResult()
|
||||
const pruneSession = vi.spyOn(prune, 'pruneSession')
|
||||
|
||||
expect(await compactIfNeeded(compact, session)).toBeNull()
|
||||
expect(pruneSession).not.toHaveBeenCalled()
|
||||
expect(compact.calls).toHaveLength(0)
|
||||
expect(session.surface.replaceGeneration).toBe(0)
|
||||
})
|
||||
|
||||
it('skips LLM summarization when pruning alone clears pressure', async () => {
|
||||
const ctx = createContext(1_000)
|
||||
void new ToolResultPruneService(ctx, pruneConfig)
|
||||
const compact = new TestCompactService(ctx, {
|
||||
auto: false,
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 50,
|
||||
})
|
||||
const session = oversizedToolResult()
|
||||
|
||||
expect(ctx.tokenMeter.measure(session).totalTokens).toBeGreaterThanOrEqual(500)
|
||||
expect(await compactIfNeeded(compact, session)).toBeNull()
|
||||
expect(ctx.tokenMeter.measure(session).totalTokens).toBeLessThan(500)
|
||||
expect(compact.calls).toHaveLength(0)
|
||||
expect(session.surface.replaceGeneration).toBe(1)
|
||||
})
|
||||
|
||||
it('summarizes the pruned surface when pruning is insufficient', async () => {
|
||||
const ctx = createContext(2_000)
|
||||
void new ToolResultPruneService(ctx, pruneConfig)
|
||||
const compact = new TestCompactService(ctx, {
|
||||
auto: false,
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 50,
|
||||
})
|
||||
const session = toolConversation()
|
||||
|
||||
expect(await compactIfNeeded(compact, session)).not.toBeNull()
|
||||
expect(compact.calls).toHaveLength(1)
|
||||
expect(compact.calls[0]!.text).toContain('tool result middle pruned')
|
||||
expect(compact.calls[0]!.text).not.toContain('result 1 '.repeat(300))
|
||||
})
|
||||
|
||||
it('retains the original compact-basic behavior without the optional plugin', async () => {
|
||||
const ctx = createContext(2_000)
|
||||
const compact = new TestCompactService(ctx, {
|
||||
auto: false,
|
||||
thresholdRatio: 0.5,
|
||||
retainTokens: 50,
|
||||
})
|
||||
const session = oversizedToolResult(3_000, true)
|
||||
|
||||
expect(await compactIfNeeded(compact, session)).not.toBeNull()
|
||||
expect(compact.calls).toHaveLength(1)
|
||||
const original = session.events.find(event => event.type === 'tool/result')
|
||||
expect(original?.type === 'tool/result' && original.data.content[0])
|
||||
.toEqual({ type: 'text', text: 'X'.repeat(3_000) })
|
||||
expect(session.events.filter(event =>
|
||||
event.type === 'tool/result' && event.surfaceOp !== 'append')).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('compaction region transaction', () => {
|
||||
it('lands a framed, replayable checkpoint with exact pricing provenance', async () => {
|
||||
const compact = service()
|
||||
@@ -950,9 +1066,9 @@ describe('default one-shot summarizer', () => {
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ kind: 'error', message: 'provider failed', code: 'PROVIDER' }, 'PROVIDER', /provider failed/],
|
||||
[{ kind: 'error', message: 'opaque' }, undefined, /opaque/],
|
||||
[{ kind: 'aborted' }, 'ABORTED', /aborted/],
|
||||
[{ kind: 'error', failure: { message: 'provider failed', code: 'PROVIDER' } }, 'PROVIDER', /provider failed/],
|
||||
[{ kind: 'error', failure: { message: 'opaque', code: 'UNKNOWN' } }, 'UNKNOWN', /opaque/],
|
||||
[{ kind: 'aborted', failure: { message: 'summarization aborted', code: 'ABORTED' } }, 'ABORTED', /aborted/],
|
||||
[{ kind: 'max-tokens' }, 'MAX_TOKENS', /token cap/],
|
||||
] as Array<[(StreamChunk & { type: 'finish' })['reason'], string | undefined, RegExp]>) (
|
||||
'rejects terminal finish %#',
|
||||
@@ -990,7 +1106,9 @@ describe('automatic listener and loader composition', () => {
|
||||
signal = SIGNAL,
|
||||
next: () => Promise<{ action: 'fail' | 'retry' }> = () => Promise.resolve({ action: 'fail' }),
|
||||
): Promise<{ action: 'fail' | 'retry' }> {
|
||||
return ctx.waterfall('agent/request-error', owner, 1, 1, error, retryAttempt, signal, next)
|
||||
const failure: LlmFailure = { message: error.message, code: error.code ?? 'UNKNOWN' }
|
||||
const priorFailures = Object.freeze(Array.from({ length: retryAttempt }, () => failure))
|
||||
return ctx.waterfall('agent/request-error', owner, 1, 1, error, failure, priorFailures, signal, next)
|
||||
}
|
||||
|
||||
function overflow(message = 'provider overflow'): Error & { code: string } {
|
||||
@@ -1101,6 +1219,89 @@ describe('automatic listener and loader composition', () => {
|
||||
expect(session.surface.nodes).toContain(retainedSeq)
|
||||
})
|
||||
|
||||
it('authorizes overflow retry when pruning alone advances an indivisible surface', async () => {
|
||||
const ctx = createContext(10_000)
|
||||
void new ToolResultPruneService(ctx, {
|
||||
thresholdChars: 100,
|
||||
headChars: 20,
|
||||
tailChars: 10,
|
||||
})
|
||||
const compact = new TestCompactService(ctx, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 900,
|
||||
})
|
||||
const session = oversizedToolResult()
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
|
||||
expect(session.surface.replaceGeneration).toBe(1)
|
||||
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
|
||||
expect(compact.calls).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('continues overflow recovery with summarization on the pruned surface', async () => {
|
||||
const ctx = createContext(10_000)
|
||||
void new ToolResultPruneService(ctx, {
|
||||
thresholdChars: 100,
|
||||
headChars: 20,
|
||||
tailChars: 10,
|
||||
})
|
||||
const compact = new TestCompactService(ctx, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 900,
|
||||
})
|
||||
const session = toolConversation()
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
|
||||
expect(session.events.some(event => event.type === 'compact/summary')).toBe(true)
|
||||
expect(compact.calls).toHaveLength(1)
|
||||
expect(compact.calls[0]!.text).toContain('tool result middle pruned')
|
||||
})
|
||||
|
||||
it('retries from a durable prune when later overflow summarization throws', async () => {
|
||||
const ctx = createContext(10_000)
|
||||
const warnings: string[] = []
|
||||
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
|
||||
void new ToolResultPruneService(ctx, {
|
||||
thresholdChars: 100,
|
||||
headChars: 20,
|
||||
tailChars: 10,
|
||||
})
|
||||
const compact = new TestCompactService(ctx, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 900,
|
||||
})
|
||||
compact.error = new Error('summary unavailable after prune')
|
||||
const session = oversizedToolResult(3_000, true)
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
|
||||
expect(session.surface.replaceGeneration).toBe(1)
|
||||
expect(session.events.filter(event => event.type === 'tool/result')).toHaveLength(2)
|
||||
expect(session.events.findLast(event => event.type === 'compact/end')?.data)
|
||||
.toMatchObject({ error: 'summary unavailable after prune' })
|
||||
expect(warnings).toContainEqual(expect.stringContaining('retrying from the replacement surface'))
|
||||
})
|
||||
|
||||
it('lets cancellation win when summary throws after a durable prune', async () => {
|
||||
const ctx = createContext(10_000)
|
||||
const controller = new AbortController()
|
||||
void new ToolResultPruneService(ctx, {
|
||||
thresholdChars: 100,
|
||||
headChars: 20,
|
||||
tailChars: 10,
|
||||
})
|
||||
const compact = new TestCompactService(ctx, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 900,
|
||||
})
|
||||
compact.mutateDuringSummary = () => { controller.abort('cancelled during summary') }
|
||||
compact.error = new Error('summary cancelled after prune')
|
||||
const session = oversizedToolResult(3_000, true)
|
||||
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow(), 0, controller.signal))
|
||||
.toEqual({ action: 'fail' })
|
||||
expect(session.surface.replaceGeneration).toBe(1)
|
||||
})
|
||||
|
||||
it('preserves the newest whole tool-call/result pair during forced overflow compaction', async () => {
|
||||
const ctx = createContext()
|
||||
void new TestCompactService(ctx, {
|
||||
@@ -1241,6 +1442,23 @@ describe('automatic listener and loader composition', () => {
|
||||
expect(compactSpy).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('applies the routed model override to the overflow retry cap', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx, {
|
||||
maxOverflowRetries: 2,
|
||||
modelPolicies: [{
|
||||
provider: MODEL,
|
||||
model: MODEL,
|
||||
maxOverflowRetries: 1,
|
||||
}],
|
||||
})
|
||||
const compactSpy = vi.spyOn(compact, 'compactIfNeeded')
|
||||
|
||||
expect(await recover(ctx, agent(conversation(3), MODEL), overflow(), 1))
|
||||
.toEqual({ action: 'fail' })
|
||||
expect(compactSpy).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does not retry when cancellation lands during an awaited compaction', async () => {
|
||||
const ctx = createContext()
|
||||
const compact = new TestCompactService(ctx)
|
||||
|
||||
@@ -11,6 +11,7 @@ import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-test
|
||||
import * as Invariants from '@deepseek-ai/dsh-invariants'
|
||||
import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import * as LlmRetry from '@deepseek-ai/dsh-llm-retry'
|
||||
import { SessionId, type SurfaceEvent } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
@@ -65,7 +66,10 @@ class OverflowRecoveryAdapter extends LlmAdapter {
|
||||
readonly conversationRequests: GenerateOptions[] = []
|
||||
readonly summaryRequests: GenerateOptions[] = []
|
||||
|
||||
constructor(private readonly delivery: 'thrown' | 'in-band') {
|
||||
constructor(
|
||||
private readonly delivery: 'thrown' | 'in-band',
|
||||
private readonly transientAfterOverflow = false,
|
||||
) {
|
||||
super()
|
||||
}
|
||||
|
||||
@@ -91,12 +95,17 @@ class OverflowRecoveryAdapter extends LlmAdapter {
|
||||
type: 'finish',
|
||||
reason: {
|
||||
kind: 'error',
|
||||
message: 'request too large for model context',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
failure: {
|
||||
message: 'request too large for model context',
|
||||
code: CONTEXT_WINDOW_EXCEEDED_CODE,
|
||||
},
|
||||
},
|
||||
}
|
||||
return
|
||||
}
|
||||
if (this.transientAfterOverflow && this.conversationRequests.length === 2) {
|
||||
throw new LlmError('temporary provider outage', 'SERVER')
|
||||
}
|
||||
yield { type: 'block-start', index: 0, blockType: 'text' }
|
||||
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } }
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
@@ -142,6 +151,29 @@ function waitForIdle(ctx: Context, agent: Agent): Promise<void> {
|
||||
})
|
||||
}
|
||||
|
||||
function seedOverflowHistory(agent: Agent): void {
|
||||
for (let turn = 1; turn <= 2; turn += 1) {
|
||||
const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY'
|
||||
agent.session.append('turn/start', {
|
||||
turn,
|
||||
trigger: { kind: 'message', source: { kind: 'user' } },
|
||||
})
|
||||
agent.session.append('user/message', {
|
||||
content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
agent.session.append('step/start', { turn, step: 1 })
|
||||
agent.session.append('assistant/message', {
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
turn,
|
||||
step: 1,
|
||||
content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }],
|
||||
}, { surfaceOp: 'append' })
|
||||
agent.session.append('step/end', { turn, step: 1 })
|
||||
agent.session.append('turn/end', { turn, reason: { kind: 'completed' } })
|
||||
}
|
||||
}
|
||||
|
||||
describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () => {
|
||||
it('uses the model actually routed by agent/request for post-step pressure', async () => {
|
||||
const { ctx } = await harness(8)
|
||||
@@ -249,26 +281,7 @@ describe('context-overflow recovery across the real loop and compact-basic', ()
|
||||
provider: 'unconfigured-agent-fallback',
|
||||
model: 'unconfigured-agent-fallback',
|
||||
})
|
||||
for (let turn = 1; turn <= 2; turn += 1) {
|
||||
const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY'
|
||||
agent.session.append('turn/start', {
|
||||
turn,
|
||||
trigger: { kind: 'message', source: { kind: 'user' } },
|
||||
})
|
||||
agent.session.append('user/message', {
|
||||
content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }],
|
||||
source: { kind: 'user' },
|
||||
}, { surfaceOp: 'append' })
|
||||
agent.session.append('step/start', { turn, step: 1 })
|
||||
agent.session.append('assistant/message', {
|
||||
provenance: { provider: 'mock', model: 'mock' },
|
||||
turn,
|
||||
step: 1,
|
||||
content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }],
|
||||
}, { surfaceOp: 'append' })
|
||||
agent.session.append('step/end', { turn, step: 1 })
|
||||
agent.session.append('turn/end', { turn, reason: { kind: 'completed' } })
|
||||
}
|
||||
seedOverflowHistory(agent)
|
||||
|
||||
agent.send([{ type: 'text', text: 'continue from history' }])
|
||||
await agent.whenIdle()
|
||||
@@ -307,4 +320,47 @@ describe('context-overflow recovery across the real loop and compact-basic', ()
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
it('keeps context-overflow and transient retry budgets independent in one sequence', async () => {
|
||||
const ctx = new Context()
|
||||
const adapter = new OverflowRecoveryAdapter('thrown', true)
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await ctx.plugin(Invariants)
|
||||
await ctx.plugin(LlmRetry, {
|
||||
maxTransientRetries: 1,
|
||||
initialDelayMs: 1,
|
||||
maxDelayMs: 1,
|
||||
jitterRatio: 0,
|
||||
})
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(TokenMeterService)
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
await ctx.plugin(BasicCompactService, {
|
||||
thresholdRatio: 1,
|
||||
retainTokens: 100,
|
||||
maxTokens: 64,
|
||||
compactionRetries: 0,
|
||||
maxOverflowRetries: 1,
|
||||
})
|
||||
|
||||
try {
|
||||
const agent = ctx.agentLoop.create(SessionId('alternating-recovery'), { provider: 'mock', model: 'mock' })
|
||||
seedOverflowHistory(agent)
|
||||
agent.send([{ type: 'text', text: 'continue from history' }])
|
||||
await agent.whenIdle()
|
||||
|
||||
expect(adapter.conversationRequests).toHaveLength(3)
|
||||
expect(adapter.summaryRequests).toHaveLength(1)
|
||||
expect(agent.session.events.filter(event => event.type === 'llm/retry').map(event => event.data))
|
||||
.toEqual([expect.objectContaining({ step: 2, retry: 1, failure: { message: 'temporary provider outage', code: 'SERVER' } })])
|
||||
expect(agent.session.events.filter(event => event.type === 'step/start').slice(-3).map(event => event.data.step))
|
||||
.toEqual([1, 2, 3])
|
||||
expect(agent.session.events.at(-1)).toMatchObject({
|
||||
type: 'turn/end',
|
||||
data: { reason: { kind: 'completed' } },
|
||||
})
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -9,6 +9,7 @@ import Include from '@cordisjs/plugin-include'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import BasicCompactService from '@deepseek-ai/dsh-compact-basic'
|
||||
import ToolResultPruneService from '@deepseek-ai/dsh-compact-tool-result-prune'
|
||||
|
||||
let root: string | undefined
|
||||
let context: Context | undefined
|
||||
@@ -32,6 +33,7 @@ async function loadYaml(lines: readonly string[]): Promise<Context> {
|
||||
const modules = new Map<string, unknown>([
|
||||
['@deepseek-ai/dsh-llm', LlmService],
|
||||
['@deepseek-ai/dsh-token-meter', TokenMeterService],
|
||||
['@deepseek-ai/dsh-compact-tool-result-prune', ToolResultPruneService],
|
||||
['@deepseek-ai/dsh-compact-basic', BasicCompactService],
|
||||
])
|
||||
context.loader.internal = {
|
||||
@@ -50,10 +52,15 @@ async function loadYaml(lines: readonly string[]): Promise<Context> {
|
||||
}
|
||||
|
||||
describe('real Loader composition', () => {
|
||||
it('loads the flat token-meter and compact-basic YAML shape', async () => {
|
||||
it('loads the shipped token-meter, pruning, and compact-basic YAML order', async () => {
|
||||
const loaded = await loadYaml([
|
||||
"- name: '@deepseek-ai/dsh-llm'",
|
||||
"- name: '@deepseek-ai/dsh-token-meter'",
|
||||
"- name: '@deepseek-ai/dsh-compact-tool-result-prune'",
|
||||
' config:',
|
||||
' thresholdChars: 100',
|
||||
' headChars: 20',
|
||||
' tailChars: 10',
|
||||
"- name: '@deepseek-ai/dsh-compact-basic'",
|
||||
' config:',
|
||||
' thresholdRatio: 0.5',
|
||||
@@ -65,6 +72,7 @@ describe('real Loader composition', () => {
|
||||
.filter(entry => entry.fiber === undefined && !entry.disabled)
|
||||
.map(entry => entry.options.name)
|
||||
expect(unloaded).toEqual([])
|
||||
expect(loaded.get('toolResultPrune')).toBeInstanceOf(ToolResultPruneService)
|
||||
expect(loaded.get('compact')).toBeInstanceOf(BasicCompactService)
|
||||
expect((loaded.compact as BasicCompactService).config).toMatchObject({
|
||||
thresholdRatio: 0.5,
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
{ "path": "../../llm/token-meter" },
|
||||
{ "path": "../../core/session" },
|
||||
{ "path": "../../core/agent" },
|
||||
{ "path": "../compact" }
|
||||
{ "path": "../compact" },
|
||||
{ "path": "../compact-tool-result-prune" }
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user