simplify(llm): drop unconsumed adapter-change event and assembled call surfaces
The LLM service exposed three call surfaces (stream/streamBlocks/generate) but the only production consumer — the agent loop — uses stream() exclusively, feeding raw chunks through its own BlockAssembler for replay fidelity. Drop the speculative convenience surfaces and the registry-change event that no listener consumed, leaving stream() as the single model-call contract for both production and tests. - Remove LlmService.streamBlocks() and generate(), the llm/generate waterfall, and GenerateResult. - Remove the llm/adapter-change event (declaration + emits) and the listener-throw rollback ordering that existed only to protect it; keep the HMR rollback disposer. - Remove BlockAssembler.flushReady()/flushRemaining()/result() and the flushed cursor — the streaming-flush slice existed only for streamBlocks(). - Adapter tests drive a stream()+BlockAssembler helper (tests/assemble.ts) instead of generate(), exercising the same path production uses. - Land the AGENTS.md "RFCs are proposals, not golden truth" principle and move both RFCs proposed -> implemented. Implements: - docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md - docs/rfc/implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md
This commit is contained in:
Tianyi Cui
@@ -16,6 +16,12 @@ Before you preserve a behavior solely to keep a test green, ask: is this behavio
|
||||
|
||||
The worked example is [Drop the mutable session summary](docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md): an entire `SessionSummary` type, a `SessionPersistence.update()` method, a JSONL sidecar, and SQLite columns existed and were exercised by their own contract test — yet **nothing in production CONSUMED any of it, and `update()` had no production caller**. (The backends did *write* summary state — JSONL touched the sidecar after a durable append, SQLite bumped `updated_at` in the append transaction — but those writes fed only reads that nothing performed.) The tests documented the behavior perfectly; the behavior was dead. Deleting the behavior and its tests together removed ~400 lines and erased a durability divergence the next refactor would have had to model. (This is the test-tier echo of "verify the world, not a synthetic stand-in" in § Defensive patterns: a test agrees with whatever it was written to assert; only a real consumer proves the behavior matters.)
|
||||
|
||||
## RFCs are proposals, not golden truth
|
||||
|
||||
The same discipline applies one level up, to the RFCs in `docs/rfc/`. A **proposed** RFC records an *intended* change argued at a point in time; it is not a contract to implement verbatim. The author reasoned from the code as they understood it then — and they can be wrong, or the code can have moved. So before implementing an RFC, **validate its premise against the current code first**: confirm the thing it wants removed or changed is actually dead/safe, and that the migration it proposes is genuinely cleaner than what exists.
|
||||
|
||||
When carrying out the change fights back — a removal forces an awkward migration, deletes machinery that turns out to be load-bearing, or pushes consumers onto a more brittle hand-rolled equivalent — treat that friction as **evidence the RFC over-reached**, not as work to push through. Keep, split, or amend the change to match what the code actually wants, and say so in the PR. An RFC that ships in amended form gets its text amended on the way to `implemented/`, so the landed RFC describes what actually shipped rather than the original guess. The discipline cuts both ways: an RFC is also not a reason to *avoid* a change a maintainer would otherwise make — it is one input, weighed against the code in front of you.
|
||||
|
||||
## Architecture
|
||||
|
||||
This codebase is based on the **Cordis** framework, built microkernel-style: **everything is a plugin**. All necessary Cordis dependencies are copied into this monorepo as vendored source (under `vendor/`) instead of being depended on via npm.
|
||||
|
||||
@@ -45,7 +45,7 @@ Dependency rule: plugins depend on interface packages, never on `dsh-agent-loop`
|
||||
|
||||
| ctx key | Class | Package | Role |
|
||||
|---|---|---|---|
|
||||
| `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` / `streamBlocks()` / `generate()` |
|
||||
| `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` |
|
||||
| `ctx.sessions` | `SessionStore` | dsh-session | creates/holds event-sourced `Session`s |
|
||||
| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list sessions |
|
||||
| `ctx.systemPrompt` | `SystemPrompt` | dsh-system-prompt | ordered sections + tool schemas → `assemble()` |
|
||||
|
||||
@@ -11,7 +11,7 @@ The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary
|
||||
|
||||
## Events
|
||||
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 24 events across 5 scopes.
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 22 events across 5 scopes.
|
||||
|
||||
### `agent/*`
|
||||
|
||||
@@ -185,28 +185,6 @@ Source: [`packages/core/agent/src/types.ts:166`](../../packages/core/agent/src/t
|
||||
|
||||
### `llm/*`
|
||||
|
||||
#### `llm/adapter-change` — emit
|
||||
|
||||
An adapter was registered or unregistered (the model→adapter map changed).
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/adapter-change'(): void
|
||||
```
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:43`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
#### `llm/generate` — waterfall
|
||||
|
||||
Waterfall around every non-streaming model call. Bound to the LlmService; call `next()` to delegate to the adapter.
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/generate'(this: LlmService, options: GenerateOptions, next: () => Promise<GenerateResult>): Promise<GenerateResult>
|
||||
```
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [GenerateResult](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:38`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
#### `llm/stream` — waterfall
|
||||
|
||||
Waterfall around every streaming model call (retry, caching, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit.
|
||||
@@ -217,7 +195,7 @@ Waterfall around every streaming model call (retry, caching, routing). Bound to
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:32`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:31`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
### `session/*`
|
||||
|
||||
@@ -369,13 +347,11 @@ The abstract `llm` service: an adapter registry plus streaming / non-streaming c
|
||||
registerAdapter(models: string[], adapter: LlmAdapter): () => void
|
||||
models(): string[]
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
async * streamBlocks(options: GenerateOptions): AsyncIterable<ContentBlock>
|
||||
generate(options: GenerateOptions): Promise<GenerateResult>
|
||||
```
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md) · [GenerateResult](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:81`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:69`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
### `ctx.sessionPersistence` — `SessionPersistence` (abstract seam)
|
||||
|
||||
|
||||
@@ -112,9 +112,9 @@ Adapters emit a raw **chunk** protocol; the loop logs the chunks (replay fidelit
|
||||
|
||||
The full union, the adapter contract (usage-before-finish, raw-JSON tool arguments, the two sanctioned error paths), and `BlockAssembler` live on **[llm-streaming.md](llm-streaming.md)**.
|
||||
|
||||
## The model request and result
|
||||
## The model request
|
||||
|
||||
One model call is a fully-assembled `GenerateOptions`; the non-streaming result is `GenerateResult`.
|
||||
One model call is a fully-assembled `GenerateOptions`. The adapter answers with a raw `StreamChunk` stream; the consumer assembles it with `BlockAssembler` (see [llm-streaming.md](llm-streaming.md)).
|
||||
|
||||
Source: [`packages/llm/llm/src/types.ts`](../../packages/llm/llm/src/types.ts)
|
||||
|
||||
@@ -140,14 +140,6 @@ interface GenerateOptions {
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
interface GenerateResult {
|
||||
message: Message
|
||||
usage?: TokenUsage
|
||||
finish: FinishReason
|
||||
}
|
||||
```
|
||||
|
||||
Why a model response stopped is a merge-extensible reason:
|
||||
|
||||
```ts type-equiv
|
||||
|
||||
@@ -49,7 +49,7 @@ interface TokenUsage {
|
||||
|
||||
## The seam
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, register with `ctx.llm.registerAdapter(models, adapter)`. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()` / `streamBlocks()` / `generate()`) and the `llm/stream` waterfall are described in [architecture.md § The vocabulary](../architecture.md#the-vocabulary-dsh-llm).
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, register with `ctx.llm.registerAdapter(models, adapter)`. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()`) and the `llm/stream` waterfall are described in [architecture.md § The vocabulary](../architecture.md#the-vocabulary-dsh-llm).
|
||||
|
||||
`ContentBlockType` (the key set the `index`-correlated blocks carry) derives from `ContentBlockMap`:
|
||||
|
||||
|
||||
+2
-2
@@ -52,8 +52,6 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Unify the agent id and the session id](proposed/simplification/2026-06-20-unify-agent-and-session-id.md) | 2026-06-20 |
|
||||
| [Stop mirroring durable boundaries as agent events](proposed/simplification/2026-06-20-remove-agent-boundary-mirror-events.md) | 2026-06-20 |
|
||||
| [Keep one public stop primitive](proposed/simplification/2026-06-20-public-agent-stop-surface.md) | 2026-06-20 |
|
||||
| [Drop unconsumed assembled LLM convenience surfaces](proposed/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md) | 2026-06-20 |
|
||||
| [Drop the unconsumed `llm/adapter-change` event](proposed/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md) | 2026-06-20 |
|
||||
| [Prune dead methods from the persistence and bash seams](proposed/simplification/2026-06-20-prune-dead-seam-methods.md) | 2026-06-20 |
|
||||
| [Fold trace-only session facts into load-bearing events](proposed/simplification/2026-06-20-collapse-trace-only-session-events.md) | 2026-06-20 |
|
||||
|
||||
@@ -96,6 +94,8 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| Title | First proposed |
|
||||
|---|---|
|
||||
| [Drop the mutable session summary](implemented/simplification/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 |
|
||||
| [Drop unconsumed assembled LLM convenience surfaces](implemented/simplification/2026-06-20-drop-unconsumed-llm-assembled-surfaces.md) | 2026-06-20 |
|
||||
| [Drop the unconsumed `llm/adapter-change` event](implemented/simplification/2026-06-20-drop-unconsumed-llm-adapter-change-event.md) | 2026-06-20 |
|
||||
|
||||
### Architecture
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ The product principle (see the 微内核Harness实现思路 design doc) is "ever
|
||||
|
||||
Pure Cordis event taxonomy. The loop's extension seams are typed events with deliberate dispatch modes:
|
||||
|
||||
- **waterfall** (around-middleware) where plugins mutate or veto: `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/execute`, `llm/stream`, `llm/generate`, `system-prompt/assemble`.
|
||||
- **waterfall** (around-middleware) where plugins mutate or veto: `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/execute`, `llm/stream`, `system-prompt/assemble`.
|
||||
- **emit** (sync fire-and-forget) for notifications: turn/step boundaries, stream chunks, lifecycle, errors.
|
||||
- **parallel** (awaited) for the one durability checkpoint: `session/flush`.
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
# RFC: Drop the unconsumed `llm/adapter-change` event
|
||||
|
||||
Status: proposed
|
||||
Status: implemented (proposed and accepted 2026-06-20)
|
||||
|
||||
## Problem
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
# RFC: Drop unconsumed assembled LLM convenience surfaces
|
||||
|
||||
Status: proposed
|
||||
Status: implemented (proposed and accepted 2026-06-20)
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -8,13 +8,13 @@ Status: implemented (proposed 2026-06-11, accepted 2026-06-14)
|
||||
|
||||
## Context
|
||||
|
||||
Example-based tests pin the cases we thought of. The harness's core is protocol-shaped — chunk streams, event logs, schema conversion, inbox scheduling — where the input space is combinatorial and the interesting bugs live in interleavings nobody wrote an example for. The motivating evidence: a `streamBlocks` ordering bug once survived 100% line coverage of the happy paths. Per-file 100% coverage proves every line ran, not that every interleaving is correct.
|
||||
Example-based tests pin the cases we thought of. The harness's core is protocol-shaped — chunk streams, event logs, schema conversion, inbox scheduling — where the input space is combinatorial and the interesting bugs live in interleavings nobody wrote an example for. The motivating evidence: a block-assembly ordering bug once survived 100% line coverage of the happy paths. Per-file 100% coverage proves every line ran, not that every interleaving is correct.
|
||||
|
||||
## Decision
|
||||
|
||||
Adopt `fast-check` (a root devDependency) with one `tests/properties.spec.ts` per protocol-shaped package, generators tuned for *realistic-but-adversarial* inputs (not uniform noise) and `numRuns` kept so the suite stays well under ~10s locally. Failures print a reproducible seed. (The original proposal also sketched a nightly CI job running 100× the iterations; that was not shipped — the property suite runs only in the normal `push`/`pull_request` CI, and a scheduled high-iteration job remains possible future work.)
|
||||
|
||||
- **dsh-llm / BlockAssembler:** arbitrary chunk streams (valid + malformed: duplicate indices, stragglers, missing block-start). Invariants: `flushReady()+flushRemaining() ≡ blocks()` in order; the streamed prefix is always a prefix of final `blocks()`; partial count ≤ distinct indices; re-assembly idempotent.
|
||||
- **dsh-llm / BlockAssembler:** arbitrary chunk streams (valid + malformed: duplicate indices, stragglers, missing block-start). Invariants: the blocks `push()` returns incrementally are a prefix of the final `blocks()`, in order; partial count ≤ distinct indices; re-assembly idempotent; streaming and one-shot consumers agree on usage and finish.
|
||||
- **dsh-session:** arbitrary event logs. Invariants: `deriveMessages` deterministic; replay-from-seed identical; seq strictly monotonic; non-message events never affect derived history; derived content is decoupled from the log.
|
||||
- **dsh-tools:** arbitrary `SchemaSpec`. Invariants: JSON Schema `required` equals the `required:true` keys at every level; conversion total; **and the composition with [runtime arg validation](../architecture/2026-06-11-runtime-arg-validation.md)** — generated args satisfying a spec pass `validateArgs`, and targeted corruptions (dropped required key, non-object top level) are rejected. This closes the validator/`InferArgs` drift risk.
|
||||
- **dsh-agent-loop:** arbitrary send schedules against a never-exhausting adapter, driven through the `agent/status` settle signal (no wall-clock sleeps). Invariants: no message lost; turn numbers strictly increase; status transitions stay on the legal machine.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId, ContentBlock, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import LlmService, { CallId, MessageSource, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { Session, SessionEvent, SessionId, TurnEndReason } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
@@ -446,90 +446,6 @@ describe('MEDIUM: turn numbering continues across seeded (forked) sessions', ()
|
||||
})
|
||||
})
|
||||
|
||||
describe('LOW: BlockAssembler and streamBlocks edge cases', () => {
|
||||
it('ignores deltas arriving after block-end for the same index (malformed stream)', async () => {
|
||||
const { BlockAssembler } = await import('@deepseek-ai/dsh-llm')
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'block-start', index: 0, blockType: 'text' })
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'good' })
|
||||
assembler.push({ type: 'block-end', index: 0, block: { type: 'text', text: 'good' } })
|
||||
assembler.push({ type: 'text-delta', index: 0, text: ' straggler' })
|
||||
expect(assembler.blocks()).toEqual([{ type: 'text', text: 'good' }])
|
||||
})
|
||||
|
||||
it('assembles tool-call blocks from deltas without block-end', async () => {
|
||||
const { BlockAssembler } = await import('@deepseek-ai/dsh-llm')
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'tool-call-delta', index: 0, id: CallId('c9'), name: 'echo', argumentsDelta: '{"a"' })
|
||||
assembler.push({ type: 'tool-call-delta', index: 0, id: CallId('c9'), argumentsDelta: ':1}' })
|
||||
expect(assembler.blocks()).toEqual([
|
||||
{ type: 'tool-call', id: CallId('c9'), name: 'echo', arguments: '{"a":1}' },
|
||||
])
|
||||
})
|
||||
|
||||
it('streamBlocks flushes delta-only blocks at end of stream (matches generate())', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const deltaOnly: StreamChunk[] = [
|
||||
{ type: 'text-delta', index: 0, text: 'no ' },
|
||||
{ type: 'text-delta', index: 0, text: 'block-end' },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
ctx.llm.registerAdapter(['m'], new MockAdapter([deltaOnly, deltaOnly]))
|
||||
|
||||
const blocks: ContentBlock[] = []
|
||||
for await (const block of ctx.llm.streamBlocks({ model: 'm', messages: [] })) blocks.push(block)
|
||||
expect(blocks).toEqual([{ type: 'text', text: 'no block-end' }])
|
||||
|
||||
const generated = await ctx.llm.generate({ model: 'm', messages: [] })
|
||||
expect(generated.message.content).toEqual(blocks)
|
||||
})
|
||||
|
||||
it('streamBlocks preserves stream order when an open block precedes a closed one', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
// index 0 never gets block-end (delta-only); index 1 closes mid-stream.
|
||||
const interleaved: StreamChunk[] = [
|
||||
{ type: 'text-delta', index: 0, text: 'first, open' },
|
||||
{ type: 'block-start', index: 1, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 1, text: 'second, closed' },
|
||||
{ type: 'block-end', index: 1, block: { type: 'text', text: 'second, closed' } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
ctx.llm.registerAdapter(['m'], new MockAdapter([interleaved, interleaved]))
|
||||
|
||||
const blocks: ContentBlock[] = []
|
||||
for await (const block of ctx.llm.streamBlocks({ model: 'm', messages: [] })) blocks.push(block)
|
||||
expect(blocks).toEqual([
|
||||
{ type: 'text', text: 'first, open' },
|
||||
{ type: 'text', text: 'second, closed' },
|
||||
])
|
||||
|
||||
// identical to generate()'s assembled order
|
||||
const generated = await ctx.llm.generate({ model: 'm', messages: [] })
|
||||
expect(generated.message.content).toEqual(blocks)
|
||||
})
|
||||
|
||||
it('streamBlocks yields closed blocks incrementally once preceding blocks close', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const script: StreamChunk[] = [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: 'a' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'a' } },
|
||||
{ type: 'block-start', index: 1, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 1, text: 'b' },
|
||||
{ type: 'block-end', index: 1, block: { type: 'text', text: 'b' } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
ctx.llm.registerAdapter(['m'], new MockAdapter([script]))
|
||||
|
||||
const blocks: ContentBlock[] = []
|
||||
for await (const block of ctx.llm.streamBlocks({ model: 'm', messages: [] })) blocks.push(block)
|
||||
expect(blocks).toEqual([{ type: 'text', text: 'a' }, { type: 'text', text: 'b' }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('LOW: discriminated SessionEvent narrows without casts', () => {
|
||||
it('narrows event.data from event.type', () => {
|
||||
const session = new Session(SessionId('s'))
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateResult, Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import type { Config } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { assemble, type AssembledResult } from './assemble.ts'
|
||||
|
||||
/**
|
||||
* Real-API e2e for the hand-rolled adapter: V4 Flash + V4 Pro across
|
||||
@@ -31,7 +32,7 @@ function ask(text: string): Message[] {
|
||||
return [{ role: 'user', content: [{ type: 'text', text }] }]
|
||||
}
|
||||
|
||||
function textOf(result: GenerateResult): string {
|
||||
function textOf(result: AssembledResult): string {
|
||||
return result.message.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
@@ -51,7 +52,7 @@ const weatherTool: ToolSchema = {
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () => {
|
||||
it('flash + thinking disabled: plain text generation', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'disabled' })
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model: FLASH,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
@@ -65,7 +66,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', ()
|
||||
|
||||
it('flash + thinking enabled (effort high): reasoning blocks + reasoning tokens', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'enabled', reasoningEffort: 'high' })
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model: FLASH,
|
||||
messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'),
|
||||
maxTokens: 2000,
|
||||
@@ -82,7 +83,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', ()
|
||||
const ctx = await harness(PRO, { thinking: 'enabled', reasoningEffort: effort })
|
||||
|
||||
// Turn 1: the model must call the tool (and think before it).
|
||||
const first = await ctx.llm.generate({
|
||||
const first = await assemble(ctx,{
|
||||
model: PRO,
|
||||
messages: ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
tools: [weatherTool],
|
||||
@@ -96,7 +97,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', ()
|
||||
|
||||
// Turn 2: send the tool result back WITH the assistant's reasoning
|
||||
// block in history (the official thinking+tools passback rule).
|
||||
const second = await ctx.llm.generate({
|
||||
const second = await assemble(ctx,{
|
||||
model: PRO,
|
||||
messages: [
|
||||
...ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
@@ -120,7 +121,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', ()
|
||||
|
||||
it('pro + thinking disabled: plain generation without reasoning blocks', async () => {
|
||||
const ctx = await harness(PRO, { thinking: 'disabled' })
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model: PRO,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
|
||||
@@ -5,6 +5,7 @@ import { Context } from 'cordis'
|
||||
import LlmService, { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { DeepSeekAdapter, httpErrorCode } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { assemble } from './assemble.ts'
|
||||
|
||||
/** One scripted behavior for the next request the mock server receives. */
|
||||
type Behavior =
|
||||
@@ -90,11 +91,11 @@ async function harness(baseURL: string, config: object = {}) {
|
||||
}
|
||||
|
||||
describe('DeepSeekAdapter against a mock server', () => {
|
||||
it('streams a text generation end to end through ctx.llm.generate', async () => {
|
||||
it('streams a text generation end to end through the assembler', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx, {
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
@@ -130,7 +131,7 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url, { thinking: 'disabled', reasoningEffort: 'high' })
|
||||
|
||||
await ctx.llm.generate({
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
@@ -155,15 +156,15 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
}
|
||||
const server = await mockServer([behavior, behavior, behavior])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(`failed with ${status}`)
|
||||
await expect(
|
||||
ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
.catch((error: unknown) => (error as LlmError).code),
|
||||
).resolves.toBe(code)
|
||||
// The numeric HTTP status is carried on the error for explicit handling.
|
||||
await expect(
|
||||
ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
.catch((error: unknown) => (error as LlmError).status),
|
||||
).resolves.toBe(status)
|
||||
})
|
||||
@@ -171,14 +172,14 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
it('keeps the status-line message for JSON error bodies without a message', async () => {
|
||||
const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/HTTP 500/)
|
||||
})
|
||||
|
||||
it('keeps the status-line message for non-JSON error bodies', async () => {
|
||||
const server = await mockServer([{ kind: 'http-error', status: 502, body: 'Bad Gateway', contentType: 'text/plain' }])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/HTTP 502/)
|
||||
})
|
||||
|
||||
@@ -207,7 +208,7 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
events: ['{"choices":[{"delta":{"content":"par"}}]}'],
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
await expect(assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/terminated|socket|without \[DONE\]/)
|
||||
})
|
||||
|
||||
@@ -278,7 +279,7 @@ describe('plugin registration and config', () => {
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', 'http://env-host:1')
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url) // harness passes explicit config
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests).toHaveLength(1) // hit the explicit URL, not env
|
||||
})
|
||||
|
||||
@@ -288,7 +289,7 @@ describe('plugin registration and config', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { apiKey: 'k', models: ['deepseek-v4-flash'] })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests).toHaveLength(1)
|
||||
})
|
||||
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Test helper: drive `ctx.llm.stream()` through a `BlockAssembler` and return
|
||||
* the assembled message + usage + finish reason. This exercises the same
|
||||
* streaming path production uses (the loop), rather than a service-level
|
||||
* one-shot convenience method.
|
||||
*/
|
||||
|
||||
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
|
||||
import type { Context } from 'cordis'
|
||||
import type { FinishReason, GenerateOptions, Message, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
export interface AssembledResult {
|
||||
message: Message
|
||||
usage?: TokenUsage
|
||||
finish: FinishReason
|
||||
}
|
||||
|
||||
export async function assemble(ctx: Context, options: GenerateOptions): Promise<AssembledResult> {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of ctx.llm.stream(options)) assembler.push(chunk)
|
||||
return {
|
||||
message: assembler.message(),
|
||||
...assembler.usage !== undefined ? { usage: assembler.usage } : {},
|
||||
finish: assembler.finish,
|
||||
}
|
||||
}
|
||||
@@ -47,7 +47,7 @@ describe('translate: text', () => {
|
||||
))) {
|
||||
assembler.push(chunk)
|
||||
}
|
||||
const result = assembler.result()
|
||||
const result = { message: assembler.message(), finish: assembler.finish }
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
})
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateResult, Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import type { Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import type { Config } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { assemble, type AssembledResult } from './assemble.ts'
|
||||
|
||||
/**
|
||||
* Real-API e2e for the pi-ai-backed adapter: V4 Flash + V4 Pro across all
|
||||
@@ -33,14 +34,14 @@ function ask(text: string): Message[] {
|
||||
return [{ role: 'user', content: [{ type: 'text', text }] }]
|
||||
}
|
||||
|
||||
function textOf(result: GenerateResult): string {
|
||||
function textOf(result: AssembledResult): string {
|
||||
return result.message.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
function blockKinds(result: GenerateResult): string[] {
|
||||
function blockKinds(result: AssembledResult): string[] {
|
||||
return result.message.content.map(block => block.type)
|
||||
}
|
||||
|
||||
@@ -57,7 +58,7 @@ const weatherTool: ToolSchema = {
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => {
|
||||
it.each([FLASH, PRO])('%s + reasoning off: plain text generation', async (model) => {
|
||||
const ctx = await harness(model, { reasoning: 'off' })
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
@@ -69,7 +70,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
|
||||
it.each([FLASH, PRO])('%s + reasoning high: reasoning blocks present', async (model) => {
|
||||
const ctx = await harness(model, { reasoning: 'high' })
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model,
|
||||
messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'),
|
||||
maxTokens: 2000,
|
||||
@@ -82,7 +83,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
it('pro + reasoning xhigh (wire max): tool-call round trip', async () => {
|
||||
const ctx = await harness(PRO, { reasoning: 'xhigh' })
|
||||
|
||||
const first = await ctx.llm.generate({
|
||||
const first = await assemble(ctx,{
|
||||
model: PRO,
|
||||
messages: ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
tools: [weatherTool],
|
||||
@@ -94,7 +95,7 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
expect(call!.name).toBe('get_weather')
|
||||
expect(JSON.parse(call!.arguments)).toMatchObject({ city: expect.stringMatching(/paris/i) as string })
|
||||
|
||||
const second = await ctx.llm.generate({
|
||||
const second = await assemble(ctx,{
|
||||
model: PRO,
|
||||
messages: [
|
||||
...ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
@@ -128,8 +129,8 @@ describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () =>
|
||||
|
||||
const prompt = ask('Reply with exactly the word: pong')
|
||||
const [fromDeepSeek, fromPiAi] = await Promise.all([
|
||||
deepseekCtx.llm.generate({ model: FLASH, messages: prompt, maxTokens: 50 }),
|
||||
piCtx.llm.generate({ model: FLASH, messages: prompt, maxTokens: 50 }),
|
||||
assemble(deepseekCtx, { model: FLASH, messages: prompt, maxTokens: 50 }),
|
||||
assemble(piCtx, { model: FLASH, messages: prompt, maxTokens: 50 }),
|
||||
])
|
||||
expect(blockKinds(fromPiAi)).toEqual(blockKinds(fromDeepSeek))
|
||||
expect(fromPiAi.finish.kind).toBe(fromDeepSeek.finish.kind)
|
||||
|
||||
@@ -5,6 +5,7 @@ import { Context } from 'cordis'
|
||||
import LlmService, { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { buildModel, PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { assemble } from './assemble.ts'
|
||||
|
||||
/** Scripted SSE responses, one per request (OpenAI chat-completions shape). */
|
||||
interface MockServer {
|
||||
@@ -79,11 +80,11 @@ async function harness(baseURL: string, config: object = {}) {
|
||||
}
|
||||
|
||||
describe('PiAiAdapter against a mock server', () => {
|
||||
it('streams a text generation through ctx.llm.generate', async () => {
|
||||
it('streams a text generation through the assembler', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx, {
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
@@ -96,7 +97,7 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
const server = await mockServer([{ events: toolEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'weather?' }] }],
|
||||
tools: [{
|
||||
@@ -114,7 +115,7 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
const server = await mockServer([{ events: thinkingEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'high' })
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'think' }] }],
|
||||
})
|
||||
@@ -127,7 +128,7 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
it('sends DeepSeek thinking fields when reasoning is configured', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'xhigh' })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
thinking: { type: 'enabled' },
|
||||
reasoning_effort: 'max', // xhigh maps to max via thinkingLevelMap
|
||||
@@ -137,21 +138,21 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
it('disables thinking for reasoning: off', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'off' })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests[0]).toMatchObject({ thinking: { type: 'disabled' } })
|
||||
})
|
||||
|
||||
it('injects stop sequences through onPayload', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [], stop: ['END'] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [], stop: ['END'] })
|
||||
expect(server.requests[0]).toMatchObject({ stop: ['END'] })
|
||||
})
|
||||
|
||||
it('preserves per-tool strict exactly through onPayload', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
await ctx.llm.generate({
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
tools: [
|
||||
@@ -173,7 +174,7 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
it('preserves raw replayed tool-call arguments in the provider payload', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
await ctx.llm.generate({
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
@@ -192,7 +193,7 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
body: JSON.stringify({ error: { message: 'bad key' } }),
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
const result = await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(result.finish).toMatchObject({ kind: 'error', code: 'AUTH' })
|
||||
expect((result.finish as { message: string }).message).toMatch(/bad key|401/)
|
||||
})
|
||||
@@ -204,13 +205,13 @@ describe('PiAiAdapter against a mock server', () => {
|
||||
] as const)('maps HTTP %s to stable error code %s', async (status, code) => {
|
||||
const server = await mockServer([{ status, body: JSON.stringify({ error: { message: `provider ${status}` } }) }])
|
||||
const ctx = await harness(server.url)
|
||||
const result = await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
const result = await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(result.finish).toMatchObject({ kind: 'error', code })
|
||||
})
|
||||
|
||||
it('rejects prefill with UNSUPPORTED', async () => {
|
||||
const ctx = await harness('http://127.0.0.1:1')
|
||||
await expect(ctx.llm.generate({
|
||||
await expect(assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
prefill: [{ type: 'text', text: 'Sure' }],
|
||||
@@ -244,7 +245,7 @@ describe('option spreads and env fallbacks', () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
const controller = new AbortController()
|
||||
await ctx.llm.generate({
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
temperature: 0.5,
|
||||
@@ -262,7 +263,7 @@ describe('option spreads and env fallbacks', () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, { models: ['deepseek-v4-flash'] })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests).toHaveLength(1)
|
||||
} finally {
|
||||
vi.unstubAllEnvs()
|
||||
@@ -311,7 +312,7 @@ describe('review fixes', () => {
|
||||
it('defaults omitted reasoning config to thinking ENABLED (provider default)', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url) // no reasoning key at all
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
await assemble(ctx,{ model: 'deepseek-v4-flash', messages: [] })
|
||||
const request = server.requests[0] as Record<string, unknown>
|
||||
expect(request.thinking).toEqual({ type: 'enabled' })
|
||||
expect('reasoning_effort' in request).toBe(false)
|
||||
@@ -320,7 +321,7 @@ describe('review fixes', () => {
|
||||
it('replays reasoning_content on assistant tool-call turns (passback rule)', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
await ctx.llm.generate({
|
||||
await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [
|
||||
{ role: 'user', content: [{ type: 'text', text: 'weather?' }] },
|
||||
@@ -376,7 +377,7 @@ describe('review fixes: abort wiring', () => {
|
||||
const controller = new AbortController()
|
||||
controller.abort('already cancelled')
|
||||
// pi-ai surfaces the abort as an in-stream error event → aborted finish.
|
||||
const result = await ctx.llm.generate({
|
||||
const result = await assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
signal: controller.signal,
|
||||
@@ -388,7 +389,7 @@ describe('review fixes: abort wiring', () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
const controller = new AbortController()
|
||||
const pending = ctx.llm.generate({
|
||||
const pending = assemble(ctx,{
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
signal: controller.signal,
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Test helper: drive `ctx.llm.stream()` through a `BlockAssembler` and return
|
||||
* the assembled message + usage + finish reason. This exercises the same
|
||||
* streaming path production uses (the loop), rather than a service-level
|
||||
* one-shot convenience method.
|
||||
*/
|
||||
|
||||
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
|
||||
import type { Context } from 'cordis'
|
||||
import type { FinishReason, GenerateOptions, Message, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
export interface AssembledResult {
|
||||
message: Message
|
||||
usage?: TokenUsage
|
||||
finish: FinishReason
|
||||
}
|
||||
|
||||
export async function assemble(ctx: Context, options: GenerateOptions): Promise<AssembledResult> {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of ctx.llm.stream(options)) assembler.push(chunk)
|
||||
return {
|
||||
message: assembler.message(),
|
||||
...assembler.usage !== undefined ? { usage: assembler.usage } : {},
|
||||
finish: assembler.finish,
|
||||
}
|
||||
}
|
||||
@@ -4,28 +4,24 @@ Provider-neutral LLM vocabulary and abstract service. This package defines the c
|
||||
|
||||
## Service: `LlmService` (ctx key: `llm`)
|
||||
|
||||
An adapter registry plus streaming / non-streaming call surfaces. Both call surfaces are interceptable via waterfall events.
|
||||
An adapter registry plus a single streaming call surface, interceptable via a waterfall event.
|
||||
|
||||
### Public API
|
||||
|
||||
- `ctx.llm.registerAdapter(models: string[], adapter: LlmAdapter): () => void` Register an adapter for the given model names. Disposed with the calling fiber.
|
||||
- `ctx.llm.models(): string[]` — model names with a registered adapter.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas).
|
||||
- `ctx.llm.streamBlocks(options: GenerateOptions): AsyncIterable<ContentBlock>` Stream as completed content blocks (convenience view).
|
||||
- `ctx.llm.generate(options: GenerateOptions): Promise<GenerateResult>` One model call, fully assembled.
|
||||
- `ctx.llm.stream(options: GenerateOptions): AsyncIterable<StreamChunk>` Stream one model call as raw chunks (token-level deltas). Consumers assemble the chunks into blocks/messages with `BlockAssembler`.
|
||||
|
||||
### Events
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
|---|---|---|
|
||||
| `llm/stream` | waterfall | Intercept/wrap every streaming model call (retry, caching, routing) |
|
||||
| `llm/generate` | waterfall | Intercept/wrap every non-streaming model call |
|
||||
| `llm/adapter-change` | emit | An adapter was registered or unregistered |
|
||||
|
||||
### Extension points
|
||||
|
||||
- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(models, adapter)` to add a new model provider.
|
||||
- Wrap `llm/stream` or `llm/generate` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
- Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, retry, logging, rate-limiting, etc.
|
||||
|
||||
### Content-block vocabulary (`types.ts`)
|
||||
|
||||
@@ -36,8 +32,7 @@ Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta
|
||||
### Classes
|
||||
|
||||
- `LlmAdapter` — abstract base class for provider adapters. The only required method is `stream()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. Used by the agent loop (raw chunks for replay
|
||||
+ assembled for history) and by `streamBlocks()`/`generate()`.
|
||||
- `BlockAssembler` — incrementally assembles raw chunks into complete content blocks and an assistant message. The agent loop feeds it raw chunks (logging them for replay) while reading the assembled blocks/message for history.
|
||||
- `HarnessError` — base class for the harness error taxonomy: a stable `code` string (distinct from the human `message`) plus `cause` chaining. Lives here, in the leaf package every other imports, so a single base is shared without a new dependency edge. Per-package errors (`LlmError`, `ToolArgsError`, `InvariantError`, …) extend it. `isHarnessError(value)` narrows at seams.
|
||||
- `LlmError` — extends `HarnessError`; `code` string (`NO_ADAPTER`, `DUPLICATE_ADAPTER`, and adapter codes like `AUTH`/`RATE_LIMIT`) plus an optional numeric `status` when the failure came from a non-2xx provider response.
|
||||
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
/**
|
||||
* Incremental chunk-to-message assembler. This is the single canonical assembly
|
||||
* algorithm used by both the agent loop and the LLM service convenience views.
|
||||
* algorithm used by the agent loop to build an assistant message from a chunk
|
||||
* stream while logging the raw chunks for replay fidelity.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm/assembler
|
||||
*/
|
||||
|
||||
import { CallId } from './brand.ts'
|
||||
import { assertNever } from './never.ts'
|
||||
import type { ContentBlock, FinishReason, GenerateResult, Message, StreamChunk, TokenUsage } from './types.ts'
|
||||
import type { ContentBlock, FinishReason, Message, StreamChunk, TokenUsage } from './types.ts'
|
||||
|
||||
interface PartialBlock {
|
||||
blockType: string
|
||||
@@ -23,9 +24,8 @@ interface PartialBlock {
|
||||
* Incrementally assembles raw {@link StreamChunk}s into complete
|
||||
* {@link ContentBlock}s and a final assistant {@link Message}.
|
||||
*
|
||||
* This is the single shared assembly implementation: the agent loop feeds it
|
||||
* while logging raw chunks for replay fidelity, and `LlmService.generate()` /
|
||||
* `streamBlocks()` use it to offer assembled views of the same stream.
|
||||
* The agent loop feeds it while logging raw chunks for replay fidelity, then
|
||||
* reads `blocks()` / `message()` / `usage` / `finish` once the stream ends.
|
||||
*
|
||||
* Tolerant of delta-only protocols (no block-start/end); deltas arriving for
|
||||
* an index already closed by `block-end` are ignored (malformed stream) so a
|
||||
@@ -34,7 +34,6 @@ interface PartialBlock {
|
||||
export class BlockAssembler {
|
||||
private partials = new Map<number, PartialBlock>()
|
||||
private order: number[] = []
|
||||
private flushed = 0
|
||||
private _usage: TokenUsage | undefined
|
||||
private _finish: FinishReason | undefined
|
||||
|
||||
@@ -129,44 +128,6 @@ export class BlockAssembler {
|
||||
return this.order.map(index => this.assemble(this.mustGet(index), index))
|
||||
}
|
||||
|
||||
/**
|
||||
* Streaming flush: returns (once) every block that is complete AND has no
|
||||
* incomplete block before it in stream order. Call after each `push()`;
|
||||
* blocks come out strictly in stream order, so a streaming consumer sees
|
||||
* exactly the sequence `blocks()` would produce.
|
||||
*/
|
||||
flushReady(): ContentBlock[] {
|
||||
const ready: ContentBlock[] = []
|
||||
while (this.flushed < this.order.length) {
|
||||
const index = this.order[this.flushed]
|
||||
/* v8 ignore next 3 -- noUncheckedIndexedAccess guard: loop condition guarantees index exists in a non-empty array */
|
||||
if (index === undefined) break
|
||||
const partial = this.mustGet(index)
|
||||
if (!partial.block) break
|
||||
ready.push(partial.block)
|
||||
this.flushed += 1
|
||||
}
|
||||
return ready
|
||||
}
|
||||
|
||||
/**
|
||||
* End-of-stream flush: returns (once) all not-yet-flushed blocks, in stream
|
||||
* order, assembling still-open ones from their deltas (delta-only
|
||||
* protocols). After this, `flushReady()` + `flushRemaining()` together have
|
||||
* yielded exactly `blocks()`.
|
||||
*/
|
||||
flushRemaining(): ContentBlock[] {
|
||||
const remaining: ContentBlock[] = []
|
||||
while (this.flushed < this.order.length) {
|
||||
const index = this.order[this.flushed]
|
||||
/* v8 ignore next 3 -- noUncheckedIndexedAccess guard: loop condition guarantees index exists */
|
||||
if (index === undefined) break
|
||||
remaining.push(this.assemble(this.mustGet(index), index))
|
||||
this.flushed += 1
|
||||
}
|
||||
return remaining
|
||||
}
|
||||
|
||||
get usage(): TokenUsage | undefined {
|
||||
return this._usage
|
||||
}
|
||||
@@ -179,13 +140,4 @@ export class BlockAssembler {
|
||||
message(): Message {
|
||||
return { role: 'assistant', content: this.blocks() }
|
||||
}
|
||||
|
||||
/** The assembled non-streaming result. */
|
||||
result(): GenerateResult {
|
||||
return {
|
||||
message: this.message(),
|
||||
...this._usage !== undefined ? { usage: this._usage } : {},
|
||||
finish: this.finish,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,14 +1,13 @@
|
||||
/**
|
||||
* LLM service: adapter registry with waterfall-interceptable streaming and
|
||||
* non-streaming call surfaces. Exports the `LlmService` default, the abstract
|
||||
* `LlmAdapter` for provider backends, and `BlockAssembler` for chunk assembly.
|
||||
* LLM service: adapter registry with a waterfall-interceptable streaming call
|
||||
* surface. Exports the `LlmService` default, the abstract `LlmAdapter` for
|
||||
* provider backends, and `BlockAssembler` for chunk assembly.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import type { ContentBlock, GenerateOptions, GenerateResult, StreamChunk } from './types.ts'
|
||||
import { BlockAssembler } from './assembler.ts'
|
||||
import type { GenerateOptions, StreamChunk } from './types.ts'
|
||||
import { HarnessError } from './error.ts'
|
||||
|
||||
export * from './brand.ts'
|
||||
@@ -30,17 +29,6 @@ declare module 'cordis' {
|
||||
* @mode waterfall
|
||||
*/
|
||||
'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable<StreamChunk>): AsyncIterable<StreamChunk>
|
||||
/**
|
||||
* Waterfall around every non-streaming model call. Bound to the
|
||||
* {@link LlmService}; call `next()` to delegate to the adapter.
|
||||
* @mode waterfall
|
||||
*/
|
||||
'llm/generate'(this: LlmService, options: GenerateOptions, next: () => Promise<GenerateResult>): Promise<GenerateResult>
|
||||
/**
|
||||
* An adapter was registered or unregistered (the model→adapter map changed).
|
||||
* @mode emit
|
||||
*/
|
||||
'llm/adapter-change'(): void
|
||||
}
|
||||
}
|
||||
|
||||
@@ -88,8 +76,7 @@ export class LlmService extends Service {
|
||||
/**
|
||||
* Register an adapter for the given model names. Throws `LlmError` with code
|
||||
* `DUPLICATE_ADAPTER` if any model already has an adapter (all-or-nothing).
|
||||
* Emits `llm/adapter-change` on registration and disposal. Disposed with the
|
||||
* fiber.
|
||||
* Disposed with the fiber.
|
||||
*/
|
||||
registerAdapter(models: string[], adapter: LlmAdapter): () => void {
|
||||
const dispose = this.ctx.effect(function* (this: LlmService) {
|
||||
@@ -99,17 +86,9 @@ export class LlmService extends Service {
|
||||
}
|
||||
}
|
||||
for (const model of models) this.adapters.set(model, adapter)
|
||||
// Yield the rollback BEFORE emitting the change event: a generator effect
|
||||
// collects each yielded disposer before running the next step, so a
|
||||
// throwing `llm/adapter-change` listener rolls the mutation back instead
|
||||
// of leaking the entry (which would wedge the duplicate check until
|
||||
// restart). The duplicate throws above fire before any mutation, so they
|
||||
// correctly leak nothing.
|
||||
yield () => {
|
||||
for (const model of models) this.adapters.delete(model)
|
||||
this.ctx.emit('llm/adapter-change')
|
||||
}
|
||||
this.ctx.emit('llm/adapter-change')
|
||||
}.bind(this), 'llm.registerAdapter()')
|
||||
// ctx.effect's disposer returns Promise<void>; our disposer API is
|
||||
// synchronous fire-and-forget — discard the (always-resolved) promise.
|
||||
@@ -137,36 +116,6 @@ export class LlmService extends Service {
|
||||
return this.adapter(options.model).stream(options)
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream one model call as completed content blocks — a convenience view
|
||||
* for consumers that don't care about token-level deltas. Blocks are
|
||||
* yielded strictly in stream order as soon as they (and everything before
|
||||
* them) complete; blocks left open at end of stream (delta-only protocols)
|
||||
* are assembled and flushed last, so the sequence always equals
|
||||
* `generate()`'s `message.content`.
|
||||
*/
|
||||
async * streamBlocks(options: GenerateOptions): AsyncIterable<ContentBlock> {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of this.stream(options)) {
|
||||
assembler.push(chunk)
|
||||
yield * assembler.flushReady()
|
||||
}
|
||||
yield * assembler.flushRemaining()
|
||||
}
|
||||
|
||||
/**
|
||||
* One model call, fully assembled (drains the chunk stream). Dispatches
|
||||
* through the `llm/generate` waterfall (and the inner stream through
|
||||
* `llm/stream`). Same completion guarantees as `streamBlocks()`.
|
||||
*/
|
||||
generate(options: GenerateOptions): Promise<GenerateResult> {
|
||||
return this.ctx.waterfall(this, 'llm/generate', options, async () => {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of this.stream(options)) assembler.push(chunk)
|
||||
return assembler.result()
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
export default LlmService
|
||||
@@ -193,10 +193,3 @@ export interface GenerateOptions {
|
||||
stop?: string[]
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
/** Non-streaming result, assembled from the chunk stream. */
|
||||
export interface GenerateResult {
|
||||
message: Message
|
||||
usage?: TokenUsage
|
||||
finish: FinishReason
|
||||
}
|
||||
@@ -81,36 +81,6 @@ describe('BlockAssembler', () => {
|
||||
expect(() => assembler.blocks()).toThrow('BlockAssembler invariant violated')
|
||||
})
|
||||
|
||||
it('assembles open blocks at end of stream via flushRemaining', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'open' })
|
||||
assembler.push({ type: 'reasoning-delta', index: 1, text: 'thinking' })
|
||||
|
||||
// flushReady returns nothing because index 0 is incomplete and blocking
|
||||
const ready = assembler.flushReady()
|
||||
expect(ready).toEqual([])
|
||||
|
||||
// flushRemaining assembles everything still open
|
||||
const remaining = assembler.flushRemaining()
|
||||
expect(remaining).toEqual([
|
||||
{ type: 'text', text: 'open' },
|
||||
{ type: 'reasoning', text: 'thinking' },
|
||||
])
|
||||
|
||||
// blocks() now matches the flushed view
|
||||
expect(assembler.blocks()).toEqual(remaining)
|
||||
})
|
||||
|
||||
it('result() omits usage key when no usage was received', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'msg' })
|
||||
const result = assembler.result()
|
||||
expect(result.message).toBeDefined()
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
// usage should NOT be present on the object at all
|
||||
expect('usage' in result).toBe(false)
|
||||
})
|
||||
|
||||
it('ignores duplicate block-start for the same index', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'block-start', index: 0, blockType: 'text' })
|
||||
@@ -142,13 +112,11 @@ describe('BlockAssembler', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('includes usage in result() when usage was received', () => {
|
||||
it('exposes usage via the getter when a usage chunk was received', () => {
|
||||
const assembler = new BlockAssembler()
|
||||
assembler.push({ type: 'text-delta', index: 0, text: 'msg' })
|
||||
assembler.push({ type: 'usage', usage: { inputTokens: 5, outputTokens: 3 } })
|
||||
const result = assembler.result()
|
||||
expect(result.usage).toEqual({ inputTokens: 5, outputTokens: 3 })
|
||||
expect('usage' in result).toBe(true)
|
||||
expect(assembler.usage).toEqual({ inputTokens: 5, outputTokens: 3 })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -172,25 +140,25 @@ describe('BlockAssembler regressions (property-test findings)', () => {
|
||||
// Found by fast-check (the property-testing RFC): two block-ends at the same index made the
|
||||
// streamed prefix (first block) disagree with final blocks() (second
|
||||
// block). The first close must win — same straggler rule as post-close
|
||||
// deltas — so streaming and one-shot assembly stay identical.
|
||||
// deltas — so the prefix returned incrementally by push() and the final
|
||||
// blocks() stay identical.
|
||||
const chunks: StreamChunk[] = [
|
||||
{ type: 'block-end', index: 0, block: { type: 'reasoning', text: 'first' } },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'second' } },
|
||||
]
|
||||
const streaming = new BlockAssembler()
|
||||
const flushed = []
|
||||
const closed = []
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
flushed.push(...streaming.flushReady())
|
||||
const block = streaming.push(chunk)
|
||||
if (block) closed.push(block)
|
||||
}
|
||||
flushed.push(...streaming.flushRemaining())
|
||||
|
||||
const oneShot = new BlockAssembler()
|
||||
for (const chunk of chunks) oneShot.push(chunk)
|
||||
|
||||
expect(flushed).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(closed).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(oneShot.blocks()).toEqual([{ type: 'reasoning', text: 'first' }])
|
||||
expect(flushed).toEqual(oneShot.blocks())
|
||||
expect(closed).toEqual(oneShot.blocks())
|
||||
})
|
||||
|
||||
it('push returns undefined for a duplicate block-end (it closed nothing)', () => {
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import fc from 'fast-check'
|
||||
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
// A small pool of indices so collisions (duplicate-index bugs) are common.
|
||||
@@ -55,38 +55,6 @@ function feed(chunks: StreamChunk[]): BlockAssembler {
|
||||
}
|
||||
|
||||
describe('BlockAssembler properties', () => {
|
||||
it('flushReady() ++ flushRemaining() === blocks(), in order', () => {
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
const streaming = new BlockAssembler()
|
||||
const flushed: ContentBlock[] = []
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
flushed.push(...streaming.flushReady())
|
||||
}
|
||||
flushed.push(...streaming.flushRemaining())
|
||||
|
||||
const oneShot = feed(chunks).blocks()
|
||||
expect(flushed).toEqual(oneShot)
|
||||
}))
|
||||
})
|
||||
|
||||
it('streamBlocks-style flush never yields a block before an earlier open one', () => {
|
||||
// flushReady is strict-order: once it stops at an open index, no later
|
||||
// index may be emitted until that one closes. We assert the flushed prefix
|
||||
// is always a prefix of the final blocks() order.
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
const streaming = new BlockAssembler()
|
||||
const flushed: ContentBlock[] = []
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
flushed.push(...streaming.flushReady())
|
||||
}
|
||||
const finalSoFar = streaming.blocks()
|
||||
// Everything flushed mid-stream is a prefix of the full ordered blocks.
|
||||
expect(finalSoFar.slice(0, flushed.length)).toEqual(flushed)
|
||||
}))
|
||||
})
|
||||
|
||||
it('partials map size never exceeds the number of distinct indices seen', () => {
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
const distinct = new Set<number>()
|
||||
@@ -134,13 +102,9 @@ describe('BlockAssembler properties', () => {
|
||||
|
||||
it('streaming and one-shot assembly agree on usage and finish', () => {
|
||||
fc.assert(fc.property(streamArb, (chunks) => {
|
||||
// Streaming consumer: push + flush as it goes.
|
||||
// Streaming consumer: push as it goes.
|
||||
const streaming = new BlockAssembler()
|
||||
for (const chunk of chunks) {
|
||||
streaming.push(chunk)
|
||||
streaming.flushReady()
|
||||
}
|
||||
streaming.flushRemaining()
|
||||
for (const chunk of chunks) streaming.push(chunk)
|
||||
// One-shot consumer: push all, then read.
|
||||
const oneShot = feed(chunks)
|
||||
expect(streaming.usage).toEqual(oneShot.usage)
|
||||
|
||||
@@ -19,24 +19,22 @@ const SCRIPT: StreamChunk[] = [
|
||||
]
|
||||
|
||||
describe('LlmService', () => {
|
||||
it('routes stream() to the registered adapter and generate() assembles it', async () => {
|
||||
it('routes stream() to the registered adapter', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT))
|
||||
|
||||
const chunks: StreamChunk[] = []
|
||||
for await (const chunk of ctx.llm.stream({ model: 'test-model', messages: [] })) chunks.push(chunk)
|
||||
expect(chunks).toHaveLength(3)
|
||||
|
||||
const result = await ctx.llm.generate({ model: 'test-model', messages: [] })
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
expect(chunks).toEqual(SCRIPT)
|
||||
})
|
||||
|
||||
it('throws NO_ADAPTER for unregistered models', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.llm.generate({ model: 'nope', messages: [] })).rejects.toThrow('no adapter registered')
|
||||
await expect((async () => {
|
||||
for await (const _ of ctx.llm.stream({ model: 'nope', messages: [] })) { /* drain */ }
|
||||
})()).rejects.toThrow('no adapter registered')
|
||||
})
|
||||
|
||||
it('unregisters adapters when the owning fiber is disposed (HMR safety)', async () => {
|
||||
@@ -71,21 +69,6 @@ describe('LlmService', () => {
|
||||
expect(chunks[0]).toMatchObject({ index: 99 })
|
||||
})
|
||||
|
||||
it('lets llm/generate waterfall listeners intercept and transform the result', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
ctx.llm.registerAdapter(['test-model'], new ScriptedAdapter(SCRIPT))
|
||||
|
||||
ctx.on('llm/generate', async function (_options, next) {
|
||||
const result = await next()
|
||||
return { ...result, finish: { kind: 'max-tokens' } as const }
|
||||
})
|
||||
|
||||
const result = await ctx.llm.generate({ model: 'test-model', messages: [] })
|
||||
expect(result.finish).toEqual({ kind: 'max-tokens' })
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }])
|
||||
})
|
||||
|
||||
it('creates LlmError with a code for programmatic handling', () => {
|
||||
const err = new LlmError('something went wrong', 'CUSTOM_CODE')
|
||||
expect(err).toBeInstanceOf(Error)
|
||||
@@ -116,20 +99,13 @@ describe('LlmService', () => {
|
||||
expect(isHarnessError('nope')).toBe(false)
|
||||
})
|
||||
|
||||
it('disposes adapter registration on adapter-change event emission', async () => {
|
||||
it('removes the adapter when the returned disposer is called', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
const changes: string[][] = []
|
||||
ctx.on('llm/adapter-change', () => {
|
||||
changes.push([...ctx.llm.models()])
|
||||
})
|
||||
|
||||
const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(changes).toEqual([['m1']])
|
||||
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
dispose()
|
||||
expect(changes).toEqual([['m1'], []])
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
|
||||
@@ -147,25 +123,19 @@ describe('LlmService', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rolls back the adapter entry when an adapter-change listener throws (P1-1)', async () => {
|
||||
it('re-registers a model after its prior registration is disposed', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
|
||||
// A change listener that throws on the FIRST emit only.
|
||||
let threw = false
|
||||
ctx.on('llm/adapter-change', () => {
|
||||
if (!threw) { threw = true; throw new Error('boom change listener') }
|
||||
})
|
||||
|
||||
// The throwing emit must roll the mutation back, not leak it.
|
||||
expect(() => ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))).toThrow('boom change listener')
|
||||
expect(ctx.llm.models()).toEqual([]) // entry rolled back, not leaked
|
||||
|
||||
// A subsequent listener-free register of the SAME model succeeds and
|
||||
// contributes exactly once (the duplicate check is not wedged).
|
||||
const dispose = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
|
||||
// The duplicate check is not wedged: the same model registers cleanly again.
|
||||
const disposeAgain = ctx.llm.registerAdapter(['m1'], new ScriptedAdapter(SCRIPT))
|
||||
expect(ctx.llm.models()).toEqual(['m1'])
|
||||
disposeAgain()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -7,7 +7,6 @@
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "MessageSourceMap", "source": "packages/llm/llm/src/types.ts" },
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "FinishReasonMap", "source": "packages/llm/llm/src/types.ts" },
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "GenerateOptions", "source": "packages/llm/llm/src/types.ts" },
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "GenerateResult", "source": "packages/llm/llm/src/types.ts" },
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "ToolSchema", "source": "packages/llm/llm/src/types.ts" },
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "SessionEvent", "source": "packages/core/session/src/types.ts" },
|
||||
{ "doc": "docs/core-data-structures/core.md", "symbol": "Agent", "source": "packages/core/agent/src/types.ts" },
|
||||
|
||||
Reference in New Issue
Block a user