Add two DeepSeek LLM adapters: dsh-llm-deepseek and dsh-llm-pi-ai
The first real LlmAdapter implementations, shipped as a deliberate pair:
same models and wire protocol, completely different internals, so the
StreamChunk protocol is verified across independent implementations.
- dsh-llm-deepseek: hand-rolled fetch + SSE parser + chunk-translation
state machine against the official chat-completions format (thinking
mode via top-level thinking/reasoning_effort; the empty-string
reasoning_content first chunk; usage attached to the finish chunk or
trailing; reasoning_content passback on tool-call turns; disjoint
cache-token accounting).
- dsh-llm-pi-ai: the same endpoint through @earendil-works/pi-ai,
mapping its event vocabulary (parsed tool arguments, in-stream error
events, folded reasoning tokens) onto the same chunks.
The agent loop now honors the in-band error path: an adapter that ends
its stream with finish {kind:error|aborted} (the only option for
adapters that can't throw mid-stream, like pi-ai) is translated into a
step error, so the turn ends error/aborted with a logged error event
instead of a normal completed assistant message. This makes the
StreamChunk error contract real for both adapters; docs/architecture.md
and the StreamChunk doc are updated accordingly.
New yarn test:e2e (vitest.e2e.config.ts, *.e2e.ts) runs key-gated
real-API matrices for both adapters across V4 Flash/Pro and all
thinking/effort levels; it self-skips without DEEPSEEK_API_KEY. Unit
suites run against local node:http mock SSE servers at 100% per-file
coverage.
This commit is contained in:
Tianyi Cui
@@ -21,7 +21,9 @@ vendor/ Vendored Cordis framework source (original npm names, private).
|
||||
and the upstream sync procedure. Do NOT edit casually — every
|
||||
divergence must be logged there.
|
||||
packages/ Harness packages, all named @deepseek-ai/dsh-<name>:
|
||||
llm/ abstract LLM service + content-block vocabulary (no real adapter yet)
|
||||
llm/ abstract LLM service + content-block vocabulary
|
||||
llm-deepseek/ DeepSeek API adapter (hand-rolled fetch/SSE)
|
||||
llm-pi-ai/ DeepSeek adapter via @earendil-works/pi-ai (design twin)
|
||||
session/ event-sourced session log + in-memory store
|
||||
system-prompt/ prompt-section + tool-schema assembly registry
|
||||
tools/ tool registry + tools/execute waterfall
|
||||
@@ -45,12 +47,29 @@ scripts/ repo maintenance scripts (vendor-manifest guard, publint runner).
|
||||
```sh
|
||||
yarn install # Yarn 4 workspaces (node-modules linker), node >= 24
|
||||
yarn test # vitest run (packages/*/tests/**/*.spec.ts)
|
||||
yarn test:e2e # real-API tests (packages|examples/*/tests/**/*.e2e.ts);
|
||||
# self-skips without DEEPSEEK_API_KEY — see Secrets below
|
||||
yarn typecheck # tsc -b tsconfig.build.json (declarations only)
|
||||
yarn build # typecheck + tsdown JS bundles into each package's lib/
|
||||
yarn demo # run examples/echo-agent (needs --expose-internals, the
|
||||
# script passes it; type "echo hi" to see a tool call)
|
||||
```
|
||||
|
||||
## Secrets / .env
|
||||
|
||||
Real-API e2e tests (`yarn test:e2e`) read `DEEPSEEK_API_KEY` (and optionally
|
||||
`DEEPSEEK_BASE_URL`) from the environment, or from a gitignored `.env` at the
|
||||
repo root loaded via Node's native `process.loadEnvFile()`:
|
||||
|
||||
```
|
||||
DEEPSEEK_API_KEY=sk-…
|
||||
DEEPSEEK_BASE_URL=https://… # optional; defaults to the public API
|
||||
```
|
||||
|
||||
cordis.yml configs reference env vars with the `!!js` tag:
|
||||
`apiKey: !!js process.env.DEEPSEEK_API_KEY`. Never commit real credentials;
|
||||
CI has no secrets and e2e suites must self-skip without them.
|
||||
|
||||
Dev/test/demo run **unbuilt** via tsx + the `paths` map in the root
|
||||
`tsconfig.json` (`vitest` resolves through `tsconfig.test.json`). Building is
|
||||
only needed for publishing/consumption outside the repo — with one exception:
|
||||
@@ -108,6 +127,15 @@ unresolved-type `no-unsafe-*` errors.
|
||||
`BashExecSpec` (required, what `run`/`start` act on); the tool layer calls
|
||||
`ctx.bash.resolve()` between them. The reader of a `BashExecSpec` never has
|
||||
to wonder where the working directory came from.
|
||||
- **An empty `catch` must name what it swallows and why nothing else can hit
|
||||
it**: a bare `catch {}` hides bugs. When you deliberately ignore a throw, the
|
||||
comment must (a) name the single expected failure, (b) say why ignoring it is
|
||||
correct — usually because the useful state was already captured *before* the
|
||||
`try` — and (c) make clear nothing else of consequence can reach the catch
|
||||
(ideally the `try` wraps a single statement). Example: the error-body
|
||||
`response.json()` parse in `dsh-llm-deepseek`'s adapter sets `code` + HTTP
|
||||
`status` from the status line before the `try`, so a malformed provider body
|
||||
can only cost a richer message, never the real error.
|
||||
- **Tests**: vitest, colocated under `packages/<name>/tests/*.spec.ts`. Every
|
||||
registry needs an HMR-safety test (dispose the contributing fiber, assert
|
||||
cleanup). **Excessive tests are welcome** — when in doubt, write the test;
|
||||
@@ -132,12 +160,20 @@ tool authors zero-cast typed `execute` args, and the cost of the conditional
|
||||
types stays inside the core package.
|
||||
|
||||
Verbose documentation is fine **as long as docs and code stay strictly in
|
||||
sync**. Out-of-sync docs are worse than no docs. Every module has a module-level
|
||||
doc comment explaining its role. Every exported class, interface, type,
|
||||
function, and non-obvious method has a JSDoc that explains semantics (not just
|
||||
the name) — contracts (what events fire when), disposal behavior, error
|
||||
behavior, and extension intent. Internal helpers get docs only where non-obvious.
|
||||
Prefer one-liners when one line suffices.
|
||||
sync**. Out-of-sync docs are worse than no docs. **When you change code, update
|
||||
its docs in the SAME change** — grep the package README and the module/JSDoc
|
||||
comments for the old behavior (config keys, defaults, error codes, wire field
|
||||
names, event names) and fix every hit. CI has no doc-sync gate, so this is on
|
||||
the author. Every module has a module-level doc comment explaining its role.
|
||||
Every exported class, interface, type, function, and non-obvious method has a
|
||||
JSDoc that explains semantics (not just the name) — contracts (what events fire
|
||||
when), disposal behavior, error behavior, and extension intent. Internal
|
||||
helpers get docs only where non-obvious. Prefer one-liners when one line
|
||||
suffices.
|
||||
|
||||
**Editing these instructions**: `AGENTS.md` is the real file; `CLAUDE.md` is a
|
||||
symlink to it (at the repo root and in `packages/`). Always edit `AGENTS.md` —
|
||||
never write through the `CLAUDE.md` symlink or replace it with a regular file.
|
||||
|
||||
## Vendoring Policy
|
||||
|
||||
|
||||
+18
-9
@@ -112,9 +112,14 @@ into blocks/messages; the loop logs raw chunks (replay fidelity) while feeding
|
||||
the same chunks through an assembler.
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, call
|
||||
`ctx.llm.registerAdapter(models, adapter)`.
|
||||
**TODO**: the DeepSeek V4 adapter is the first real adapter (next phase); the
|
||||
streaming protocol gets a careful review then.
|
||||
`ctx.llm.registerAdapter(models, adapter)`. Two real adapters implement it —
|
||||
`dsh-llm-deepseek` (hand-rolled fetch/SSE against the DeepSeek API) and
|
||||
`dsh-llm-pi-ai` (the same endpoint through the `@earendil-works/pi-ai`
|
||||
library). They exist as a pair deliberately: two independent internals over
|
||||
one contract verified the StreamChunk protocol, which is now documented (in
|
||||
`dsh-llm/src/types.ts`) with the conventions that review pinned down — usage
|
||||
before finish, nothing after finish, raw-string tool arguments, and the two
|
||||
sanctioned error paths (thrown vs `finish {kind:'error'}`).
|
||||
|
||||
## Event-sourced sessions (dsh-session)
|
||||
|
||||
@@ -206,6 +211,9 @@ forever:
|
||||
req = waterfall agent/request ⟵ hooks, compaction, model switch
|
||||
stream ctx.llm.stream(req) ⟵ waterfall llm/stream (raw chunks)
|
||||
session('assistant/chunk'); emit agent/stream-chunk
|
||||
if assembler.finish is error/aborted: throw ⟵ adapter's in-band error path →
|
||||
step error (turn ends error/aborted,
|
||||
not a normal completed message)
|
||||
msg = waterfall agent/step-result ⟵ runs BEFORE the log append, so the
|
||||
session('assistant/message', 'usage') log records what tool dispatch uses
|
||||
each tool-call (sequential, abort-checked between calls):
|
||||
@@ -225,9 +233,12 @@ forever:
|
||||
|
||||
Error containment: a throwing `agent/turn-continuation` listener or a
|
||||
rejecting `session/flush` ends the **turn** with an `error` event — never the
|
||||
driver loop. `abort()` is honored mid-stream **and** between tool calls;
|
||||
disposal mid-turn ends the turn with reason `disposed` and emits
|
||||
`agent/status('disposed')`.
|
||||
driver loop. An adapter that ends its stream with a `finish {kind:'error'}`
|
||||
or `{kind:'aborted'}` chunk (the in-band error path, for adapters that can't
|
||||
throw mid-stream) is likewise translated into a step error, so the turn ends
|
||||
`error`/`aborted` instead of logging a normal `completed` assistant message.
|
||||
`abort()` is honored mid-stream **and** between tool calls; disposal mid-turn
|
||||
ends the turn with reason `disposed` and emits `agent/status('disposed')`.
|
||||
|
||||
### Event taxonomy
|
||||
|
||||
@@ -293,7 +304,7 @@ implements it **without modifying the loop**:
|
||||
| Scheduled tasks (cron) | plugin registers model-callable scheduling tools; timer fires → `send(…, {source: {kind: 'cron', …}})` when idle / `inject()` notification when busy |
|
||||
| UI (GUI; CLI emits JSONL) | listen `agent/stream-chunk` + `session/event`; input → `send()` |
|
||||
| Telemetry / replayable trace | `session/event` → JSONL; replay = `sessions.create(id, seed)` |
|
||||
| DeepSeek V4 (and other) models | `LlmAdapter` subclass via `registerAdapter` |
|
||||
| DeepSeek V4 (and other) models | `LlmAdapter` subclass via `registerAdapter`. **Implemented twice**: `dsh-llm-deepseek` (hand-rolled) and `dsh-llm-pi-ai` (pi-ai-backed) |
|
||||
| Plugin hot-reload | every registration is a `ctx.effect` → vendored HMR just works |
|
||||
|
||||
## Extension cookbook
|
||||
@@ -375,8 +386,6 @@ Tracked here deliberately — each is designed-for but not implemented:
|
||||
- **Compaction implementation** (auto thresholds, summarization prompts) on
|
||||
the `agent/request` seam, with its session-event types added by declaration
|
||||
merging.
|
||||
- **DeepSeek V4 adapter** — first real `LlmAdapter`; triggers the
|
||||
streaming-protocol review (`TODO(review)` markers in dsh-llm).
|
||||
- **Parallel tool execution** (concurrency-safety hints on ToolDefinition).
|
||||
- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based
|
||||
forking.
|
||||
|
||||
@@ -10,6 +10,14 @@
|
||||
"packages/*": {
|
||||
"entry": ["tests/**/*.spec.ts"],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
},
|
||||
"packages/llm-deepseek": {
|
||||
"entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
},
|
||||
"packages/llm-pi-ai": {
|
||||
"entry": ["tests/**/*.spec.ts", "tests/**/*.e2e.ts"],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,7 @@
|
||||
"lint:fix": "eslint . --fix",
|
||||
"test": "vitest run",
|
||||
"test:coverage": "vitest run --coverage",
|
||||
"test:e2e": "vitest run --config vitest.e2e.config.ts",
|
||||
"knip": "knip",
|
||||
"publint": "tsx scripts/publint-all.ts",
|
||||
"hygiene": "yarn knip && yarn publint && yarn constraints",
|
||||
|
||||
@@ -22,6 +22,9 @@ Naming notes:
|
||||
- Files `src/index.ts` export the service default + all public types
|
||||
- `src/types.ts` contain only types — no runtime code
|
||||
- Tests live at package level under `tests/`, not `src/__tests__/`
|
||||
- A package's README and module/JSDoc comments are part of the change: when you
|
||||
alter behavior (config keys, defaults, error codes, wire fields), update them
|
||||
in the same commit. CI has no doc-sync gate, so stale docs are on the author.
|
||||
|
||||
Read the per-package README.md for package-specific details: service API,
|
||||
events, extension points, TODOs.
|
||||
@@ -8,7 +8,7 @@
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { FinishReason, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import { BlockAssembler } from '@deepseek-ai/dsh-llm'
|
||||
import type { Session, TurnEndReason, TurnTrigger } from '@deepseek-ai/dsh-session'
|
||||
import { renderPrompt } from '@deepseek-ai/dsh-system-prompt'
|
||||
@@ -23,6 +23,40 @@ function toError(error: unknown): CodedError {
|
||||
return error instanceof Error ? error : new Error(String(error))
|
||||
}
|
||||
|
||||
/**
|
||||
* Map a model-call {@link FinishReason} to the step error it should raise, or
|
||||
* `undefined` when the step completed normally.
|
||||
*
|
||||
* Adapters report provider/transport failures one of two sanctioned ways (see
|
||||
* the StreamChunk contract in dsh-llm): throw from `stream()` (handled by the
|
||||
* caller's try/catch), OR end the stream with a finish-error/aborted chunk
|
||||
* (the only option for adapters that can't throw mid-stream, e.g.
|
||||
* library-backed ones). This translates the latter into a thrown step error
|
||||
* so the turn ends error/aborted with a logged `error` event, never as a
|
||||
* normal `completed` assistant message.
|
||||
*
|
||||
* `FinishReason` is merge-extensible (plugins/adapters can add `kind`s), so
|
||||
* the switch handles the known terminal-failure kinds and treats every other
|
||||
* kind — `stop`, `tool-calls`, `max-tokens`, future additions — as success.
|
||||
*/
|
||||
function finishError(finish: FinishReason): CodedError | undefined {
|
||||
switch (finish.kind) {
|
||||
case 'error': {
|
||||
const error: CodedError = new Error(finish.message)
|
||||
if (finish.code !== undefined) error.code = finish.code
|
||||
return error
|
||||
}
|
||||
case 'aborted': {
|
||||
const error: CodedError = new Error('model stream aborted')
|
||||
error.code = 'ABORTED'
|
||||
return error
|
||||
}
|
||||
// stop / tool-calls / max-tokens / plugin-added kinds → not a failure.
|
||||
default:
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the `{ message, code? }` part of an error payload, omitting the
|
||||
* `code` key entirely when absent (exactOptionalPropertyTypes-correct).
|
||||
@@ -266,6 +300,14 @@ async function runStep(
|
||||
assembler.push(chunk)
|
||||
}
|
||||
|
||||
// Adapters report provider/transport failures one of two sanctioned ways
|
||||
// (see the StreamChunk contract in dsh-llm): throw from stream() — already
|
||||
// handled by the caller's try/catch — OR end the stream with a
|
||||
// finish-error/aborted chunk. finishError() maps the latter to the step
|
||||
// error to raise (turn ends error/aborted, not a normal completed message).
|
||||
const stepError = finishError(assembler.finish)
|
||||
if (stepError) throw stepError
|
||||
|
||||
// The step-result waterfall runs BEFORE the session append so the log (the
|
||||
// source of truth for derived history and replay) records the message that
|
||||
// tool dispatch actually uses.
|
||||
|
||||
@@ -546,3 +546,67 @@ describe('LOW: discriminated SessionEvent narrows without casts', () => {
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('HIGH: a finish-error stream chunk ends the turn as error, not completed', () => {
|
||||
it('translates finish {kind:error} into a turn error with a logged error event', async () => {
|
||||
// The second sanctioned adapter error path (besides throwing): an
|
||||
// adapter that cannot throw mid-stream ends the stream with a
|
||||
// finish-error chunk (e.g. the pi-ai adapter mapping a provider 401).
|
||||
// The loop must NOT log a normal assistant/message + completed turn.
|
||||
const errorStream: StreamChunk[] = [
|
||||
{ type: 'finish', reason: { kind: 'error', message: 'provider 401', code: 'AUTH' } },
|
||||
]
|
||||
const adapter = new MockAdapter([errorStream])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create('a-finish-error', { model: 'mock' })
|
||||
|
||||
const reasons: TurnEndReason[] = []
|
||||
ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason))
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reasons).toEqual([{ kind: 'error', message: 'provider 401', code: 'AUTH' }])
|
||||
|
||||
const events = [...agent.session.events]
|
||||
expect(events.some(event => event.type === 'error'
|
||||
&& event.data.message === 'provider 401' && event.data.code === 'AUTH')).toBe(true)
|
||||
// Crucially: no assistant/message was logged for the failed step.
|
||||
expect(events.some(event => event.type === 'assistant/message')).toBe(false)
|
||||
})
|
||||
|
||||
it('translates finish {kind:aborted} into a turn error coded ABORTED', async () => {
|
||||
const abortedStream: StreamChunk[] = [
|
||||
{ type: 'finish', reason: { kind: 'aborted' } },
|
||||
]
|
||||
const adapter = new MockAdapter([abortedStream])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create('a-finish-aborted', { model: 'mock' })
|
||||
|
||||
const reasons: TurnEndReason[] = []
|
||||
ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason))
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reasons).toEqual([{ kind: 'error', message: 'model stream aborted', code: 'ABORTED' }])
|
||||
expect([...agent.session.events].some(event => event.type === 'assistant/message')).toBe(false)
|
||||
})
|
||||
|
||||
it('handles a finish error without a code (code key omitted)', async () => {
|
||||
const errorStream: StreamChunk[] = [
|
||||
{ type: 'finish', reason: { kind: 'error', message: 'codeless failure' } },
|
||||
]
|
||||
const adapter = new MockAdapter([errorStream])
|
||||
const ctx = await harness(adapter)
|
||||
const agent = ctx.agentLoop.create('a-finish-error-nocode', { model: 'mock' })
|
||||
|
||||
const reasons: TurnEndReason[] = []
|
||||
ctx.on('agent/turn-end', (_agent, _turn, reason) => void reasons.push(reason))
|
||||
|
||||
send(agent, 'go')
|
||||
await waitForIdle(ctx, agent)
|
||||
|
||||
expect(reasons).toEqual([{ kind: 'error', message: 'codeless failure' }])
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,83 @@
|
||||
# @deepseek-ai/dsh-llm-deepseek
|
||||
|
||||
DeepSeek chat-completions adapter for the harness LLM seam: hand-rolled
|
||||
`fetch` + SSE translation from the official wire format (source of truth:
|
||||
the API docs — guides/thinking_mode, guides/tool_calls,
|
||||
api/create-chat-completion) into the `StreamChunk` protocol.
|
||||
|
||||
A second, independent implementation of the same seam exists in
|
||||
`@deepseek-ai/dsh-llm-pi-ai` (library-backed). Same Config shape — pick one
|
||||
per context (registering both for the same model names throws by design).
|
||||
|
||||
## Config
|
||||
|
||||
```yaml
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
config:
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY # or rely on the env fallback
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL # default: https://api.deepseek.com
|
||||
models: [deepseek-v4-flash, deepseek-v4-pro] # one adapter, registered for each name
|
||||
thinking: enabled # optional; provider default is enabled
|
||||
reasoningEffort: high # optional; high | max — omitted ⇒ not sent
|
||||
```
|
||||
|
||||
`models` lists every model name this one adapter instance serves: the adapter
|
||||
registers itself for each (the harness model name IS the wire `model` string),
|
||||
so a `generate`/`stream` call routes to it whenever `options.model` is any of
|
||||
them. Registering a second adapter for a name already taken throws
|
||||
`LlmError('DUPLICATE_ADAPTER')` (the LLM service enforces one adapter per
|
||||
model, all-or-nothing).
|
||||
|
||||
`reasoningEffort` is **omitted by default** — when unset, the `reasoning_effort`
|
||||
wire field is not sent and the server applies its own default for the model.
|
||||
The only accepted values are `high` and `max` (DeepSeek's official effort
|
||||
levels). It is meaningful only with thinking enabled (the provider default).
|
||||
|
||||
`thinking`/`reasoningEffort` are adapter-level request defaults serialized as
|
||||
the official top-level `thinking: {type}` / `reasoning_effort` wire fields.
|
||||
They live in adapter config (not `GenerateOptions`) to keep the core
|
||||
vocabulary provider-neutral.
|
||||
|
||||
## Wire-format notes (verified live + against the official docs)
|
||||
|
||||
- Streaming only (`stream_options.include_usage` always on). `usage` may
|
||||
arrive attached to the finish chunk or as a trailing usage-only chunk —
|
||||
the translator defers both to `[DONE]`, so `usage` always precedes
|
||||
`finish` and nothing follows `finish`.
|
||||
- The first thinking-mode chunk carries `reasoning_content: ""` — handled
|
||||
(no spurious reasoning block).
|
||||
- **Reasoning passback rule**: on assistant turns that carried tool calls,
|
||||
`reasoning_content` is serialized back in history (required by the API in
|
||||
thinking mode); on tool-call-free turns it is dropped (ignored anyway —
|
||||
saves tokens).
|
||||
- `strict` on tool schemas passes through (officially Beta; the public API
|
||||
wants the `/beta` base URL for it, the internal endpoint accepts it
|
||||
directly).
|
||||
- Cache accounting: `cacheReadTokens` ← `prompt_cache_hit_tokens` /
|
||||
`prompt_tokens_details.cached_tokens`; DeepSeek reports no cache-write
|
||||
metric.
|
||||
|
||||
## Limitations (MVP, documented deliberately)
|
||||
|
||||
- `prefill` throws `LlmError('UNSUPPORTED')` — DeepSeek's chat-prefix
|
||||
completion is a Beta feature on the `/beta` base URL; future work.
|
||||
- `image` blocks are skipped (no vision support on these models).
|
||||
- `tool_choice` is not mapped (not part of the core vocabulary).
|
||||
|
||||
## Errors
|
||||
|
||||
Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403),
|
||||
`RATE_LIMIT` (429), `INVALID_REQUEST` (400), `SERVER` (5xx), `HTTP_<status>`
|
||||
otherwise. Protocol violations throw `STREAM_CLOSED` (no `[DONE]`) or
|
||||
`MALFORMED_RESPONSE` (bad JSON payload). Unknown wire `finish_reason`s
|
||||
(e.g. `content_filter`, `insufficient_system_resource`) become
|
||||
`finish {kind: 'error', code: <REASON>}` chunks.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit suites run against a local `node:http` mock SSE server (no network).
|
||||
Real-API coverage lives in `tests/adapter.e2e.ts` (`yarn test:e2e`,
|
||||
key-gated): V4 Flash + V4 Pro across thinking enabled/disabled and both
|
||||
official effort levels, including the thinking+tools round trip with
|
||||
reasoning passback.
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-llm-deepseek",
|
||||
"description": "DeepSeek chat-completions adapter for the DeepSeek Harness LLM seam",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* `DeepSeekAdapter`: fetch + SSE against a DeepSeek (OpenAI-compatible)
|
||||
* chat-completions endpoint, emitting harness StreamChunks.
|
||||
*
|
||||
* @module dsh-llm-deepseek/adapter
|
||||
*/
|
||||
|
||||
import { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { serializeRequest } from './serialize.ts'
|
||||
import type { RequestDefaults } from './serialize.ts'
|
||||
import { parseSse } from './sse.ts'
|
||||
import { translate } from './translate.ts'
|
||||
import type { WireError } from './types.ts'
|
||||
|
||||
export interface DeepSeekAdapterOptions {
|
||||
apiKey: string
|
||||
/** Endpoint base; `/chat/completions` is appended. */
|
||||
baseURL: string
|
||||
/** Request defaults applied to every call (thinking mode, effort). */
|
||||
defaults?: RequestDefaults
|
||||
}
|
||||
|
||||
/**
|
||||
* Attribution header sent on every request so the provider can identify the
|
||||
* client. Bump in lockstep with this package's version (no build-time version
|
||||
* injection is wired in this repo yet).
|
||||
*/
|
||||
const USER_AGENT = 'deepseek-harness/0.0.1'
|
||||
|
||||
/** Map an HTTP status to a stable LlmError code. */
|
||||
export function httpErrorCode(status: number): string {
|
||||
if (status === 401 || status === 403) return 'AUTH'
|
||||
if (status === 429) return 'RATE_LIMIT'
|
||||
if (status === 400) return 'INVALID_REQUEST'
|
||||
if (status >= 500) return 'SERVER'
|
||||
return `HTTP_${status}`
|
||||
}
|
||||
|
||||
/**
|
||||
* The first real `LlmAdapter`. One instance serves every model name it was
|
||||
* registered under (the harness model name IS the wire model name).
|
||||
*
|
||||
* Abort: `options.signal` is handed to fetch — both the initial request and
|
||||
* the body stream reject on abort, which surfaces to the loop as a rejected
|
||||
* step (the loop already contains step errors).
|
||||
*/
|
||||
export class DeepSeekAdapter extends LlmAdapter {
|
||||
constructor(private readonly options: DeepSeekAdapterOptions) {
|
||||
super()
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
const body = serializeRequest(options, this.options.defaults ?? {})
|
||||
|
||||
// TODO(http): deliberately raw `fetch` for the hand-rolled SSE body.
|
||||
// `@cordisjs/plugin-http` (ctx.http) would give proxy/intercept/timeout
|
||||
// uniformity AND can stream (`responseType: 'stream'` yields the same
|
||||
// ReadableStream<Uint8Array> parseSse consumes), but adopting it today
|
||||
// costs a hard `undici` dependency (it does `require('undici')` with no
|
||||
// globalThis.fetch fallback) plus an unconditional `@cordisjs/fetch-file`
|
||||
// import (pulling file-type + mime-types) for a file:// path we never hit.
|
||||
// Revisit when a second adapter wants shared proxy/intercept config.
|
||||
const response = await fetch(`${this.options.baseURL}/chat/completions`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'authorization': `Bearer ${this.options.apiKey}`,
|
||||
'content-type': 'application/json',
|
||||
'accept': 'text/event-stream',
|
||||
'user-agent': USER_AGENT,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
...options.signal ? { signal: options.signal } : {},
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const code = httpErrorCode(response.status)
|
||||
let message = `DeepSeek API error (HTTP ${response.status})`
|
||||
try {
|
||||
const parsed = await response.json() as WireError
|
||||
if (parsed.error?.message) message = parsed.error.message
|
||||
} catch {
|
||||
// Paranoid by design: `code` and the HTTP status are ALREADY captured
|
||||
// above (and passed to LlmError below), so the only thing this `try`
|
||||
// can add is a richer provider-supplied message. A malformed, empty,
|
||||
// or non-JSON error body is a normal thing for gateways/proxies to
|
||||
// return on a 5xx/429 — swallowing the parse failure keeps the usable
|
||||
// status-line message instead of letting a JSON.parse throw mask the
|
||||
// real HTTP error. Nothing else reaches this catch: response.json()
|
||||
// is the sole statement, and any non-parse failure (e.g. body already
|
||||
// consumed) is equally non-actionable here.
|
||||
}
|
||||
throw new LlmError(message, code, response.status)
|
||||
}
|
||||
if (!response.body) {
|
||||
throw new LlmError('DeepSeek API returned no response body', 'EMPTY_RESPONSE')
|
||||
}
|
||||
|
||||
yield* translate(parseSse(response.body))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* DeepSeek LLM adapter plugin: registers a {@link DeepSeekAdapter} for the
|
||||
* configured model names on `ctx.llm`.
|
||||
*
|
||||
* Config is cordis-native (schemastery). Secrets flow per the repo policy:
|
||||
* `apiKey` from cordis.yml via the `!!js` tag (`!!js process.env.DEEPSEEK_API_KEY`)
|
||||
* or from the environment directly; never from ad-hoc files.
|
||||
*
|
||||
* ```yaml
|
||||
* - id: llm-deepseek
|
||||
* name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
* config:
|
||||
* apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
* baseURL: !!js process.env.DEEPSEEK_BASE_URL
|
||||
* models: [deepseek-v4-flash, deepseek-v4-pro]
|
||||
* ```
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-deepseek
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type {} from '@deepseek-ai/dsh-llm'
|
||||
import { DeepSeekAdapter } from './adapter.ts'
|
||||
|
||||
export { DeepSeekAdapter, httpErrorCode } from './adapter.ts'
|
||||
export type { DeepSeekAdapterOptions } from './adapter.ts'
|
||||
export { serializeMessages, serializeRequest } from './serialize.ts'
|
||||
export type { RequestDefaults } from './serialize.ts'
|
||||
export { DONE, parseSse } from './sse.ts'
|
||||
export { mapFinishReason, mapUsage, translate } from './translate.ts'
|
||||
export type * from './types.ts'
|
||||
|
||||
export const name = 'llm-deepseek'
|
||||
export const inject = ['llm']
|
||||
|
||||
export interface Config {
|
||||
/** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */
|
||||
apiKey?: string
|
||||
/** Endpoint base; falls back to $DEEPSEEK_BASE_URL, then the public API. */
|
||||
baseURL?: string
|
||||
/** Model names to register (sent verbatim on the wire). */
|
||||
models?: string[]
|
||||
/** Thinking-mode default for every request (provider default: enabled). */
|
||||
thinking?: 'enabled' | 'disabled'
|
||||
/** Thinking effort (only meaningful with thinking enabled). */
|
||||
reasoningEffort?: 'high' | 'max'
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
apiKey: z.string(),
|
||||
baseURL: z.string(),
|
||||
models: z.array(z.string()).default(['deepseek-v4-flash', 'deepseek-v4-pro']),
|
||||
thinking: z.union(['enabled', 'disabled']),
|
||||
reasoningEffort: z.union(['high', 'max']),
|
||||
})
|
||||
|
||||
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
|
||||
export const PUBLIC_BASE_URL = 'https://api.deepseek.com'
|
||||
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
const apiKey = config.apiKey ?? process.env.DEEPSEEK_API_KEY
|
||||
if (apiKey === undefined || apiKey.length === 0) {
|
||||
throw new Error('llm-deepseek: an API key is required (Config.apiKey or $DEEPSEEK_API_KEY)')
|
||||
}
|
||||
const baseURL = config.baseURL ?? process.env.DEEPSEEK_BASE_URL ?? PUBLIC_BASE_URL
|
||||
// schemastery's .default() guarantees models is set after validation.
|
||||
const models = config.models as string[]
|
||||
|
||||
ctx.llm.registerAdapter(models, new DeepSeekAdapter({
|
||||
apiKey,
|
||||
baseURL,
|
||||
defaults: {
|
||||
thinking: config.thinking,
|
||||
reasoningEffort: config.reasoningEffort,
|
||||
},
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
/**
|
||||
* Serialize harness vocabulary (`GenerateOptions`, `Message[]`) into the
|
||||
* DeepSeek chat-completions request body.
|
||||
*
|
||||
* Block-type mapping (core types handled explicitly; merge-extensible unions
|
||||
* mean plugin-added block types exist — they are skipped, never errors):
|
||||
*
|
||||
* - user `text` → string content (joined)
|
||||
* - assistant `text` → `content`; `reasoning` → `reasoning_content`, but
|
||||
* ONLY on assistant messages that carry tool calls (the official passback
|
||||
* rule for thinking mode — required there, ignored elsewhere, so we save
|
||||
* the tokens elsewhere); `tool-call` → `tool_calls[]`
|
||||
* - `tool-result` → its own `{role: 'tool'}` message (text flattened)
|
||||
* - `image` → skipped (MVP limitation, documented in the README)
|
||||
*
|
||||
* @module dsh-llm-deepseek/serialize
|
||||
*/
|
||||
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { WireMessage, WireRequest, WireTool } from './types.ts'
|
||||
|
||||
/** Adapter-level request defaults (from plugin config). */
|
||||
export interface RequestDefaults {
|
||||
thinking?: 'enabled' | 'disabled' | undefined
|
||||
reasoningEffort?: 'high' | 'max' | undefined
|
||||
}
|
||||
|
||||
/** Join the text blocks of a message (used for user/tool-result content). */
|
||||
function flattenText(blocks: ContentBlock[]): string {
|
||||
return blocks
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
/** Serialize one assistant message (text + reasoning + tool calls). */
|
||||
function serializeAssistant(message: Message): WireMessage {
|
||||
const text = flattenText(message.content)
|
||||
const reasoning = message.content
|
||||
.filter(block => block.type === 'reasoning')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
const toolCalls = message.content
|
||||
.filter(block => block.type === 'tool-call')
|
||||
.map(block => ({
|
||||
id: block.id,
|
||||
type: 'function' as const,
|
||||
function: { name: block.name, arguments: block.arguments },
|
||||
}))
|
||||
|
||||
return {
|
||||
role: 'assistant',
|
||||
// Tool-call turns send "" rather than null: the live API answers both,
|
||||
// but the official samples replay message.content verbatim (which is ""
|
||||
// for pure tool-call responses) and some gateways reject null outright.
|
||||
content: text.length > 0 ? text : toolCalls.length > 0 ? '' : null,
|
||||
// Official passback rule (guides/thinking_mode.mdx): reasoning_content
|
||||
// must return on tool-call turns; it is ignored on plain turns, so we
|
||||
// drop it there to save tokens.
|
||||
...toolCalls.length > 0 && reasoning.length > 0 ? { reasoning_content: reasoning } : {},
|
||||
...toolCalls.length > 0 ? { tool_calls: toolCalls } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize the conversation. `tool-result` blocks become standalone
|
||||
* `{role: 'tool'}` messages; the harness puts each tool result in its own
|
||||
* user-role message, so a mixed user message contributes its text first and
|
||||
* its tool results as separate wire messages after.
|
||||
*/
|
||||
export function serializeMessages(messages: Message[]): WireMessage[] {
|
||||
const wire: WireMessage[] = []
|
||||
for (const message of messages) {
|
||||
if (message.role === 'system') {
|
||||
wire.push({ role: 'system', content: flattenText(message.content) })
|
||||
continue
|
||||
}
|
||||
if (message.role === 'assistant') {
|
||||
wire.push(serializeAssistant(message))
|
||||
continue
|
||||
}
|
||||
// user role: tool results ride in user messages in the harness
|
||||
// vocabulary, but DeepSeek wants them as role:'tool' messages.
|
||||
const toolResults = message.content.filter(block => block.type === 'tool-result')
|
||||
const text = flattenText(message.content)
|
||||
if (text.length > 0 || toolResults.length === 0) {
|
||||
wire.push({ role: 'user', content: text })
|
||||
}
|
||||
for (const result of toolResults) {
|
||||
wire.push({
|
||||
role: 'tool',
|
||||
tool_call_id: result.toolCallId,
|
||||
// Empty tool output still needs SOME content on the wire.
|
||||
content: flattenText(result.content) || '(no output)',
|
||||
})
|
||||
}
|
||||
}
|
||||
return wire
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the full wire request. Throws `LlmError('UNSUPPORTED')` for
|
||||
* `prefill` (DeepSeek's chat-prefix completion is a Beta feature on a
|
||||
* different base URL — see README).
|
||||
*/
|
||||
export function serializeRequest(options: GenerateOptions, defaults: RequestDefaults = {}): WireRequest {
|
||||
if (options.prefill !== undefined) {
|
||||
throw new LlmError(
|
||||
'prefill is not supported by the DeepSeek adapter (Beta chat-prefix completion is future work)',
|
||||
'UNSUPPORTED',
|
||||
)
|
||||
}
|
||||
|
||||
const messages: WireMessage[] = []
|
||||
if (options.system !== undefined) {
|
||||
messages.push({ role: 'system', content: options.system })
|
||||
}
|
||||
messages.push(...serializeMessages(options.messages))
|
||||
|
||||
const tools: WireTool[] | undefined = options.tools?.map(tool => ({
|
||||
type: 'function',
|
||||
function: {
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
parameters: tool.parameters,
|
||||
// strict is officially supported (Beta); pass the tool author's choice.
|
||||
...tool.strict !== undefined ? { strict: tool.strict } : {},
|
||||
},
|
||||
}))
|
||||
|
||||
return {
|
||||
model: options.model,
|
||||
messages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
...defaults.thinking !== undefined ? { thinking: { type: defaults.thinking } } : {},
|
||||
...defaults.reasoningEffort !== undefined ? { reasoning_effort: defaults.reasoningEffort } : {},
|
||||
...tools !== undefined && tools.length > 0 ? { tools } : {},
|
||||
...options.temperature !== undefined ? { temperature: options.temperature } : {},
|
||||
...options.maxTokens !== undefined ? { max_tokens: options.maxTokens } : {},
|
||||
...options.stop !== undefined ? { stop: options.stop } : {},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
/**
|
||||
* Minimal SSE (text/event-stream) parser for the chat-completions stream.
|
||||
*
|
||||
* Yields each event's `data:` payload as a string, ending with the literal
|
||||
* `'[DONE]'` sentinel so the consumer owns end-of-stream flushing. A stream
|
||||
* that closes WITHOUT `[DONE]` is a protocol violation → `LlmError`.
|
||||
*
|
||||
* Handles the wire realities: payloads split across network reads at
|
||||
* arbitrary byte positions (including mid-UTF-8), CRLF line endings,
|
||||
* multi-`data:` events (joined with newlines per the SSE spec), comment
|
||||
* lines, and non-data fields (ignored).
|
||||
*
|
||||
* @module dsh-llm-deepseek/sse
|
||||
*/
|
||||
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** The terminal payload DeepSeek (and OpenAI) send after the last chunk. */
|
||||
export const DONE = '[DONE]'
|
||||
|
||||
/** Extract the joined data payload from one raw SSE event block. */
|
||||
function eventData(block: string): string | undefined {
|
||||
const data: string[] = []
|
||||
for (const rawLine of block.split('\n')) {
|
||||
const line = rawLine.endsWith('\r') ? rawLine.slice(0, -1) : rawLine
|
||||
if (line.startsWith('data:')) {
|
||||
// The spec strips ONE leading space after the colon.
|
||||
data.push(line.startsWith('data: ') ? line.slice(6) : line.slice(5))
|
||||
}
|
||||
// Comments (':…') and other fields (event:, id:, retry:) are ignored.
|
||||
}
|
||||
if (data.length === 0) return undefined
|
||||
return data.join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a byte stream into SSE data payloads. Yields `[DONE]` as the final
|
||||
* value and returns; throws `LlmError('STREAM_CLOSED')` when the stream ends
|
||||
* without it (truncated response — the model call cannot be trusted).
|
||||
*/
|
||||
export async function* parseSse(stream: AsyncIterable<Uint8Array>): AsyncGenerator<string> {
|
||||
const decoder = new TextDecoder()
|
||||
let buffer = ''
|
||||
|
||||
for await (const bytes of stream) {
|
||||
buffer += decoder.decode(bytes, { stream: true })
|
||||
// Events are separated by a blank line (\n\n; tolerate \r\n\r\n via the
|
||||
// per-line \r strip in eventData and a normalized split here).
|
||||
let boundary: number
|
||||
while ((boundary = buffer.search(/\r?\n\r?\n/)) !== -1) {
|
||||
const matched = /\r?\n\r?\n/.exec(buffer.slice(boundary))
|
||||
const block = buffer.slice(0, boundary)
|
||||
// matched cannot be null: search() just found the same pattern at 0.
|
||||
buffer = buffer.slice(boundary + (matched as RegExpExecArray)[0].length)
|
||||
const data = eventData(block)
|
||||
if (data === undefined) continue
|
||||
yield data
|
||||
if (data === DONE) return
|
||||
}
|
||||
}
|
||||
|
||||
// Flush any final un-terminated event (servers usually end with \n\n, but
|
||||
// a trailing block without one is still parseable).
|
||||
buffer += decoder.decode()
|
||||
const data = eventData(buffer)
|
||||
if (data !== undefined) {
|
||||
yield data
|
||||
if (data === DONE) return
|
||||
}
|
||||
throw new LlmError('SSE stream ended without [DONE]', 'STREAM_CLOSED')
|
||||
}
|
||||
@@ -0,0 +1,169 @@
|
||||
/**
|
||||
* Translate DeepSeek wire chunks into the harness `StreamChunk` protocol.
|
||||
*
|
||||
* A small state machine over the SSE payload stream:
|
||||
* - `delta.content` / `delta.reasoning_content` / `delta.tool_calls[i]` each
|
||||
* own one harness block (index allocated on first sight). The first
|
||||
* thinking-mode chunk carries `reasoning_content: ""` — that must NOT open
|
||||
* a reasoning block.
|
||||
* - `finish_reason` and `usage` are DEFERRED: emitted only at the `[DONE]`
|
||||
* sentinel, so the wire's two usage shapes (attached to the finish chunk,
|
||||
* or a trailing usage-only chunk) both work and nothing ever follows
|
||||
* `finish`. Last usage wins.
|
||||
*
|
||||
* @module dsh-llm-deepseek/translate
|
||||
*/
|
||||
|
||||
import { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, FinishReason, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE } from './sse.ts'
|
||||
import type { WireChunk, WireUsage } from './types.ts'
|
||||
|
||||
/** One open block under assembly. */
|
||||
interface OpenBlock {
|
||||
index: number
|
||||
kind: 'text' | 'reasoning' | 'tool-call'
|
||||
text: string
|
||||
/** tool-call only */
|
||||
callId?: string
|
||||
name?: string
|
||||
}
|
||||
|
||||
/** Map the wire finish_reason vocabulary to the harness FinishReason. */
|
||||
export function mapFinishReason(reason: string): FinishReason {
|
||||
switch (reason) {
|
||||
case 'stop': return { kind: 'stop' }
|
||||
case 'tool_calls': return { kind: 'tool-calls' }
|
||||
case 'length': return { kind: 'max-tokens' }
|
||||
default:
|
||||
// content_filter, insufficient_system_resource, future additions.
|
||||
return { kind: 'error', message: `model stopped: ${reason}`, code: reason.toUpperCase() }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Map wire usage fields. DeepSeek's `prompt_tokens` INCLUDES cache hits
|
||||
* (`prompt_tokens = prompt_cache_hit_tokens + prompt_cache_miss_tokens`,
|
||||
* api/create-chat-completion); the harness TokenUsage convention is
|
||||
* DISJOINT counts, so cache reads are subtracted out of `inputTokens`.
|
||||
*/
|
||||
export function mapUsage(usage: WireUsage): TokenUsage {
|
||||
const cacheRead = usage.prompt_tokens_details?.cached_tokens ?? usage.prompt_cache_hit_tokens
|
||||
const reasoning = usage.completion_tokens_details?.reasoning_tokens
|
||||
return {
|
||||
inputTokens: usage.prompt_tokens - (cacheRead ?? 0),
|
||||
outputTokens: usage.completion_tokens,
|
||||
...cacheRead !== undefined ? { cacheReadTokens: cacheRead } : {},
|
||||
...reasoning !== undefined ? { reasoningTokens: reasoning } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/** Assemble the final ContentBlock for one open block. */
|
||||
function closeBlock(block: OpenBlock): ContentBlock {
|
||||
switch (block.kind) {
|
||||
case 'text': return { type: 'text', text: block.text }
|
||||
case 'reasoning': return { type: 'reasoning', text: block.text }
|
||||
case 'tool-call': return {
|
||||
type: 'tool-call',
|
||||
id: CallId(block.callId ?? ''),
|
||||
name: block.name ?? '',
|
||||
arguments: block.text,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Consume SSE data payloads (ending with `[DONE]`) and yield StreamChunks.
|
||||
* Malformed JSON payloads abort the stream with `MALFORMED_RESPONSE`.
|
||||
*/
|
||||
export async function* translate(payloads: AsyncIterable<string>): AsyncGenerator<StreamChunk> {
|
||||
let nextIndex = 0
|
||||
let textBlock: OpenBlock | undefined
|
||||
let reasoningBlock: OpenBlock | undefined
|
||||
const toolBlocks = new Map<number, OpenBlock>()
|
||||
const order: OpenBlock[] = []
|
||||
let pendingFinish: FinishReason | undefined
|
||||
let pendingUsage: TokenUsage | undefined
|
||||
|
||||
function open(kind: OpenBlock['kind']): OpenBlock {
|
||||
const block: OpenBlock = { index: nextIndex++, kind, text: '' }
|
||||
order.push(block)
|
||||
return block
|
||||
}
|
||||
|
||||
for await (const payload of payloads) {
|
||||
if (payload === DONE) {
|
||||
for (const block of order) {
|
||||
yield { type: 'block-end', index: block.index, block: closeBlock(block) }
|
||||
}
|
||||
if (pendingUsage) yield { type: 'usage', usage: pendingUsage }
|
||||
yield { type: 'finish', reason: pendingFinish ?? { kind: 'stop' } }
|
||||
return
|
||||
}
|
||||
|
||||
let chunk: WireChunk
|
||||
try {
|
||||
chunk = JSON.parse(payload) as WireChunk
|
||||
} catch {
|
||||
throw new LlmError(`malformed SSE payload: ${payload.slice(0, 120)}`, 'MALFORMED_RESPONSE')
|
||||
}
|
||||
|
||||
for (const choice of chunk.choices ?? []) {
|
||||
const delta = choice.delta
|
||||
|
||||
// Reasoning first: thinking mode interleaves it before text. The
|
||||
// empty-string first chunk must not open a block.
|
||||
const reasoning = delta?.reasoning_content
|
||||
if (typeof reasoning === 'string' && reasoning.length > 0) {
|
||||
if (!reasoningBlock) {
|
||||
reasoningBlock = open('reasoning')
|
||||
yield { type: 'block-start', index: reasoningBlock.index, blockType: 'reasoning' }
|
||||
}
|
||||
reasoningBlock.text += reasoning
|
||||
yield { type: 'reasoning-delta', index: reasoningBlock.index, text: reasoning }
|
||||
}
|
||||
|
||||
const content = delta?.content
|
||||
if (typeof content === 'string' && content.length > 0) {
|
||||
if (!textBlock) {
|
||||
textBlock = open('text')
|
||||
yield { type: 'block-start', index: textBlock.index, blockType: 'text' }
|
||||
}
|
||||
textBlock.text += content
|
||||
yield { type: 'text-delta', index: textBlock.index, text: content }
|
||||
}
|
||||
|
||||
for (const call of delta?.tool_calls ?? []) {
|
||||
let block = toolBlocks.get(call.index)
|
||||
if (!block) {
|
||||
block = open('tool-call')
|
||||
toolBlocks.set(call.index, block)
|
||||
yield { type: 'block-start', index: block.index, blockType: 'tool-call' }
|
||||
}
|
||||
if (call.id !== undefined) block.callId = call.id
|
||||
if (call.function?.name !== undefined) block.name = call.function.name
|
||||
const fragment = call.function?.arguments ?? ''
|
||||
block.text += fragment
|
||||
yield {
|
||||
type: 'tool-call-delta',
|
||||
index: block.index,
|
||||
id: CallId(block.callId ?? ''),
|
||||
...block.name !== undefined ? { name: block.name } : {},
|
||||
argumentsDelta: fragment,
|
||||
}
|
||||
}
|
||||
|
||||
if (typeof choice.finish_reason === 'string') {
|
||||
pendingFinish = mapFinishReason(choice.finish_reason)
|
||||
}
|
||||
}
|
||||
|
||||
// Usage may arrive attached to the finish chunk or as a trailing
|
||||
// usage-only chunk — keep the latest.
|
||||
if (chunk.usage) pendingUsage = mapUsage(chunk.usage)
|
||||
}
|
||||
|
||||
// parseSse guarantees the [DONE] sentinel (or throws); reaching here means
|
||||
// the payload source violated that contract.
|
||||
throw new LlmError('SSE payload stream ended without [DONE]', 'STREAM_CLOSED')
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
/**
|
||||
* DeepSeek chat-completions wire format (OpenAI-compatible). Types only.
|
||||
*
|
||||
* Source of truth: the official API docs at
|
||||
* `~/repos/deepsuite-docs/apps/docs/docs` (api/create-chat-completion,
|
||||
* guides/thinking_mode.mdx, guides/tool_calls.md), cross-checked against
|
||||
* live streams from the internal endpoint (2026-06).
|
||||
*
|
||||
* @module dsh-llm-deepseek/types
|
||||
*/
|
||||
|
||||
/** Request body for `POST {baseURL}/chat/completions`. */
|
||||
export interface WireRequest {
|
||||
model: string
|
||||
messages: WireMessage[]
|
||||
stream: true
|
||||
stream_options: { include_usage: true }
|
||||
/** Thinking-mode toggle (top level, NOT inside extra_body on the wire). */
|
||||
thinking?: { type: 'enabled' | 'disabled' }
|
||||
/** Thinking effort (official levels; low/medium map to high server-side). */
|
||||
reasoning_effort?: 'high' | 'max'
|
||||
tools?: WireTool[]
|
||||
temperature?: number
|
||||
max_tokens?: number
|
||||
/**
|
||||
* Stop sequences (OpenAI `stop`): generation halts as soon as the model
|
||||
* produces any one of these strings. Mapped from `GenerateOptions.stop`.
|
||||
*/
|
||||
stop?: string[]
|
||||
}
|
||||
|
||||
/** System-role message: a single string of instructions. */
|
||||
export interface WireSystemMessage {
|
||||
role: 'system'
|
||||
content: string
|
||||
}
|
||||
|
||||
/** User-role message: a single string of user input. */
|
||||
export interface WireUserMessage {
|
||||
role: 'user'
|
||||
content: string
|
||||
}
|
||||
|
||||
/** Tool-role message: the result of one tool call, keyed by its call id. */
|
||||
export interface WireToolMessage {
|
||||
role: 'tool'
|
||||
tool_call_id: string
|
||||
content: string
|
||||
}
|
||||
|
||||
export type WireMessage =
|
||||
| WireSystemMessage
|
||||
| WireUserMessage
|
||||
| WireAssistantMessage
|
||||
| WireToolMessage
|
||||
|
||||
export interface WireAssistantMessage {
|
||||
role: 'assistant'
|
||||
content: string | null
|
||||
/**
|
||||
* CoT passback. REQUIRED on assistant turns that carried tool calls
|
||||
* (thinking mode); ignored on tool-call-free turns (we omit it there to
|
||||
* save tokens). See guides/thinking_mode.mdx § Tool Calls.
|
||||
*/
|
||||
reasoning_content?: string
|
||||
tool_calls?: WireToolCall[]
|
||||
}
|
||||
|
||||
export interface WireToolCall {
|
||||
id: string
|
||||
type: 'function'
|
||||
function: { name: string; arguments: string }
|
||||
}
|
||||
|
||||
export interface WireTool {
|
||||
type: 'function'
|
||||
function: {
|
||||
name: string
|
||||
description: string
|
||||
parameters: Record<string, unknown>
|
||||
/** Beta: strict schema adherence (official: requires the /beta base URL). */
|
||||
strict?: boolean
|
||||
}
|
||||
}
|
||||
|
||||
/** One parsed SSE `data:` payload (a chat.completion.chunk). */
|
||||
export interface WireChunk {
|
||||
choices?: WireChoice[]
|
||||
/** Arrives attached to the finish chunk and/or as a trailing usage-only chunk. */
|
||||
usage?: WireUsage | null
|
||||
}
|
||||
|
||||
export interface WireChoice {
|
||||
delta?: WireDelta
|
||||
finish_reason?: string | null
|
||||
}
|
||||
|
||||
export interface WireDelta {
|
||||
role?: string
|
||||
/** Visible text. Null/empty on reasoning/tool-call chunks. */
|
||||
content?: string | null
|
||||
/**
|
||||
* Thinking-mode CoT. The FIRST chunk carries an empty string (must not
|
||||
* open a reasoning block); absent entirely in non-thinking mode.
|
||||
*/
|
||||
reasoning_content?: string | null
|
||||
tool_calls?: WireToolCallDelta[]
|
||||
}
|
||||
|
||||
export interface WireToolCallDelta {
|
||||
/** Disambiguates parallel tool calls; stable across a call's deltas. */
|
||||
index: number
|
||||
/** Present on the first delta of each call only. */
|
||||
id?: string
|
||||
type?: 'function'
|
||||
function?: {
|
||||
/** Present on the first delta of each call only. */
|
||||
name?: string
|
||||
/** Argument JSON fragment (concatenate across deltas). */
|
||||
arguments?: string
|
||||
}
|
||||
}
|
||||
|
||||
export interface WireUsage {
|
||||
prompt_tokens: number
|
||||
completion_tokens: number
|
||||
prompt_cache_hit_tokens?: number
|
||||
prompt_cache_miss_tokens?: number
|
||||
prompt_tokens_details?: { cached_tokens?: number }
|
||||
completion_tokens_details?: { reasoning_tokens?: number }
|
||||
}
|
||||
|
||||
/** Non-2xx error body. */
|
||||
export interface WireError {
|
||||
error?: { message?: string; type?: string; code?: string }
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateResult, Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import type { Config } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
/**
|
||||
* Real-API e2e for the hand-rolled adapter: V4 Flash + V4 Pro across
|
||||
* thinking modes and both official effort levels. Key-gated — skips
|
||||
* entirely without $DEEPSEEK_API_KEY (see vitest.e2e.config.ts).
|
||||
*/
|
||||
|
||||
const FLASH = 'deepseek-v4-flash'
|
||||
const PRO = 'deepseek-v4-pro'
|
||||
|
||||
async function harness(model: string, config: Partial<Config> = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { models: [model], ...config })
|
||||
return ctx
|
||||
}
|
||||
|
||||
function ask(text: string): Message[] {
|
||||
return [{ role: 'user', content: [{ type: 'text', text }] }]
|
||||
}
|
||||
|
||||
function textOf(result: GenerateResult): string {
|
||||
return result.message.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
const weatherTool: ToolSchema = {
|
||||
name: 'get_weather',
|
||||
description: 'Get the current weather for a city.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: { city: { type: 'string', description: 'City name' } },
|
||||
required: ['city'],
|
||||
},
|
||||
}
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-deepseek e2e (real API)', () => {
|
||||
it('flash + thinking disabled: plain text generation', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'disabled' })
|
||||
const result = await ctx.llm.generate({
|
||||
model: FLASH,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(textOf(result).toLowerCase()).toContain('pong')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(false)
|
||||
expect(result.usage?.inputTokens).toBeGreaterThan(0)
|
||||
expect(result.usage?.outputTokens).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it('flash + thinking enabled (effort high): reasoning blocks + reasoning tokens', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'enabled', reasoningEffort: 'high' })
|
||||
const result = await ctx.llm.generate({
|
||||
model: FLASH,
|
||||
messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'),
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(true)
|
||||
expect(textOf(result)).toContain('9.8')
|
||||
expect(result.usage?.reasoningTokens).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it.each(['high', 'max'] as const)(
|
||||
'pro + thinking enabled (effort %s): tool-call round trip with reasoning passback',
|
||||
async (effort) => {
|
||||
const ctx = await harness(PRO, { thinking: 'enabled', reasoningEffort: effort })
|
||||
|
||||
// Turn 1: the model must call the tool (and think before it).
|
||||
const first = await ctx.llm.generate({
|
||||
model: PRO,
|
||||
messages: ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
tools: [weatherTool],
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(first.finish.kind).toBe('tool-calls')
|
||||
const call = first.message.content.find(block => block.type === 'tool-call')
|
||||
expect(call).toBeDefined()
|
||||
expect(call!.name).toBe('get_weather')
|
||||
expect(JSON.parse(call!.arguments)).toMatchObject({ city: expect.stringMatching(/paris/i) as string })
|
||||
|
||||
// Turn 2: send the tool result back WITH the assistant's reasoning
|
||||
// block in history (the official thinking+tools passback rule).
|
||||
const second = await ctx.llm.generate({
|
||||
model: PRO,
|
||||
messages: [
|
||||
...ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
{ role: 'assistant', content: first.message.content },
|
||||
{
|
||||
role: 'user',
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: CallId(call!.id),
|
||||
content: [{ type: 'text', text: 'Sunny, 22°C' }],
|
||||
}],
|
||||
},
|
||||
],
|
||||
tools: [weatherTool],
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(second.finish.kind).toBe('stop')
|
||||
expect(textOf(second).toLowerCase()).toMatch(/sunny|22/)
|
||||
},
|
||||
)
|
||||
|
||||
it('pro + thinking disabled: plain generation without reasoning blocks', async () => {
|
||||
const ctx = await harness(PRO, { thinking: 'disabled' })
|
||||
const result = await ctx.llm.generate({
|
||||
model: PRO,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(false)
|
||||
})
|
||||
|
||||
it('streams raw chunks in protocol order', async () => {
|
||||
const ctx = await harness(FLASH, { thinking: 'disabled' })
|
||||
const kinds: string[] = []
|
||||
for await (const chunk of ctx.llm.stream({
|
||||
model: FLASH,
|
||||
messages: ask('Count from 1 to 5, digits only.'),
|
||||
maxTokens: 50,
|
||||
})) {
|
||||
kinds.push(chunk.type)
|
||||
}
|
||||
expect(kinds[0]).toBe('block-start')
|
||||
expect(kinds.at(-1)).toBe('finish')
|
||||
expect(kinds.filter(kind => kind === 'finish')).toHaveLength(1)
|
||||
// usage precedes finish (deferred-emit contract)
|
||||
expect(kinds.indexOf('usage')).toBeLessThan(kinds.indexOf('finish'))
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,309 @@
|
||||
import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { DeepSeekAdapter, httpErrorCode } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
/** One scripted behavior for the next request the mock server receives. */
|
||||
type Behavior =
|
||||
| { kind: 'sse'; events: string[]; delayMs?: number }
|
||||
| { kind: 'http-error'; status: number; body: string; contentType?: string }
|
||||
| { kind: 'close-early'; events: string[] }
|
||||
|
||||
interface MockServer {
|
||||
url: string
|
||||
/** Bodies of received requests, in order. */
|
||||
requests: unknown[]
|
||||
/** Header bags of received requests, in order (parallel to `requests`). */
|
||||
headers: IncomingMessage['headers'][]
|
||||
script: Behavior[]
|
||||
close(): Promise<void>
|
||||
}
|
||||
|
||||
const servers: Server[] = []
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
|
||||
vi.unstubAllEnvs()
|
||||
})
|
||||
|
||||
/** Local chat-completions stand-in: replays scripted behaviors per request. */
|
||||
async function mockServer(script: Behavior[]): Promise<MockServer> {
|
||||
const requests: unknown[] = []
|
||||
const headers: IncomingMessage['headers'][] = []
|
||||
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
|
||||
let body = ''
|
||||
request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') })
|
||||
request.on('end', () => {
|
||||
requests.push(JSON.parse(body))
|
||||
headers.push(request.headers)
|
||||
const behavior = script.shift()
|
||||
if (!behavior) {
|
||||
response.writeHead(500).end('mock script exhausted')
|
||||
return
|
||||
}
|
||||
if (behavior.kind === 'http-error') {
|
||||
response.writeHead(behavior.status, { 'content-type': behavior.contentType ?? 'application/json' })
|
||||
response.end(behavior.body)
|
||||
return
|
||||
}
|
||||
response.writeHead(200, { 'content-type': 'text/event-stream' })
|
||||
const write = (index: number): void => {
|
||||
if (index >= behavior.events.length) {
|
||||
if (behavior.kind === 'sse') response.end()
|
||||
else response.destroy() // close-early: drop the socket mid-stream
|
||||
return
|
||||
}
|
||||
response.write(`data: ${behavior.events[index]}\n\n`)
|
||||
setTimeout(() => { write(index + 1) }, behavior.kind === 'sse' ? behavior.delayMs ?? 0 : 5)
|
||||
}
|
||||
write(0)
|
||||
})
|
||||
})
|
||||
servers.push(server)
|
||||
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
|
||||
const address = server.address()
|
||||
if (address === null || typeof address === 'string') throw new Error('no port')
|
||||
return {
|
||||
url: `http://127.0.0.1:${address.port}`,
|
||||
requests,
|
||||
headers,
|
||||
script,
|
||||
close: () => new Promise(resolve => server.close(() => { resolve() })),
|
||||
}
|
||||
}
|
||||
|
||||
const textEvents = [
|
||||
'{"choices":[{"delta":{"role":"assistant","content":null,"reasoning_content":""}}]}',
|
||||
'{"choices":[{"delta":{"content":"hello"}}]}',
|
||||
'{"choices":[{"delta":{"content":""},"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
|
||||
'[DONE]',
|
||||
]
|
||||
|
||||
async function harness(baseURL: string, config: object = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { apiKey: 'test-key', baseURL, models: ['deepseek-v4-flash'], ...config })
|
||||
return ctx
|
||||
}
|
||||
|
||||
describe('DeepSeekAdapter against a mock server', () => {
|
||||
it('streams a text generation end to end through ctx.llm.generate', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
expect(result.usage).toEqual({ inputTokens: 3, outputTokens: 1 })
|
||||
|
||||
// The wire request carried the auth header contents we configured.
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
model: 'deepseek-v4-flash',
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
})
|
||||
// Attribution header identifies the harness to the provider.
|
||||
expect(server.headers[0]?.['user-agent']).toMatch(/^deepseek-harness\//)
|
||||
})
|
||||
|
||||
it('streams raw chunks through ctx.llm.stream', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents, delayMs: 2 }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const kinds: string[] = []
|
||||
for await (const chunk of ctx.llm.stream({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})) {
|
||||
kinds.push(chunk.type)
|
||||
}
|
||||
expect(kinds).toEqual(['block-start', 'text-delta', 'block-end', 'usage', 'finish'])
|
||||
})
|
||||
|
||||
it('forwards thinking config onto the wire', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url, { thinking: 'disabled', reasoningEffort: 'high' })
|
||||
|
||||
await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
thinking: { type: 'disabled' },
|
||||
reasoning_effort: 'high',
|
||||
})
|
||||
})
|
||||
|
||||
it.each([
|
||||
[401, 'AUTH'],
|
||||
[403, 'AUTH'],
|
||||
[429, 'RATE_LIMIT'],
|
||||
[400, 'INVALID_REQUEST'],
|
||||
[500, 'SERVER'],
|
||||
[503, 'SERVER'],
|
||||
])('maps HTTP %d to LlmError code %s with the body message', async (status, code) => {
|
||||
const behavior: Behavior = {
|
||||
kind: 'http-error',
|
||||
status,
|
||||
body: JSON.stringify({ error: { message: `failed with ${status}`, type: 't', code: 'c' } }),
|
||||
}
|
||||
const server = await mockServer([behavior, behavior, behavior])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(`failed with ${status}`)
|
||||
await expect(
|
||||
ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
.catch((error: unknown) => (error as LlmError).code),
|
||||
).resolves.toBe(code)
|
||||
// The numeric HTTP status is carried on the error for explicit handling.
|
||||
await expect(
|
||||
ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
.catch((error: unknown) => (error as LlmError).status),
|
||||
).resolves.toBe(status)
|
||||
})
|
||||
|
||||
it('keeps the status-line message for JSON error bodies without a message', async () => {
|
||||
const server = await mockServer([{ kind: 'http-error', status: 500, body: '{"error":{"type":"x"}}' }])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/HTTP 500/)
|
||||
})
|
||||
|
||||
it('keeps the status-line message for non-JSON error bodies', async () => {
|
||||
const server = await mockServer([{ kind: 'http-error', status: 502, body: 'Bad Gateway', contentType: 'text/plain' }])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/HTTP 502/)
|
||||
})
|
||||
|
||||
it('maps unusual statuses to HTTP_<status>', () => {
|
||||
expect(httpErrorCode(418)).toBe('HTTP_418')
|
||||
})
|
||||
|
||||
it('throws EMPTY_RESPONSE when the response has no body', async () => {
|
||||
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue(
|
||||
new Response(null, { status: 200 }),
|
||||
)
|
||||
try {
|
||||
const iterate = async (): Promise<void> => {
|
||||
for await (const _chunk of adapter.stream({ model: 'm', messages: [] })) { /* drain */ }
|
||||
}
|
||||
await expect(iterate()).rejects.toThrow(/no response body/)
|
||||
} finally {
|
||||
fetchSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects with STREAM_CLOSED when the server drops mid-stream', async () => {
|
||||
const server = await mockServer([{
|
||||
kind: 'close-early',
|
||||
events: ['{"choices":[{"delta":{"content":"par"}}]}'],
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
await expect(ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] }))
|
||||
.rejects.toThrow(/terminated|socket|without \[DONE\]/)
|
||||
})
|
||||
|
||||
it('aborts mid-stream via the request signal', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents, delayMs: 50 }])
|
||||
const ctx = await harness(server.url)
|
||||
const controller = new AbortController()
|
||||
|
||||
const pending = (async () => {
|
||||
const chunks = []
|
||||
for await (const chunk of ctx.llm.stream({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
signal: controller.signal,
|
||||
})) {
|
||||
chunks.push(chunk)
|
||||
}
|
||||
return chunks
|
||||
})()
|
||||
|
||||
setTimeout(() => { controller.abort() }, 30)
|
||||
await expect(pending).rejects.toThrow()
|
||||
})
|
||||
})
|
||||
|
||||
describe('plugin registration and config', () => {
|
||||
it('registers the configured models and unregisters on dispose (HMR safety)', async () => {
|
||||
const server = await mockServer([])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const fiber = await ctx.plugin(LlmDeepSeek, {
|
||||
apiKey: 'k',
|
||||
baseURL: server.url,
|
||||
models: ['deepseek-v4-flash', 'deepseek-v4-pro'],
|
||||
})
|
||||
expect(ctx.llm.models().sort()).toEqual(['deepseek-v4-flash', 'deepseek-v4-pro'])
|
||||
await fiber.dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
|
||||
it('defaults the model list', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
expect(ctx.llm.models().sort()).toEqual(['deepseek-v4-flash', 'deepseek-v4-pro'])
|
||||
})
|
||||
|
||||
it('falls back to DEEPSEEK_API_KEY and DEEPSEEK_BASE_URL env vars', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'env-key')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', 'http://127.0.0.1:1')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, {})
|
||||
expect(ctx.llm.models().length).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it('throws a clear error when no API key is available', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', '')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmDeepSeek, {}))
|
||||
.rejects.toThrow(/an API key is required/)
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
|
||||
it('prefers explicit config over env for key and base URL', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'env-key')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', 'http://env-host:1')
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const ctx = await harness(server.url) // harness passes explicit config
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests).toHaveLength(1) // hit the explicit URL, not env
|
||||
})
|
||||
|
||||
it('uses DEEPSEEK_BASE_URL when config omits baseURL', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', server.url)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmDeepSeek, { apiKey: 'k', models: ['deepseek-v4-flash'] })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('defaults to the public base URL without config or env', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'k')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', undefined)
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
// Registration succeeds; no call is made (would hit api.deepseek.com).
|
||||
await ctx.plugin(LlmDeepSeek, {})
|
||||
expect(ctx.llm.models().length).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it('adapter is constructible directly for embedding', () => {
|
||||
const adapter = new DeepSeekAdapter({ apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
expect(adapter).toBeInstanceOf(DeepSeekAdapter)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,210 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import { serializeMessages, serializeRequest } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
function request(overrides: Partial<GenerateOptions> = {}): GenerateOptions {
|
||||
return { model: 'deepseek-v4-flash', messages: [], ...overrides }
|
||||
}
|
||||
|
||||
describe('serializeMessages', () => {
|
||||
it('maps user text to string content', () => {
|
||||
const wire = serializeMessages([
|
||||
{ role: 'user', content: [{ type: 'text', text: 'hello ' }, { type: 'text', text: 'world' }] },
|
||||
])
|
||||
expect(wire).toEqual([{ role: 'user', content: 'hello world' }])
|
||||
})
|
||||
|
||||
it('maps system-role messages in history', () => {
|
||||
const wire = serializeMessages([
|
||||
{ role: 'system', content: [{ type: 'text', text: 'be brief' }] },
|
||||
])
|
||||
expect(wire).toEqual([{ role: 'system', content: 'be brief' }])
|
||||
})
|
||||
|
||||
it('maps plain assistant text without reasoning_content', () => {
|
||||
const wire = serializeMessages([
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'reasoning', text: 'thinking…' },
|
||||
{ type: 'text', text: 'answer' },
|
||||
],
|
||||
},
|
||||
])
|
||||
// Tool-call-free turn: reasoning is dropped (ignored by the API anyway).
|
||||
expect(wire).toEqual([{ role: 'assistant', content: 'answer' }])
|
||||
})
|
||||
|
||||
it('passes reasoning_content back on tool-call turns (official passback rule)', () => {
|
||||
const wire = serializeMessages([
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'reasoning', text: 'I should check the weather.' },
|
||||
{ type: 'tool-call', id: CallId('call-1'), name: 'get_weather', arguments: '{"city":"Paris"}' },
|
||||
],
|
||||
},
|
||||
])
|
||||
expect(wire).toEqual([{
|
||||
role: 'assistant',
|
||||
// "" (not null) on tool-call turns — mirrors the official samples'
|
||||
// verbatim message replay; some gateways reject null.
|
||||
content: '',
|
||||
reasoning_content: 'I should check the weather.',
|
||||
tool_calls: [{ id: 'call-1', type: 'function', function: { name: 'get_weather', arguments: '{"city":"Paris"}' } }],
|
||||
}])
|
||||
})
|
||||
|
||||
it('serializes parallel tool calls in order', () => {
|
||||
const wire = serializeMessages([
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'tool-call', id: CallId('a'), name: 'one', arguments: '{}' },
|
||||
{ type: 'tool-call', id: CallId('b'), name: 'two', arguments: '{}' },
|
||||
],
|
||||
},
|
||||
])
|
||||
const assistant = wire[0] as { tool_calls: { id: string }[] }
|
||||
expect(assistant.tool_calls.map(call => call.id)).toEqual(['a', 'b'])
|
||||
})
|
||||
|
||||
it('turns tool results into role:tool messages', () => {
|
||||
const wire = serializeMessages([
|
||||
{
|
||||
role: 'user',
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: CallId('call-1'),
|
||||
content: [{ type: 'text', text: 'Sunny 22C' }],
|
||||
}],
|
||||
},
|
||||
])
|
||||
expect(wire).toEqual([{ role: 'tool', tool_call_id: 'call-1', content: 'Sunny 22C' }])
|
||||
})
|
||||
|
||||
it('sends a sentinel for empty tool-result content', () => {
|
||||
const wire = serializeMessages([
|
||||
{
|
||||
role: 'user',
|
||||
content: [{ type: 'tool-result', toolCallId: CallId('call-1'), content: [] }],
|
||||
},
|
||||
])
|
||||
expect(wire).toEqual([{ role: 'tool', tool_call_id: 'call-1', content: '(no output)' }])
|
||||
})
|
||||
|
||||
it('splits mixed user text + tool results into separate wire messages', () => {
|
||||
const wire = serializeMessages([
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: 'context note' },
|
||||
{ type: 'tool-result', toolCallId: CallId('call-1'), content: [{ type: 'text', text: 'ok' }] },
|
||||
],
|
||||
},
|
||||
])
|
||||
expect(wire).toEqual([
|
||||
{ role: 'user', content: 'context note' },
|
||||
{ role: 'tool', tool_call_id: 'call-1', content: 'ok' },
|
||||
])
|
||||
})
|
||||
|
||||
it('skips image blocks (documented MVP limitation)', () => {
|
||||
const wire = serializeMessages([
|
||||
{ role: 'user', content: [{ type: 'image', url: 'data:image/png;base64,x' }, { type: 'text', text: 'see image' }] },
|
||||
])
|
||||
expect(wire).toEqual([{ role: 'user', content: 'see image' }])
|
||||
})
|
||||
|
||||
it('emits an empty user message rather than dropping block-less messages', () => {
|
||||
const wire = serializeMessages([{ role: 'user', content: [] }])
|
||||
expect(wire).toEqual([{ role: 'user', content: '' }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('serializeRequest', () => {
|
||||
const history: Message[] = [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }]
|
||||
|
||||
it('always streams with usage and maps the basics', () => {
|
||||
const wire = serializeRequest(request({ messages: history }))
|
||||
expect(wire).toEqual({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: 'hi' }],
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
})
|
||||
})
|
||||
|
||||
it('prepends the system prompt', () => {
|
||||
const wire = serializeRequest(request({ messages: history, system: 'be helpful' }))
|
||||
expect(wire.messages[0]).toEqual({ role: 'system', content: 'be helpful' })
|
||||
expect(wire.messages[1]).toEqual({ role: 'user', content: 'hi' })
|
||||
})
|
||||
|
||||
it('maps sampling params and stop sequences', () => {
|
||||
const wire = serializeRequest(request({ messages: history, temperature: 0.2, maxTokens: 100, stop: ['END'] }))
|
||||
expect(wire.temperature).toBe(0.2)
|
||||
expect(wire.max_tokens).toBe(100)
|
||||
expect(wire.stop).toEqual(['END'])
|
||||
})
|
||||
|
||||
it('maps tools with strict passthrough', () => {
|
||||
const wire = serializeRequest(request({
|
||||
messages: history,
|
||||
tools: [
|
||||
{ name: 'a', description: 'A', parameters: { type: 'object', properties: {} } },
|
||||
{ name: 'b', description: 'B', parameters: { type: 'object', properties: {} }, strict: true },
|
||||
],
|
||||
}))
|
||||
expect(wire.tools).toEqual([
|
||||
{ type: 'function', function: { name: 'a', description: 'A', parameters: { type: 'object', properties: {} } } },
|
||||
{ type: 'function', function: { name: 'b', description: 'B', parameters: { type: 'object', properties: {} }, strict: true } },
|
||||
])
|
||||
})
|
||||
|
||||
it('omits an empty tools array', () => {
|
||||
const wire = serializeRequest(request({ messages: history, tools: [] }))
|
||||
expect(wire.tools).toBeUndefined()
|
||||
})
|
||||
|
||||
it('applies adapter defaults for thinking and effort', () => {
|
||||
const wire = serializeRequest(request({ messages: history }), { thinking: 'enabled', reasoningEffort: 'max' })
|
||||
expect(wire.thinking).toEqual({ type: 'enabled' })
|
||||
expect(wire.reasoning_effort).toBe('max')
|
||||
})
|
||||
|
||||
it('omits thinking fields when unset (provider default applies)', () => {
|
||||
const wire = serializeRequest(request({ messages: history }))
|
||||
expect(wire.thinking).toBeUndefined()
|
||||
expect(wire.reasoning_effort).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects prefill with an UNSUPPORTED LlmError', () => {
|
||||
expect(() => serializeRequest(request({ prefill: [{ type: 'text', text: 'Sure' }] })))
|
||||
.toThrow(LlmError)
|
||||
try {
|
||||
serializeRequest(request({ prefill: [] }))
|
||||
expect.unreachable()
|
||||
} catch (error) {
|
||||
expect((error as LlmError).code).toBe('UNSUPPORTED')
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('review fixes: assistant content shapes', () => {
|
||||
it('serializes a content-less, tool-call-less assistant message as null content', () => {
|
||||
// Aborted/empty assistant turns: no text, no calls → null (the wire
|
||||
// accepts it; "" is reserved for tool-call turns per the samples).
|
||||
const wire = serializeMessages([{ role: 'assistant', content: [] }])
|
||||
expect(wire).toEqual([{ role: 'assistant', content: null }])
|
||||
})
|
||||
|
||||
it('serializes tool-call turns with empty string content, not null', () => {
|
||||
const wire = serializeMessages([{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'tool-call', id: CallId('c'), name: 'f', arguments: '{}' }],
|
||||
}])
|
||||
expect(wire[0]).toMatchObject({ content: '' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,108 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE, parseSse } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
/** Build a byte stream from string fragments (fragments = network reads). */
|
||||
async function* bytes(...fragments: (string | Uint8Array)[]): AsyncGenerator<Uint8Array> {
|
||||
const encoder = new TextEncoder()
|
||||
for (const fragment of fragments) {
|
||||
yield typeof fragment === 'string' ? encoder.encode(fragment) : fragment
|
||||
}
|
||||
}
|
||||
|
||||
async function collect(stream: AsyncIterable<string>): Promise<string[]> {
|
||||
const out: string[] = []
|
||||
for await (const item of stream) out.push(item)
|
||||
return out
|
||||
}
|
||||
|
||||
describe('parseSse', () => {
|
||||
it('parses simple events and the DONE sentinel', async () => {
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('handles events split across reads at arbitrary positions', async () => {
|
||||
const events = await collect(parseSse(bytes('da', 'ta: {"a"', ':1}\n', '\ndata: [DO', 'NE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('handles multi-byte UTF-8 split across reads', async () => {
|
||||
const encoded = new TextEncoder().encode('data: {"text":"日本語"}\n\ndata: [DONE]\n\n')
|
||||
// Split inside the 3-byte sequence for 日.
|
||||
const splitAt = 16
|
||||
const events = await collect(parseSse(bytes(encoded.slice(0, splitAt), encoded.slice(splitAt))))
|
||||
expect(events).toEqual(['{"text":"日本語"}', DONE])
|
||||
})
|
||||
|
||||
it('tolerates CRLF line endings', async () => {
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\r\n\r\ndata: [DONE]\r\n\r\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('joins multi-data events with newlines (SSE spec)', async () => {
|
||||
const events = await collect(parseSse(bytes('data: line1\ndata: line2\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['line1\nline2', DONE])
|
||||
})
|
||||
|
||||
it('ignores comments and non-data fields', async () => {
|
||||
const events = await collect(parseSse(bytes(': keepalive\nevent: chunk\nid: 7\ndata: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('skips blocks without data fields', async () => {
|
||||
const events = await collect(parseSse(bytes(': ping\n\ndata: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('preserves data lines without the optional space', async () => {
|
||||
const events = await collect(parseSse(bytes('data:{"a":1}\n\ndata:[DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('parses several events from one read', async () => {
|
||||
const events = await collect(parseSse(bytes('data: 1\n\ndata: 2\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['1', '2', DONE])
|
||||
})
|
||||
|
||||
it('flushes a final un-terminated DONE at stream end', async () => {
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\n\ndata: [DONE]')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('throws STREAM_CLOSED when the stream ends without DONE', async () => {
|
||||
await expect(collect(parseSse(bytes('data: {"a":1}\n\n')))).rejects.toThrow(LlmError)
|
||||
await expect(collect(parseSse(bytes('data: {"a":1}\n\n')))).rejects.toThrow(/without \[DONE\]/)
|
||||
})
|
||||
|
||||
it('throws STREAM_CLOSED for an empty stream', async () => {
|
||||
await expect(collect(parseSse(bytes()))).rejects.toThrow(/without \[DONE\]/)
|
||||
})
|
||||
|
||||
it('throws STREAM_CLOSED for a mid-event close', async () => {
|
||||
await expect(collect(parseSse(bytes('data: {"a"')))).rejects.toThrow(/without \[DONE\]/)
|
||||
})
|
||||
|
||||
it('stops yielding after DONE even when more data follows', async () => {
|
||||
const events = await collect(parseSse(bytes('data: [DONE]\n\ndata: {"late":1}\n\n')))
|
||||
expect(events).toEqual([DONE])
|
||||
})
|
||||
})
|
||||
|
||||
describe('parseSse edge branches', () => {
|
||||
it('handles a lone CR-terminated data line', async () => {
|
||||
// Exercises the \r-strip branch on a line that is ONLY "data:…\r".
|
||||
const events = await collect(parseSse(bytes('data: {"a":1}\r\n\r\ndata:[DONE]\r\n\r\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('strips CR from non-data field lines too', async () => {
|
||||
const events = await collect(parseSse(bytes('event: chunk\r\ndata: {"a":1}\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['{"a":1}', DONE])
|
||||
})
|
||||
|
||||
it('treats bare "data:" lines as empty payload entries', async () => {
|
||||
const events = await collect(parseSse(bytes('data:\ndata: x\n\ndata: [DONE]\n\n')))
|
||||
expect(events).toEqual(['\nx', DONE])
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,307 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { BlockAssembler, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { DONE, mapFinishReason, mapUsage, translate } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
async function* feed(...payloads: (string | object)[]): AsyncGenerator<string> {
|
||||
for (const payload of payloads) {
|
||||
yield typeof payload === 'string' ? payload : JSON.stringify(payload)
|
||||
}
|
||||
}
|
||||
|
||||
async function collect(stream: AsyncIterable<StreamChunk>): Promise<StreamChunk[]> {
|
||||
const out: StreamChunk[] = []
|
||||
for await (const chunk of stream) out.push(chunk)
|
||||
return out
|
||||
}
|
||||
|
||||
/** The live first-chunk signature: role + null content + EMPTY reasoning. */
|
||||
const firstChunk = { choices: [{ delta: { role: 'assistant', content: null, reasoning_content: '' } }] }
|
||||
|
||||
describe('translate: text', () => {
|
||||
it('streams a text block and defers finish to DONE', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: 'Hel' } }] },
|
||||
{ choices: [{ delta: { content: 'lo' } }] },
|
||||
{ choices: [{ delta: { content: '' }, finish_reason: 'stop' }], usage: { prompt_tokens: 5, completion_tokens: 2 } },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: 'Hel' },
|
||||
{ type: 'text-delta', index: 0, text: 'lo' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'Hello' } },
|
||||
{ type: 'usage', usage: { inputTokens: 5, outputTokens: 2 } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('assembles into the message BlockAssembler expects', async () => {
|
||||
const assembler = new BlockAssembler()
|
||||
for await (const chunk of translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: 'hi' } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }] },
|
||||
DONE,
|
||||
))) {
|
||||
assembler.push(chunk)
|
||||
}
|
||||
const result = assembler.result()
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hi' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('translate: reasoning', () => {
|
||||
it('does NOT open a reasoning block for the empty first-chunk signature', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: 'plain' } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.some(chunk => chunk.type === 'block-start' && chunk.blockType === 'reasoning')).toBe(false)
|
||||
})
|
||||
|
||||
it('streams reasoning then text as separate blocks', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: null, reasoning_content: 'think' } }] },
|
||||
{ choices: [{ delta: { content: null, reasoning_content: 'ing' } }] },
|
||||
{ choices: [{ delta: { content: 'answer', reasoning_content: null } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'reasoning' },
|
||||
{ type: 'reasoning-delta', index: 0, text: 'think' },
|
||||
{ type: 'reasoning-delta', index: 0, text: 'ing' },
|
||||
{ type: 'block-start', index: 1, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 1, text: 'answer' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'reasoning', text: 'thinking' } },
|
||||
{ type: 'block-end', index: 1, block: { type: 'text', text: 'answer' } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('treats an entirely absent reasoning_content field as non-thinking', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
{ choices: [{ delta: { role: 'assistant', content: 'x' } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.filter(chunk => chunk.type === 'block-start')).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('translate: tool calls', () => {
|
||||
it('reassembles a tool call from fragmented argument deltas (live capture shape)', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'call_00_x', type: 'function', function: { name: 'get_weather', arguments: '' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '{"city"' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: ': "Paris"}' } }] } }] },
|
||||
{ choices: [{ delta: { content: '' }, finish_reason: 'tool_calls' }], usage: { prompt_tokens: 28, completion_tokens: 6 } },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{ type: 'tool-call-delta', index: 0, id: 'call_00_x', name: 'get_weather', argumentsDelta: '' },
|
||||
{ type: 'tool-call-delta', index: 0, id: 'call_00_x', name: 'get_weather', argumentsDelta: '{"city"' },
|
||||
{ type: 'tool-call-delta', index: 0, id: 'call_00_x', name: 'get_weather', argumentsDelta: ': "Paris"}' },
|
||||
{
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: 'call_00_x', name: 'get_weather', arguments: '{"city": "Paris"}' },
|
||||
},
|
||||
{ type: 'usage', usage: { inputTokens: 28, outputTokens: 6 } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('disambiguates parallel tool calls by wire index', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{
|
||||
choices: [{
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{ index: 0, id: 'a', type: 'function', function: { name: 'one', arguments: '{}' } },
|
||||
{ index: 1, id: 'b', type: 'function', function: { name: 'two', arguments: '' } },
|
||||
],
|
||||
},
|
||||
}],
|
||||
},
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 1, function: { arguments: '{}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
const ends = chunks.filter(chunk => chunk.type === 'block-end')
|
||||
expect(ends).toEqual([
|
||||
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: 'a', name: 'one', arguments: '{}' } },
|
||||
{ type: 'block-end', index: 1, block: { type: 'tool-call', id: 'b', name: 'two', arguments: '{}' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('interleaves text and tool-call blocks with distinct indices', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: 'Checking.' } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'c', type: 'function', function: { name: 'f', arguments: '{}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
const starts = chunks.filter(chunk => chunk.type === 'block-start')
|
||||
expect(starts).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'block-start', index: 1, blockType: 'tool-call' },
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('translate: finish and usage handling', () => {
|
||||
it('takes usage from a trailing usage-only chunk (docs shape)', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: 'x' } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }], usage: null },
|
||||
{ choices: [], usage: { prompt_tokens: 9, completion_tokens: 1 } },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.at(-2)).toEqual({ type: 'usage', usage: { inputTokens: 9, outputTokens: 1 } })
|
||||
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'stop' } })
|
||||
})
|
||||
|
||||
it('last usage wins when both attached and trailing arrive', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: {}, finish_reason: 'stop' }], usage: { prompt_tokens: 1, completion_tokens: 1 } },
|
||||
{ choices: [], usage: { prompt_tokens: 2, completion_tokens: 2 } },
|
||||
DONE,
|
||||
)))
|
||||
const usage = chunks.find(chunk => chunk.type === 'usage')
|
||||
expect(usage).toEqual({ type: 'usage', usage: { inputTokens: 2, outputTokens: 2 } })
|
||||
})
|
||||
|
||||
it('defaults to finish stop when no finish_reason ever arrives', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { content: 'x' } }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'stop' } })
|
||||
})
|
||||
|
||||
it('omits the usage chunk when none arrived', async () => {
|
||||
const chunks = await collect(translate(feed(firstChunk, DONE)))
|
||||
expect(chunks.some(chunk => chunk.type === 'usage')).toBe(false)
|
||||
})
|
||||
|
||||
it('handles chunks with no choices at all', async () => {
|
||||
const chunks = await collect(translate(feed({}, DONE)))
|
||||
expect(chunks).toEqual([{ type: 'finish', reason: { kind: 'stop' } }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('translate: errors', () => {
|
||||
it('throws MALFORMED_RESPONSE for invalid JSON payloads', async () => {
|
||||
await expect(collect(translate(feed('{bad json')))).rejects.toThrow(LlmError)
|
||||
await expect(collect(translate(feed('{bad json')))).rejects.toThrow(/malformed SSE payload/)
|
||||
})
|
||||
|
||||
it('throws STREAM_CLOSED when the payload source ends without DONE', async () => {
|
||||
await expect(collect(translate(feed(firstChunk)))).rejects.toThrow(/without \[DONE\]/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('mapFinishReason', () => {
|
||||
it.each([
|
||||
['stop', { kind: 'stop' }],
|
||||
['tool_calls', { kind: 'tool-calls' }],
|
||||
['length', { kind: 'max-tokens' }],
|
||||
])('maps %s', (wire, expected) => {
|
||||
expect(mapFinishReason(wire)).toEqual(expected)
|
||||
})
|
||||
|
||||
it.each(['content_filter', 'insufficient_system_resource', 'mystery_reason'])(
|
||||
'maps %s to an error kind with the wire code',
|
||||
(wire) => {
|
||||
expect(mapFinishReason(wire)).toEqual({
|
||||
kind: 'error',
|
||||
message: `model stopped: ${wire}`,
|
||||
code: wire.toUpperCase(),
|
||||
})
|
||||
},
|
||||
)
|
||||
})
|
||||
|
||||
describe('mapUsage', () => {
|
||||
it('maps the full live-capture shape', () => {
|
||||
expect(mapUsage({
|
||||
prompt_tokens: 283,
|
||||
completion_tokens: 69,
|
||||
prompt_cache_hit_tokens: 256,
|
||||
prompt_cache_miss_tokens: 27,
|
||||
prompt_tokens_details: { cached_tokens: 256 },
|
||||
completion_tokens_details: { reasoning_tokens: 24 },
|
||||
})).toEqual({
|
||||
// 283 wire prompt_tokens minus the 256 cached → 27 uncached input
|
||||
// (TokenUsage counts are disjoint).
|
||||
inputTokens: 27,
|
||||
outputTokens: 69,
|
||||
cacheReadTokens: 256,
|
||||
reasoningTokens: 24,
|
||||
})
|
||||
})
|
||||
|
||||
it('falls back to prompt_cache_hit_tokens when details are absent', () => {
|
||||
expect(mapUsage({ prompt_tokens: 10, completion_tokens: 2, prompt_cache_hit_tokens: 8 }))
|
||||
.toEqual({ inputTokens: 2, outputTokens: 2, cacheReadTokens: 8 })
|
||||
})
|
||||
|
||||
it('omits optional fields when the wire omits them', () => {
|
||||
expect(mapUsage({ prompt_tokens: 10, completion_tokens: 2 }))
|
||||
.toEqual({ inputTokens: 10, outputTokens: 2 })
|
||||
})
|
||||
})
|
||||
|
||||
describe('translate: defensive tool-call branches', () => {
|
||||
it('handles deltas that never carry id or name (empty-string fallbacks)', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
// Hypothetical lenient wire: argument fragments with no id/name at all.
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '{}' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{ type: 'tool-call-delta', index: 0, id: '', argumentsDelta: '{}' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: '', name: '', arguments: '{}' } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('handles tool_call deltas with a function object but no arguments field', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'c', type: 'function', function: { name: 'f' } }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks[1]).toEqual({ type: 'tool-call-delta', index: 0, id: 'c', name: 'f', argumentsDelta: '' })
|
||||
})
|
||||
|
||||
it('handles tool_call deltas with no function object at all', async () => {
|
||||
const chunks = await collect(translate(feed(
|
||||
firstChunk,
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: 'c' }] } }] },
|
||||
{ choices: [{ delta: {}, finish_reason: 'tool_calls' }] },
|
||||
DONE,
|
||||
)))
|
||||
expect(chunks[1]).toEqual({ type: 'tool-call-delta', index: 0, id: 'c', argumentsDelta: '' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib"
|
||||
},
|
||||
"include": ["src"],
|
||||
"references": [
|
||||
{ "path": "../../vendor/cosmokit" },
|
||||
{ "path": "../../vendor/cordis" },
|
||||
{ "path": "../../vendor/schemastery" },
|
||||
{ "path": "../llm" }
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
# @deepseek-ai/dsh-llm-pi-ai
|
||||
|
||||
DeepSeek adapter for the harness LLM seam backed by
|
||||
[`@earendil-works/pi-ai`](https://www.npmjs.com/package/@earendil-works/pi-ai)
|
||||
(the LLM library behind the pi agent).
|
||||
|
||||
## Why a second adapter exists
|
||||
|
||||
`@deepseek-ai/dsh-llm-deepseek` already talks to the same endpoint. This
|
||||
package is its **design-verification twin**: same models, same wire
|
||||
protocol, completely different internals — a unified LLM library with its
|
||||
own event vocabulary versus hand-rolled fetch/SSE. Anything the harness
|
||||
`StreamChunk` protocol cannot express for BOTH implementations is a
|
||||
core-vocabulary bug. The differences it exercised on purpose:
|
||||
|
||||
- pi-ai hands back tool-call `arguments` as **parsed objects**; the harness
|
||||
keeps raw JSON strings (re-stringified at `block-end`).
|
||||
- pi-ai reports failures as **in-stream error events** (it never throws
|
||||
mid-stream); these map to `finish {kind:'error'|'aborted'}` chunks — the
|
||||
protocol's other sanctioned error path besides throwing (which
|
||||
llm-deepseek uses).
|
||||
- pi-ai folds reasoning tokens into `usage.output`; there is no separate
|
||||
reasoning count to map.
|
||||
- pi-ai's options omit stop sequences; `GenerateOptions.stop` is injected
|
||||
via its `onPayload` hook.
|
||||
|
||||
## Config
|
||||
|
||||
Same shape as llm-deepseek (one-line swap in cordis.yml), with pi-ai's
|
||||
thinking-level vocabulary:
|
||||
|
||||
```yaml
|
||||
- id: llm
|
||||
name: '@deepseek-ai/dsh-llm-pi-ai'
|
||||
config:
|
||||
apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
baseURL: !!js process.env.DEEPSEEK_BASE_URL
|
||||
models: [deepseek-v4-flash, deepseek-v4-pro]
|
||||
reasoning: high # off | high | xhigh (xhigh → wire 'max')
|
||||
```
|
||||
|
||||
## Dependency weight
|
||||
|
||||
pi-ai declares the openai/anthropic/google/mistral/AWS SDKs as install-time
|
||||
dependencies. They are lazy-loaded — only the openai SDK actually loads for
|
||||
this adapter — but they do land in `node_modules`. Accepted for a package
|
||||
whose purpose is design verification.
|
||||
|
||||
## Limitations
|
||||
|
||||
Same MVP contract as llm-deepseek: `prefill` throws `UNSUPPORTED`, images
|
||||
are not representable, `tool_choice` is not mapped.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit suites run against a local `node:http` mock SSE server (pi-ai's openai
|
||||
SDK happily talks to any base URL). Real-API coverage in
|
||||
`tests/adapter.e2e.ts` (`yarn test:e2e`, key-gated): V4 Flash + V4 Pro across
|
||||
all exposed reasoning levels (off/high/xhigh), the thinking+tools round trip,
|
||||
and a cross-adapter structural-equivalence check against llm-deepseek.
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-llm-pi-ai",
|
||||
"description": "pi-ai-backed DeepSeek adapter for the DeepSeek Harness LLM seam (design-verification twin of dsh-llm-deepseek)",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"@earendil-works/pi-ai": "^0.79.1",
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm-deepseek": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
/**
|
||||
* `PiAiAdapter`: the `@earendil-works/pi-ai`-backed implementation of the
|
||||
* harness LLM seam, pointed at a DeepSeek (OpenAI-compatible) endpoint.
|
||||
*
|
||||
* This adapter exists as a design-verification twin of
|
||||
* `@deepseek-ai/dsh-llm-deepseek`: same models, same wire protocol,
|
||||
* completely different internals (a unified LLM library with its own event
|
||||
* vocabulary vs hand-rolled fetch/SSE). Anything the StreamChunk protocol
|
||||
* cannot express for BOTH implementations is a core-vocabulary bug.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/adapter
|
||||
*/
|
||||
|
||||
import { stream as piStream } from '@earendil-works/pi-ai'
|
||||
import type { Model } from '@earendil-works/pi-ai'
|
||||
import { LlmAdapter, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { toPiContext, toStreamChunks } from './convert.ts'
|
||||
|
||||
/** Reasoning levels surfaced by this adapter (DeepSeek wire: high|max). */
|
||||
export type PiAiReasoning = 'off' | 'high' | 'xhigh'
|
||||
|
||||
export interface PiAiAdapterOptions {
|
||||
apiKey: string
|
||||
baseURL: string
|
||||
/** Thinking level applied to every request ('off' disables thinking). */
|
||||
reasoning?: PiAiReasoning | undefined
|
||||
}
|
||||
|
||||
/** Build the inline pi-ai model descriptor for one DeepSeek model name. */
|
||||
export function buildModel(modelId: string, options: PiAiAdapterOptions): Model<'openai-completions'> {
|
||||
return {
|
||||
id: modelId,
|
||||
name: modelId,
|
||||
api: 'openai-completions',
|
||||
provider: 'deepseek',
|
||||
baseUrl: options.baseURL,
|
||||
// Always true: pi-ai only emits the DeepSeek `thinking` field for
|
||||
// reasoning-capable models, deriving enabled/disabled from whether a
|
||||
// reasoningEffort option is passed. DeepSeek's provider default is
|
||||
// ENABLED, so 'off' must send an explicit {type: 'disabled'} — which
|
||||
// requires this flag to stay on.
|
||||
reasoning: true,
|
||||
// DeepSeek's official effort levels: high|max (xhigh maps to max).
|
||||
thinkingLevelMap: { minimal: null, low: null, medium: null, high: 'high', xhigh: 'max' },
|
||||
input: ['text'],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 128_000,
|
||||
maxTokens: 64_000,
|
||||
compat: {
|
||||
// Auto-detection only fires for *.deepseek.com base URLs; the internal
|
||||
// endpoint (and test mocks) need these set explicitly.
|
||||
thinkingFormat: 'deepseek',
|
||||
requiresReasoningContentOnAssistantMessages: true,
|
||||
supportsReasoningEffort: true,
|
||||
// DeepSeek documents max_tokens (not OpenAI's max_completion_tokens).
|
||||
maxTokensField: 'max_tokens',
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* pi-ai-backed adapter. One instance serves every registered model name.
|
||||
*
|
||||
* Implementation notes:
|
||||
* - `GenerateOptions.stop` is injected via pi-ai's `onPayload` hook (its
|
||||
* public options omit stop sequences).
|
||||
* - `prefill` throws UNSUPPORTED (same contract as dsh-llm-deepseek).
|
||||
* - pi-ai reports request failures as in-stream error events; convert.ts
|
||||
* maps them to `finish {kind:'error'|'aborted'}` chunks rather than
|
||||
* throwing — both are sanctioned StreamChunk error paths.
|
||||
*/
|
||||
export class PiAiAdapter extends LlmAdapter {
|
||||
constructor(private readonly options: PiAiAdapterOptions) {
|
||||
super()
|
||||
}
|
||||
|
||||
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (options.prefill !== undefined) {
|
||||
throw new LlmError(
|
||||
'prefill is not supported by the pi-ai adapter',
|
||||
'UNSUPPORTED',
|
||||
)
|
||||
}
|
||||
|
||||
const model = buildModel(options.model, this.options)
|
||||
// Undefined config means "provider default" (DeepSeek: thinking ENABLED),
|
||||
// matching llm-deepseek's omission semantics. pi-ai derives the wire
|
||||
// thinking toggle from whether reasoningEffort is passed, so undefined
|
||||
// maps to 'high' here; only an explicit 'off' disables thinking.
|
||||
const reasoning = this.options.reasoning ?? 'high'
|
||||
|
||||
// pi-ai's event stream has no iterator-return cancellation hook: if our
|
||||
// consumer stops early (break / loop abort), the underlying HTTP stream
|
||||
// would keep draining. Chain an internal controller onto the caller's
|
||||
// signal and abort it when this generator exits for any reason.
|
||||
const controller = new AbortController()
|
||||
const onCallerAbort = (): void => { controller.abort(options.signal?.reason) }
|
||||
if (options.signal?.aborted) controller.abort(options.signal.reason)
|
||||
else options.signal?.addEventListener('abort', onCallerAbort, { once: true })
|
||||
|
||||
try {
|
||||
const events = piStream(model, toPiContext(options), {
|
||||
apiKey: this.options.apiKey,
|
||||
...options.temperature !== undefined ? { temperature: options.temperature } : {},
|
||||
...options.maxTokens !== undefined ? { maxTokens: options.maxTokens } : {},
|
||||
signal: controller.signal,
|
||||
...reasoning !== 'off' ? { reasoningEffort: reasoning } : {},
|
||||
...options.stop !== undefined ? {
|
||||
// pi-ai's options omit stop sequences; inject them into the raw body.
|
||||
onPayload: (payload: unknown) => {
|
||||
(payload as Record<string, unknown>).stop = options.stop
|
||||
return payload
|
||||
},
|
||||
} : {},
|
||||
maxRetries: 0,
|
||||
})
|
||||
|
||||
yield* toStreamChunks(events)
|
||||
} finally {
|
||||
options.signal?.removeEventListener('abort', onCallerAbort)
|
||||
controller.abort('consumer stopped streaming')
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
/**
|
||||
* Bidirectional mapping between the harness vocabulary and pi-ai's:
|
||||
* `GenerateOptions`/`Message[]` → pi-ai `Context`, and pi-ai
|
||||
* `AssistantMessageEvent`s → harness `StreamChunk`s.
|
||||
*
|
||||
* Vocabulary differences worth knowing (they are exactly why this adapter
|
||||
* exists — an independent implementation stress-tests the StreamChunk
|
||||
* protocol):
|
||||
* - pi-ai tool-call `arguments` are PARSED OBJECTS; the harness keeps the
|
||||
* raw JSON string. We parse on the way in and re-stringify on the way out.
|
||||
* - pi-ai reports errors as in-stream `error` events (it never throws
|
||||
* mid-stream); the harness expresses those as `finish {kind:'error'}` /
|
||||
* `{kind:'aborted'}` chunks.
|
||||
* - pi-ai folds reasoning tokens into `usage.output`; there is no separate
|
||||
* reasoning count to map.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/convert
|
||||
*/
|
||||
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { FinishReason, GenerateOptions, Message, StreamChunk, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
AssistantMessage,
|
||||
AssistantMessageEvent,
|
||||
Context as PiContext,
|
||||
Message as PiMessage,
|
||||
Tool as PiTool,
|
||||
Usage as PiUsage,
|
||||
} from '@earendil-works/pi-ai'
|
||||
|
||||
/** Join the text blocks of a harness message. */
|
||||
function flattenText(message: Message): string {
|
||||
return message.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
/** Parse tool-call argument JSON; tolerate model malformations with {}. */
|
||||
function parseArguments(raw: string): Record<string, unknown> {
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(raw)
|
||||
if (typeof parsed === 'object' && parsed !== null && !Array.isArray(parsed)) {
|
||||
return parsed as Record<string, unknown>
|
||||
}
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
return {}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert harness history to a pi-ai Context. Tool results need the tool
|
||||
* NAME (pi-ai's `toolName`), which the harness doesn't carry on the result
|
||||
* block — it is recovered from the preceding assistant tool-call with the
|
||||
* same id.
|
||||
*/
|
||||
export function toPiContext(options: GenerateOptions): PiContext {
|
||||
const toolNames = new Map<string, string>()
|
||||
const messages: PiMessage[] = []
|
||||
|
||||
for (const message of options.messages) {
|
||||
if (message.role === 'system') {
|
||||
// pi-ai has a single systemPrompt slot; in-history system messages are
|
||||
// folded into user messages to preserve order (rare in practice — the
|
||||
// harness sends the system prompt via options.system).
|
||||
messages.push({ role: 'user', content: flattenText(message), timestamp: 0 })
|
||||
continue
|
||||
}
|
||||
if (message.role === 'assistant') {
|
||||
const content: AssistantMessage['content'] = []
|
||||
for (const block of message.content) {
|
||||
switch (block.type) {
|
||||
case 'text':
|
||||
content.push({ type: 'text', text: block.text })
|
||||
break
|
||||
case 'reasoning':
|
||||
// thinkingSignature names the wire field pi-ai replays the CoT
|
||||
// under. Without it pi-ai falls back to reasoning_content: ""
|
||||
// (its requiresReasoningContentOnAssistantMessages shim), which
|
||||
// violates DeepSeek's thinking-mode passback rule on tool-call
|
||||
// turns (guides/thinking_mode.mdx § Tool Calls).
|
||||
content.push({ type: 'thinking', thinking: block.text, thinkingSignature: 'reasoning_content' })
|
||||
break
|
||||
case 'tool-call':
|
||||
toolNames.set(block.id, block.name)
|
||||
content.push({
|
||||
type: 'toolCall',
|
||||
id: block.id,
|
||||
name: block.name,
|
||||
arguments: parseArguments(block.arguments),
|
||||
})
|
||||
break
|
||||
default:
|
||||
// image / plugin-added block types: not representable here.
|
||||
break
|
||||
}
|
||||
}
|
||||
messages.push({
|
||||
role: 'assistant',
|
||||
content,
|
||||
api: 'openai-completions',
|
||||
provider: 'deepseek',
|
||||
model: options.model,
|
||||
usage: emptyPiUsage(),
|
||||
stopReason: content.some(piece => piece.type === 'toolCall') ? 'toolUse' : 'stop',
|
||||
timestamp: 0,
|
||||
})
|
||||
continue
|
||||
}
|
||||
// user role: text + tool results (each result becomes its own message).
|
||||
const text = flattenText(message)
|
||||
const results = message.content.filter(block => block.type === 'tool-result')
|
||||
if (text.length > 0 || results.length === 0) {
|
||||
messages.push({ role: 'user', content: text, timestamp: 0 })
|
||||
}
|
||||
for (const result of results) {
|
||||
messages.push({
|
||||
role: 'toolResult',
|
||||
toolCallId: result.toolCallId,
|
||||
toolName: toolNames.get(result.toolCallId) ?? 'unknown',
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: result.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('') || '(no output)',
|
||||
}],
|
||||
isError: result.isError ?? false,
|
||||
timestamp: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const tools: PiTool[] | undefined = options.tools?.map(tool => ({
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
// ToolSchema.parameters is a JSON Schema object; pi-ai's TSchema
|
||||
// (TypeBox) is structurally JSON Schema, so it assigns directly.
|
||||
parameters: tool.parameters,
|
||||
}))
|
||||
|
||||
return {
|
||||
...options.system !== undefined ? { systemPrompt: options.system } : {},
|
||||
messages,
|
||||
...tools !== undefined && tools.length > 0 ? { tools } : {},
|
||||
}
|
||||
}
|
||||
|
||||
function emptyPiUsage(): PiUsage {
|
||||
return {
|
||||
input: 0,
|
||||
output: 0,
|
||||
cacheRead: 0,
|
||||
cacheWrite: 0,
|
||||
totalTokens: 0,
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
|
||||
}
|
||||
}
|
||||
|
||||
/** Map pi-ai usage (reasoning folded into output by pi-ai). */
|
||||
export function mapUsage(usage: PiUsage): TokenUsage {
|
||||
return {
|
||||
inputTokens: usage.input,
|
||||
outputTokens: usage.output,
|
||||
...usage.cacheRead > 0 ? { cacheReadTokens: usage.cacheRead } : {},
|
||||
...usage.cacheWrite > 0 ? { cacheWriteTokens: usage.cacheWrite } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/** Map a terminal pi-ai event to the harness finish reason. */
|
||||
export function mapStopReason(message: AssistantMessage): FinishReason {
|
||||
switch (message.stopReason) {
|
||||
case 'stop': return { kind: 'stop' }
|
||||
case 'length': return { kind: 'max-tokens' }
|
||||
case 'toolUse': return { kind: 'tool-calls' }
|
||||
case 'aborted': return { kind: 'aborted' }
|
||||
case 'error': return {
|
||||
kind: 'error',
|
||||
message: message.errorMessage ?? 'pi-ai stream error',
|
||||
code: 'PI_AI_ERROR',
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate the pi-ai event stream into StreamChunks. pi-ai never throws
|
||||
* mid-stream — failures arrive as `error` events, which become error/aborted
|
||||
* `finish` chunks (the harness protocol's other error-delivery style).
|
||||
*/
|
||||
export async function* toStreamChunks(events: AsyncIterable<AssistantMessageEvent>): AsyncGenerator<StreamChunk> {
|
||||
// pi-ai contentIndex ↔ our block index map 1:1 (both count blocks from 0
|
||||
// in stream order), but we track ids per index for tool calls.
|
||||
const toolIds = new Map<number, { id: string; name: string }>()
|
||||
|
||||
for await (const event of events) {
|
||||
switch (event.type) {
|
||||
case 'start':
|
||||
break
|
||||
case 'text_start':
|
||||
yield { type: 'block-start', index: event.contentIndex, blockType: 'text' }
|
||||
break
|
||||
case 'text_delta':
|
||||
yield { type: 'text-delta', index: event.contentIndex, text: event.delta }
|
||||
break
|
||||
case 'text_end':
|
||||
yield { type: 'block-end', index: event.contentIndex, block: { type: 'text', text: event.content } }
|
||||
break
|
||||
case 'thinking_start':
|
||||
yield { type: 'block-start', index: event.contentIndex, blockType: 'reasoning' }
|
||||
break
|
||||
case 'thinking_delta':
|
||||
yield { type: 'reasoning-delta', index: event.contentIndex, text: event.delta }
|
||||
break
|
||||
case 'thinking_end':
|
||||
yield { type: 'block-end', index: event.contentIndex, block: { type: 'reasoning', text: event.content } }
|
||||
break
|
||||
case 'toolcall_start': {
|
||||
// The id/name live on the partial's content at this index.
|
||||
const partial = event.partial.content[event.contentIndex]
|
||||
const id = partial?.type === 'toolCall' ? partial.id : ''
|
||||
const name = partial?.type === 'toolCall' ? partial.name : ''
|
||||
toolIds.set(event.contentIndex, { id, name })
|
||||
yield { type: 'block-start', index: event.contentIndex, blockType: 'tool-call' }
|
||||
break
|
||||
}
|
||||
case 'toolcall_delta': {
|
||||
const known = toolIds.get(event.contentIndex)
|
||||
yield {
|
||||
type: 'tool-call-delta',
|
||||
index: event.contentIndex,
|
||||
id: CallId(known?.id ?? ''),
|
||||
...known?.name !== undefined && known.name.length > 0 ? { name: known.name } : {},
|
||||
argumentsDelta: event.delta,
|
||||
}
|
||||
break
|
||||
}
|
||||
case 'toolcall_end':
|
||||
yield {
|
||||
type: 'block-end',
|
||||
index: event.contentIndex,
|
||||
block: {
|
||||
type: 'tool-call',
|
||||
id: CallId(event.toolCall.id),
|
||||
name: event.toolCall.name,
|
||||
// pi-ai hands back the PARSED arguments; the harness vocabulary
|
||||
// keeps the raw string.
|
||||
arguments: JSON.stringify(event.toolCall.arguments),
|
||||
},
|
||||
}
|
||||
break
|
||||
case 'done':
|
||||
yield { type: 'usage', usage: mapUsage(event.message.usage) }
|
||||
yield { type: 'finish', reason: mapStopReason(event.message) }
|
||||
return
|
||||
case 'error':
|
||||
// In-stream error delivery (pi-ai's style) → error finish chunk
|
||||
// (the harness's other sanctioned error path besides throwing).
|
||||
yield { type: 'usage', usage: mapUsage(event.error.usage) }
|
||||
yield { type: 'finish', reason: mapStopReason(event.error) }
|
||||
return
|
||||
// no default: AssistantMessageEvent is pi-ai's closed union; a new
|
||||
// event type should fail compilation here via tsc's exhaustiveness
|
||||
// when one is added (switch covers all current variants).
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
/**
|
||||
* pi-ai-backed DeepSeek adapter plugin. Same Config shape as
|
||||
* `@deepseek-ai/dsh-llm-deepseek` (one-line swap in cordis.yml), different
|
||||
* implementation underneath — see `./adapter.ts` for why both exist.
|
||||
*
|
||||
* ```yaml
|
||||
* - id: llm
|
||||
* name: '@deepseek-ai/dsh-llm-pi-ai'
|
||||
* config:
|
||||
* apiKey: !!js process.env.DEEPSEEK_API_KEY
|
||||
* baseURL: !!js process.env.DEEPSEEK_BASE_URL
|
||||
* models: [deepseek-v4-flash, deepseek-v4-pro]
|
||||
* reasoning: high
|
||||
* ```
|
||||
*
|
||||
* @module @deepseek-ai/dsh-llm-pi-ai
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type {} from '@deepseek-ai/dsh-llm'
|
||||
import { PiAiAdapter } from './adapter.ts'
|
||||
import type { PiAiReasoning } from './adapter.ts'
|
||||
|
||||
export { buildModel, PiAiAdapter } from './adapter.ts'
|
||||
export type { PiAiAdapterOptions, PiAiReasoning } from './adapter.ts'
|
||||
export { mapStopReason, mapUsage, toPiContext, toStreamChunks } from './convert.ts'
|
||||
|
||||
export const name = 'llm-pi-ai'
|
||||
export const inject = ['llm']
|
||||
|
||||
export interface Config {
|
||||
/** API key; falls back to $DEEPSEEK_API_KEY. Required one way or the other. */
|
||||
apiKey?: string
|
||||
/** Endpoint base; falls back to $DEEPSEEK_BASE_URL, then the public API. */
|
||||
baseURL?: string
|
||||
/** Model names to register (sent verbatim on the wire). */
|
||||
models?: string[]
|
||||
/**
|
||||
* Thinking level for every request: 'off' disables thinking mode; 'high'
|
||||
* and 'xhigh' (wire 'max') set the effort. Omitted = provider default
|
||||
* (thinking enabled), matching llm-deepseek's omission semantics.
|
||||
*/
|
||||
reasoning?: PiAiReasoning
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
apiKey: z.string(),
|
||||
baseURL: z.string(),
|
||||
models: z.array(z.string()).default(['deepseek-v4-flash', 'deepseek-v4-pro']),
|
||||
reasoning: z.union(['off', 'high', 'xhigh']),
|
||||
})
|
||||
|
||||
/** Public API default; the internal endpoint comes from $DEEPSEEK_BASE_URL. */
|
||||
export const PUBLIC_BASE_URL = 'https://api.deepseek.com'
|
||||
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
const apiKey = config.apiKey ?? process.env.DEEPSEEK_API_KEY
|
||||
if (apiKey === undefined || apiKey.length === 0) {
|
||||
throw new Error('llm-pi-ai: an API key is required (Config.apiKey or $DEEPSEEK_API_KEY)')
|
||||
}
|
||||
const baseURL = config.baseURL ?? process.env.DEEPSEEK_BASE_URL ?? PUBLIC_BASE_URL
|
||||
// schemastery's .default() guarantees models is set after validation.
|
||||
const models = config.models as string[]
|
||||
|
||||
ctx.llm.registerAdapter(models, new PiAiAdapter({
|
||||
apiKey,
|
||||
baseURL,
|
||||
reasoning: config.reasoning,
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateResult, Message, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import type { Config } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
|
||||
/**
|
||||
* Real-API e2e for the pi-ai-backed adapter: V4 Flash + V4 Pro across all
|
||||
* reasoning levels the adapter exposes (off / high / xhigh→wire 'max').
|
||||
* Mirrors the llm-deepseek matrix so the two independent implementations
|
||||
* verify the same StreamChunk contract. Key-gated.
|
||||
*/
|
||||
|
||||
const FLASH = 'deepseek-v4-flash'
|
||||
const PRO = 'deepseek-v4-pro'
|
||||
|
||||
async function harness(model: string, config: Partial<Config> = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, { models: [model], ...config })
|
||||
return ctx
|
||||
}
|
||||
|
||||
function ask(text: string): Message[] {
|
||||
return [{ role: 'user', content: [{ type: 'text', text }] }]
|
||||
}
|
||||
|
||||
function textOf(result: GenerateResult): string {
|
||||
return result.message.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
function blockKinds(result: GenerateResult): string[] {
|
||||
return result.message.content.map(block => block.type)
|
||||
}
|
||||
|
||||
const weatherTool: ToolSchema = {
|
||||
name: 'get_weather',
|
||||
description: 'Get the current weather for a city.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: { city: { type: 'string', description: 'City name' } },
|
||||
required: ['city'],
|
||||
},
|
||||
}
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('llm-pi-ai e2e (real API)', () => {
|
||||
it.each([FLASH, PRO])('%s + reasoning off: plain text generation', async (model) => {
|
||||
const ctx = await harness(model, { reasoning: 'off' })
|
||||
const result = await ctx.llm.generate({
|
||||
model,
|
||||
messages: ask('Reply with exactly the word: pong'),
|
||||
maxTokens: 50,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(textOf(result).toLowerCase()).toContain('pong')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(false)
|
||||
})
|
||||
|
||||
it.each([FLASH, PRO])('%s + reasoning high: reasoning blocks present', async (model) => {
|
||||
const ctx = await harness(model, { reasoning: 'high' })
|
||||
const result = await ctx.llm.generate({
|
||||
model,
|
||||
messages: ask('Which is larger, 9.11 or 9.8? Answer with just the number.'),
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(result.finish.kind).toBe('stop')
|
||||
expect(result.message.content.some(block => block.type === 'reasoning')).toBe(true)
|
||||
expect(textOf(result)).toContain('9.8')
|
||||
})
|
||||
|
||||
it('pro + reasoning xhigh (wire max): tool-call round trip', async () => {
|
||||
const ctx = await harness(PRO, { reasoning: 'xhigh' })
|
||||
|
||||
const first = await ctx.llm.generate({
|
||||
model: PRO,
|
||||
messages: ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
tools: [weatherTool],
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(first.finish.kind).toBe('tool-calls')
|
||||
const call = first.message.content.find(block => block.type === 'tool-call')
|
||||
expect(call).toBeDefined()
|
||||
expect(call!.name).toBe('get_weather')
|
||||
expect(JSON.parse(call!.arguments)).toMatchObject({ city: expect.stringMatching(/paris/i) as string })
|
||||
|
||||
const second = await ctx.llm.generate({
|
||||
model: PRO,
|
||||
messages: [
|
||||
...ask('What is the weather in Paris right now? Use the get_weather tool.'),
|
||||
{ role: 'assistant', content: first.message.content },
|
||||
{
|
||||
role: 'user',
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: CallId(call!.id),
|
||||
content: [{ type: 'text', text: 'Sunny, 22°C' }],
|
||||
}],
|
||||
},
|
||||
],
|
||||
tools: [weatherTool],
|
||||
maxTokens: 2000,
|
||||
})
|
||||
expect(second.finish.kind).toBe('stop')
|
||||
expect(textOf(second).toLowerCase()).toMatch(/sunny|22/)
|
||||
})
|
||||
|
||||
it('produces the same block structure as llm-deepseek for the same prompt', async () => {
|
||||
// Loose structural equivalence between the two independent adapters:
|
||||
// same block KINDS in the same order for a deterministic prompt — the
|
||||
// cross-implementation check that the StreamChunk design holds.
|
||||
const deepseekCtx = new Context()
|
||||
await deepseekCtx.plugin(LlmService)
|
||||
await deepseekCtx.plugin(LlmDeepSeek, { models: [FLASH], thinking: 'disabled' })
|
||||
|
||||
const piCtx = await harness(FLASH, { reasoning: 'off' })
|
||||
|
||||
const prompt = ask('Reply with exactly the word: pong')
|
||||
const [fromDeepSeek, fromPiAi] = await Promise.all([
|
||||
deepseekCtx.llm.generate({ model: FLASH, messages: prompt, maxTokens: 50 }),
|
||||
piCtx.llm.generate({ model: FLASH, messages: prompt, maxTokens: 50 }),
|
||||
])
|
||||
expect(blockKinds(fromPiAi)).toEqual(blockKinds(fromDeepSeek))
|
||||
expect(fromPiAi.finish.kind).toBe(fromDeepSeek.finish.kind)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,354 @@
|
||||
import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService, { CallId, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { buildModel, PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
|
||||
/** Scripted SSE responses, one per request (OpenAI chat-completions shape). */
|
||||
interface MockServer {
|
||||
url: string
|
||||
requests: unknown[]
|
||||
close(): Promise<void>
|
||||
}
|
||||
|
||||
const servers: Server[] = []
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(servers.splice(0).map(server => new Promise(resolve => server.close(resolve))))
|
||||
})
|
||||
|
||||
async function mockServer(script: { status?: number; events?: string[]; body?: string }[]): Promise<MockServer> {
|
||||
const requests: unknown[] = []
|
||||
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
|
||||
let body = ''
|
||||
request.on('data', (chunk: Buffer) => { body += chunk.toString('utf8') })
|
||||
request.on('end', () => {
|
||||
requests.push(JSON.parse(body))
|
||||
const behavior = script.shift() ?? { status: 500, body: 'script exhausted' }
|
||||
if (behavior.status !== undefined && behavior.status !== 200) {
|
||||
response.writeHead(behavior.status, { 'content-type': 'application/json' })
|
||||
response.end(behavior.body ?? '{}')
|
||||
return
|
||||
}
|
||||
response.writeHead(200, { 'content-type': 'text/event-stream' })
|
||||
for (const event of behavior.events ?? []) response.write(`data: ${event}\n\n`)
|
||||
response.end()
|
||||
})
|
||||
})
|
||||
servers.push(server)
|
||||
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
|
||||
const address = server.address()
|
||||
if (address === null || typeof address === 'string') throw new Error('no port')
|
||||
return {
|
||||
url: `http://127.0.0.1:${address.port}`,
|
||||
requests,
|
||||
close: () => new Promise(resolve => server.close(() => { resolve() })),
|
||||
}
|
||||
}
|
||||
|
||||
const textEvents = [
|
||||
'{"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{"content":"hello"},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{},"index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
|
||||
'[DONE]',
|
||||
]
|
||||
|
||||
const toolEvents = [
|
||||
'{"choices":[{"delta":{"role":"assistant","content":null},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"get_weather","arguments":""}}]},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\\"city\\":\\"Paris\\"}"}}]},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{},"index":0,"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":20,"completion_tokens":6}}',
|
||||
'[DONE]',
|
||||
]
|
||||
|
||||
const thinkingEvents = [
|
||||
'{"choices":[{"delta":{"role":"assistant","content":null,"reasoning_content":""},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{"reasoning_content":"pondering"},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{"content":"answer","reasoning_content":null},"index":0,"finish_reason":null}]}',
|
||||
'{"choices":[{"delta":{},"index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":9}}',
|
||||
'[DONE]',
|
||||
]
|
||||
|
||||
async function harness(baseURL: string, config: object = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, { apiKey: 'test-key', baseURL, models: ['deepseek-v4-flash'], ...config })
|
||||
return ctx
|
||||
}
|
||||
|
||||
describe('PiAiAdapter against a mock server', () => {
|
||||
it('streams a text generation through ctx.llm.generate', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
})
|
||||
expect(result.message.content).toEqual([{ type: 'text', text: 'hello' }])
|
||||
expect(result.finish).toEqual({ kind: 'stop' })
|
||||
expect(result.usage).toMatchObject({ inputTokens: 3, outputTokens: 1 })
|
||||
})
|
||||
|
||||
it('streams tool calls with re-stringified arguments', async () => {
|
||||
const server = await mockServer([{ events: toolEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'weather?' }] }],
|
||||
tools: [{
|
||||
name: 'get_weather',
|
||||
description: 'Get weather',
|
||||
parameters: { type: 'object', properties: { city: { type: 'string' } } },
|
||||
}],
|
||||
})
|
||||
expect(result.finish).toEqual({ kind: 'tool-calls' })
|
||||
const call = result.message.content.find(block => block.type === 'tool-call')
|
||||
expect(call).toMatchObject({ name: 'get_weather', arguments: '{"city":"Paris"}' })
|
||||
})
|
||||
|
||||
it('maps reasoning_content streams to reasoning blocks', async () => {
|
||||
const server = await mockServer([{ events: thinkingEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'high' })
|
||||
|
||||
const result = await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'think' }] }],
|
||||
})
|
||||
expect(result.message.content).toEqual([
|
||||
{ type: 'reasoning', text: 'pondering' },
|
||||
{ type: 'text', text: 'answer' },
|
||||
])
|
||||
})
|
||||
|
||||
it('sends DeepSeek thinking fields when reasoning is configured', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'xhigh' })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
thinking: { type: 'enabled' },
|
||||
reasoning_effort: 'max', // xhigh maps to max via thinkingLevelMap
|
||||
})
|
||||
})
|
||||
|
||||
it('disables thinking for reasoning: off', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url, { reasoning: 'off' })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests[0]).toMatchObject({ thinking: { type: 'disabled' } })
|
||||
})
|
||||
|
||||
it('injects stop sequences through onPayload', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [], stop: ['END'] })
|
||||
expect(server.requests[0]).toMatchObject({ stop: ['END'] })
|
||||
})
|
||||
|
||||
it('maps HTTP errors to error finish chunks (pi-ai in-stream style)', async () => {
|
||||
const server = await mockServer([{
|
||||
status: 401,
|
||||
body: JSON.stringify({ error: { message: 'bad key' } }),
|
||||
}])
|
||||
const ctx = await harness(server.url)
|
||||
const result = await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(result.finish.kind).toBe('error')
|
||||
expect((result.finish as { message: string }).message).toMatch(/bad key|401/)
|
||||
})
|
||||
|
||||
it('rejects prefill with UNSUPPORTED', async () => {
|
||||
const ctx = await harness('http://127.0.0.1:1')
|
||||
await expect(ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
prefill: [{ type: 'text', text: 'Sure' }],
|
||||
})).rejects.toThrow(LlmError)
|
||||
})
|
||||
|
||||
it('registers/unregisters models on the llm service (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
const fiber = await ctx.plugin(LlmPiAi, { apiKey: 'k', baseURL: 'http://127.0.0.1:1' })
|
||||
expect(ctx.llm.models().sort()).toEqual(['deepseek-v4-flash', 'deepseek-v4-pro'])
|
||||
await fiber.dispose()
|
||||
expect(ctx.llm.models()).toEqual([])
|
||||
})
|
||||
|
||||
it('throws a clear error when no API key is available', async () => {
|
||||
const previous = process.env.DEEPSEEK_API_KEY
|
||||
delete process.env.DEEPSEEK_API_KEY
|
||||
try {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await expect(ctx.plugin(LlmPiAi, {})).rejects.toThrow(/an API key is required/)
|
||||
} finally {
|
||||
if (previous !== undefined) process.env.DEEPSEEK_API_KEY = previous
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('option spreads and env fallbacks', () => {
|
||||
it('forwards temperature, maxTokens, and signal', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
const controller = new AbortController()
|
||||
await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
temperature: 0.5,
|
||||
maxTokens: 40,
|
||||
signal: controller.signal,
|
||||
})
|
||||
expect(server.requests[0]).toMatchObject({ temperature: 0.5, max_tokens: 40 })
|
||||
})
|
||||
|
||||
it('falls back to DEEPSEEK_API_KEY / DEEPSEEK_BASE_URL env vars', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'env-key')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', server.url)
|
||||
try {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, { models: ['deepseek-v4-flash'] })
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests).toHaveLength(1)
|
||||
} finally {
|
||||
vi.unstubAllEnvs()
|
||||
}
|
||||
})
|
||||
|
||||
it('defaults to the public base URL without config or env', async () => {
|
||||
vi.stubEnv('DEEPSEEK_API_KEY', 'k')
|
||||
vi.stubEnv('DEEPSEEK_BASE_URL', undefined)
|
||||
try {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(LlmPiAi, {})
|
||||
expect(ctx.llm.models().length).toBeGreaterThan(0)
|
||||
} finally {
|
||||
vi.unstubAllEnvs()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildModel', () => {
|
||||
it('builds a DeepSeek-compat openai-completions model descriptor', () => {
|
||||
const model = buildModel('deepseek-v4-pro', { apiKey: 'k', baseURL: 'http://x', reasoning: 'high' })
|
||||
expect(model).toMatchObject({
|
||||
id: 'deepseek-v4-pro',
|
||||
api: 'openai-completions',
|
||||
provider: 'deepseek',
|
||||
baseUrl: 'http://x',
|
||||
reasoning: true,
|
||||
compat: { thinkingFormat: 'deepseek', requiresReasoningContentOnAssistantMessages: true },
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps reasoning true even for off (pi-ai gates the thinking field on it)', () => {
|
||||
// 'off' yields {thinking: {type: 'disabled'}} on the wire — pi-ai only
|
||||
// emits the field at all when model.reasoning is true.
|
||||
expect(buildModel('m', { apiKey: 'k', baseURL: 'http://x', reasoning: 'off' }).reasoning).toBe(true)
|
||||
})
|
||||
|
||||
it('adapter is constructible directly for embedding', () => {
|
||||
expect(new PiAiAdapter({ apiKey: 'k', baseURL: 'http://x' })).toBeInstanceOf(PiAiAdapter)
|
||||
})
|
||||
})
|
||||
|
||||
describe('review fixes', () => {
|
||||
it('defaults omitted reasoning config to thinking ENABLED (provider default)', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url) // no reasoning key at all
|
||||
await ctx.llm.generate({ model: 'deepseek-v4-flash', messages: [] })
|
||||
expect(server.requests[0]).toMatchObject({
|
||||
thinking: { type: 'enabled' },
|
||||
reasoning_effort: 'high',
|
||||
})
|
||||
})
|
||||
|
||||
it('replays reasoning_content on assistant tool-call turns (passback rule)', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [
|
||||
{ role: 'user', content: [{ type: 'text', text: 'weather?' }] },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'reasoning', text: 'I should check.' },
|
||||
{ type: 'tool-call', id: CallId('c1'), name: 'get_weather', arguments: '{"city":"Paris"}' },
|
||||
],
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: [{ type: 'tool-result', toolCallId: CallId('c1'), content: [{ type: 'text', text: 'Sunny' }] }],
|
||||
},
|
||||
],
|
||||
})
|
||||
const request = server.requests[0] as { messages: { role: string; reasoning_content?: string }[] }
|
||||
const assistant = request.messages.find(message => message.role === 'assistant')
|
||||
expect(assistant?.reasoning_content).toBe('I should check.')
|
||||
})
|
||||
|
||||
it('aborts the upstream request when the consumer stops streaming early', async () => {
|
||||
// Slow server: write one chunk, then hold the connection open and record
|
||||
// whether the socket closes (the adapter must cancel on early break).
|
||||
let socketClosed = false
|
||||
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
|
||||
request.on('data', () => undefined)
|
||||
request.on('end', () => {
|
||||
response.writeHead(200, { 'content-type': 'text/event-stream' })
|
||||
response.write(`data: ${textEvents[0]}\n\n`)
|
||||
response.write(`data: ${textEvents[1]}\n\n`)
|
||||
// never finish; rely on client abort
|
||||
request.socket.on('close', () => { socketClosed = true })
|
||||
})
|
||||
})
|
||||
servers.push(server)
|
||||
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
|
||||
const address = server.address()
|
||||
if (address === null || typeof address === 'string') throw new Error('no port')
|
||||
const ctx = await harness(`http://127.0.0.1:${address.port}`)
|
||||
|
||||
for await (const chunk of ctx.llm.stream({ model: 'deepseek-v4-flash', messages: [] })) {
|
||||
if (chunk.type === 'text-delta') break // stop early mid-stream
|
||||
}
|
||||
// The finally-abort must reach the server as a closed socket.
|
||||
await vi.waitFor(() => { expect(socketClosed).toBe(true) }, { timeout: 5_000 })
|
||||
})
|
||||
})
|
||||
|
||||
describe('review fixes: abort wiring', () => {
|
||||
it('honors a pre-aborted caller signal', async () => {
|
||||
const ctx = await harness('http://127.0.0.1:1')
|
||||
const controller = new AbortController()
|
||||
controller.abort('already cancelled')
|
||||
// pi-ai surfaces the abort as an in-stream error event → aborted finish.
|
||||
const result = await ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
signal: controller.signal,
|
||||
})
|
||||
expect(result.finish.kind).toBe('aborted')
|
||||
})
|
||||
|
||||
it('propagates a mid-stream caller abort to the upstream request', async () => {
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
const ctx = await harness(server.url)
|
||||
const controller = new AbortController()
|
||||
const pending = ctx.llm.generate({
|
||||
model: 'deepseek-v4-flash',
|
||||
messages: [],
|
||||
signal: controller.signal,
|
||||
})
|
||||
controller.abort()
|
||||
const result = await pending
|
||||
// Either the abort lands before any chunk (aborted) or after the tiny
|
||||
// mock stream finished (stop) — both are valid races; never a hang.
|
||||
expect(['aborted', 'stop']).toContain(result.finish.kind)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,322 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai'
|
||||
import { mapStopReason, mapUsage, toPiContext, toStreamChunks } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
|
||||
function usage(input = 0, output = 0, cacheRead = 0, cacheWrite = 0): Usage {
|
||||
return {
|
||||
input,
|
||||
output,
|
||||
cacheRead,
|
||||
cacheWrite,
|
||||
totalTokens: input + output + cacheRead + cacheWrite,
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
|
||||
}
|
||||
}
|
||||
|
||||
function assistant(overrides: Partial<AssistantMessage> = {}): AssistantMessage {
|
||||
return {
|
||||
role: 'assistant',
|
||||
content: [],
|
||||
api: 'openai-completions',
|
||||
provider: 'deepseek',
|
||||
model: 'deepseek-v4-flash',
|
||||
usage: usage(),
|
||||
stopReason: 'stop',
|
||||
timestamp: 0,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
async function* feed(...events: AssistantMessageEvent[]): AsyncGenerator<AssistantMessageEvent> {
|
||||
for (const event of events) yield event
|
||||
}
|
||||
|
||||
async function collect(stream: AsyncIterable<StreamChunk>): Promise<StreamChunk[]> {
|
||||
const out: StreamChunk[] = []
|
||||
for await (const chunk of stream) out.push(chunk)
|
||||
return out
|
||||
}
|
||||
|
||||
describe('toPiContext', () => {
|
||||
it('maps system prompt, user text, and tools', () => {
|
||||
const context = toPiContext({
|
||||
model: 'deepseek-v4-flash',
|
||||
system: 'be helpful',
|
||||
messages: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }],
|
||||
tools: [{ name: 'f', description: 'F', parameters: { type: 'object', properties: {} } }],
|
||||
})
|
||||
expect(context.systemPrompt).toBe('be helpful')
|
||||
expect(context.messages).toEqual([{ role: 'user', content: 'hi', timestamp: 0 }])
|
||||
expect(context.tools).toEqual([
|
||||
{ name: 'f', description: 'F', parameters: { type: 'object', properties: {} } },
|
||||
])
|
||||
})
|
||||
|
||||
it('omits empty tools and absent system prompt', () => {
|
||||
const context = toPiContext({ model: 'm', messages: [], tools: [] })
|
||||
expect(context.systemPrompt).toBeUndefined()
|
||||
expect(context.tools).toBeUndefined()
|
||||
})
|
||||
|
||||
it('maps assistant text/reasoning/tool-call blocks', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'reasoning', text: 'hmm' },
|
||||
{ type: 'text', text: 'calling' },
|
||||
{ type: 'tool-call', id: CallId('c1'), name: 'f', arguments: '{"a":1}' },
|
||||
],
|
||||
}],
|
||||
})
|
||||
const message = context.messages[0] as AssistantMessage
|
||||
expect(message.role).toBe('assistant')
|
||||
expect(message.stopReason).toBe('toolUse')
|
||||
expect(message.content).toEqual([
|
||||
// thinkingSignature names the replay field — DeepSeek's passback rule.
|
||||
{ type: 'thinking', thinking: 'hmm', thinkingSignature: 'reasoning_content' },
|
||||
{ type: 'text', text: 'calling' },
|
||||
{ type: 'toolCall', id: 'c1', name: 'f', arguments: { a: 1 } },
|
||||
])
|
||||
})
|
||||
|
||||
it('marks tool-call-free assistant messages with stopReason stop', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [{ role: 'assistant', content: [{ type: 'text', text: 'done' }] }],
|
||||
})
|
||||
expect((context.messages[0] as AssistantMessage).stopReason).toBe('stop')
|
||||
})
|
||||
|
||||
it('parses malformed tool-call arguments to {}', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'tool-call', id: CallId('c1'), name: 'f', arguments: '{broken' }],
|
||||
}],
|
||||
})
|
||||
const message = context.messages[0] as AssistantMessage
|
||||
expect(message.content[0]).toEqual({ type: 'toolCall', id: 'c1', name: 'f', arguments: {} })
|
||||
})
|
||||
|
||||
it('parses non-object argument JSON (arrays, scalars) to {}', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'tool-call', id: CallId('c1'), name: 'f', arguments: '[1,2]' }],
|
||||
}],
|
||||
})
|
||||
expect((context.messages[0] as AssistantMessage).content[0]).toMatchObject({ arguments: {} })
|
||||
})
|
||||
|
||||
it('recovers toolName for tool results from the preceding assistant call', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'tool-call', id: CallId('c1'), name: 'get_weather', arguments: '{}' }],
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: [{ type: 'tool-result', toolCallId: CallId('c1'), content: [{ type: 'text', text: 'Sunny' }] }],
|
||||
},
|
||||
],
|
||||
})
|
||||
expect(context.messages[1]).toEqual({
|
||||
role: 'toolResult',
|
||||
toolCallId: 'c1',
|
||||
toolName: 'get_weather',
|
||||
content: [{ type: 'text', text: 'Sunny' }],
|
||||
isError: false,
|
||||
timestamp: 0,
|
||||
})
|
||||
})
|
||||
|
||||
it('labels unmatched tool results with toolName unknown and keeps isError', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [{
|
||||
role: 'user',
|
||||
content: [{ type: 'tool-result', toolCallId: CallId('zz'), content: [], isError: true }],
|
||||
}],
|
||||
})
|
||||
expect(context.messages[0]).toMatchObject({
|
||||
role: 'toolResult',
|
||||
toolName: 'unknown',
|
||||
isError: true,
|
||||
content: [{ type: 'text', text: '(no output)' }],
|
||||
})
|
||||
})
|
||||
|
||||
it('splits mixed user text + tool results and folds history system messages', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [
|
||||
{ role: 'system', content: [{ type: 'text', text: 'rule' }] },
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: 'note' },
|
||||
{ type: 'tool-result', toolCallId: CallId('c1'), content: [{ type: 'text', text: 'ok' }] },
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
expect(context.messages.map(message => message.role)).toEqual(['user', 'user', 'toolResult'])
|
||||
})
|
||||
|
||||
it('skips image and unknown blocks in assistant content', () => {
|
||||
const context = toPiContext({
|
||||
model: 'm',
|
||||
messages: [{
|
||||
role: 'assistant',
|
||||
content: [
|
||||
{ type: 'image', url: 'data:,x' },
|
||||
{ type: 'text', text: 'visible' },
|
||||
],
|
||||
}],
|
||||
})
|
||||
expect((context.messages[0] as AssistantMessage).content).toEqual([{ type: 'text', text: 'visible' }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('toStreamChunks', () => {
|
||||
const partialWithToolCall = assistant({
|
||||
content: [{ type: 'toolCall', id: 'call-1', name: 'f', arguments: {} }],
|
||||
})
|
||||
|
||||
it('maps text events to text blocks', async () => {
|
||||
const done = assistant({ content: [{ type: 'text', text: 'hi' }], usage: usage(3, 2) })
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'start', partial: assistant() },
|
||||
{ type: 'text_start', contentIndex: 0, partial: assistant() },
|
||||
{ type: 'text_delta', contentIndex: 0, delta: 'hi', partial: assistant() },
|
||||
{ type: 'text_end', contentIndex: 0, content: 'hi', partial: assistant() },
|
||||
{ type: 'done', reason: 'stop', message: done },
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: 'hi' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: 'hi' } },
|
||||
{ type: 'usage', usage: { inputTokens: 3, outputTokens: 2 } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('maps thinking events to reasoning blocks', async () => {
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'thinking_start', contentIndex: 0, partial: assistant() },
|
||||
{ type: 'thinking_delta', contentIndex: 0, delta: 'mull', partial: assistant() },
|
||||
{ type: 'thinking_end', contentIndex: 0, content: 'mull', partial: assistant() },
|
||||
{ type: 'done', reason: 'stop', message: assistant() },
|
||||
)))
|
||||
expect(chunks.slice(0, 3)).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'reasoning' },
|
||||
{ type: 'reasoning-delta', index: 0, text: 'mull' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'reasoning', text: 'mull' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('maps tool-call events, re-stringifying parsed arguments', async () => {
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'toolcall_start', contentIndex: 0, partial: partialWithToolCall },
|
||||
{ type: 'toolcall_delta', contentIndex: 0, delta: '{"a"', partial: partialWithToolCall },
|
||||
{ type: 'toolcall_delta', contentIndex: 0, delta: ':1}', partial: partialWithToolCall },
|
||||
{
|
||||
type: 'toolcall_end',
|
||||
contentIndex: 0,
|
||||
toolCall: { type: 'toolCall', id: 'call-1', name: 'f', arguments: { a: 1 } },
|
||||
partial: partialWithToolCall,
|
||||
},
|
||||
{ type: 'done', reason: 'toolUse', message: assistant({ stopReason: 'toolUse' }) },
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{ type: 'tool-call-delta', index: 0, id: 'call-1', name: 'f', argumentsDelta: '{"a"' },
|
||||
{ type: 'tool-call-delta', index: 0, id: 'call-1', name: 'f', argumentsDelta: ':1}' },
|
||||
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: 'call-1', name: 'f', arguments: '{"a":1}' } },
|
||||
{ type: 'usage', usage: { inputTokens: 0, outputTokens: 0 } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('tolerates toolcall_start with a missing partial entry', async () => {
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'toolcall_start', contentIndex: 0, partial: assistant() },
|
||||
{ type: 'toolcall_delta', contentIndex: 0, delta: '{}', partial: assistant() },
|
||||
{ type: 'done', reason: 'stop', message: assistant() },
|
||||
)))
|
||||
expect(chunks[1]).toEqual({ type: 'tool-call-delta', index: 0, id: '', argumentsDelta: '{}' })
|
||||
})
|
||||
|
||||
it('maps error events to error finish chunks (in-stream error style)', async () => {
|
||||
const error = assistant({ stopReason: 'error', errorMessage: 'boom', usage: usage(1, 0) })
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'error', reason: 'error', error },
|
||||
)))
|
||||
expect(chunks).toEqual([
|
||||
{ type: 'usage', usage: { inputTokens: 1, outputTokens: 0 } },
|
||||
{ type: 'finish', reason: { kind: 'error', message: 'boom', code: 'PI_AI_ERROR' } },
|
||||
])
|
||||
})
|
||||
|
||||
it('maps aborted error events to aborted finish', async () => {
|
||||
const error = assistant({ stopReason: 'aborted' })
|
||||
const chunks = await collect(toStreamChunks(feed({ type: 'error', reason: 'aborted', error })))
|
||||
expect(chunks.at(-1)).toEqual({ type: 'finish', reason: { kind: 'aborted' } })
|
||||
})
|
||||
})
|
||||
|
||||
describe('mapStopReason / mapUsage', () => {
|
||||
it.each([
|
||||
['stop', { kind: 'stop' }],
|
||||
['length', { kind: 'max-tokens' }],
|
||||
['toolUse', { kind: 'tool-calls' }],
|
||||
['aborted', { kind: 'aborted' }],
|
||||
] as const)('maps %s', (stopReason, expected) => {
|
||||
expect(mapStopReason(assistant({ stopReason }))).toEqual(expected)
|
||||
})
|
||||
|
||||
it('defaults the error message when pi-ai omits it', () => {
|
||||
expect(mapStopReason(assistant({ stopReason: 'error' })))
|
||||
.toEqual({ kind: 'error', message: 'pi-ai stream error', code: 'PI_AI_ERROR' })
|
||||
})
|
||||
|
||||
it('maps cache fields only when nonzero', () => {
|
||||
expect(mapUsage(usage(10, 5, 8, 2))).toEqual({
|
||||
inputTokens: 10,
|
||||
outputTokens: 5,
|
||||
cacheReadTokens: 8,
|
||||
cacheWriteTokens: 2,
|
||||
})
|
||||
expect(mapUsage(usage(10, 5))).toEqual({ inputTokens: 10, outputTokens: 5 })
|
||||
})
|
||||
})
|
||||
|
||||
describe('toStreamChunks edge branches', () => {
|
||||
it('omits the name field for tool calls whose partial carried an empty name', async () => {
|
||||
const blank = assistant({ content: [{ type: 'toolCall', id: 'x', name: '', arguments: {} }] })
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'toolcall_start', contentIndex: 0, partial: blank },
|
||||
{ type: 'toolcall_delta', contentIndex: 0, delta: '{}', partial: blank },
|
||||
{ type: 'done', reason: 'stop', message: assistant() },
|
||||
)))
|
||||
expect(chunks[1]).toEqual({ type: 'tool-call-delta', index: 0, id: 'x', argumentsDelta: '{}' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('toStreamChunks defensive branches', () => {
|
||||
it('tolerates a toolcall_delta with no preceding toolcall_start', async () => {
|
||||
const chunks = await collect(toStreamChunks(feed(
|
||||
{ type: 'toolcall_delta', contentIndex: 0, delta: '{}', partial: assistant() },
|
||||
{ type: 'done', reason: 'stop', message: assistant() },
|
||||
)))
|
||||
expect(chunks[0]).toEqual({ type: 'tool-call-delta', index: 0, id: '', argumentsDelta: '{}' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib"
|
||||
},
|
||||
"include": ["src"],
|
||||
"references": [
|
||||
{ "path": "../../vendor/cosmokit" },
|
||||
{ "path": "../../vendor/cordis" },
|
||||
{ "path": "../../vendor/schemastery" },
|
||||
{ "path": "../llm" }
|
||||
]
|
||||
}
|
||||
@@ -30,9 +30,14 @@ declare module 'cordis' {
|
||||
}
|
||||
}
|
||||
|
||||
/** Typed error for LLM-related failures. The `code` string enables programmatic handling. */
|
||||
/**
|
||||
* Typed error for LLM-related failures. The `code` string enables programmatic
|
||||
* handling (e.g. `AUTH`, `RATE_LIMIT`, `NO_ADAPTER`); `status` carries the HTTP
|
||||
* status when the error originated from a non-2xx provider response (absent for
|
||||
* protocol/usage errors that have no HTTP status).
|
||||
*/
|
||||
export class LlmError extends Error {
|
||||
constructor(message: string, public code: string) {
|
||||
constructor(message: string, public code: string, public status?: number) {
|
||||
super(message)
|
||||
this.name = 'LlmError'
|
||||
}
|
||||
@@ -45,8 +50,10 @@ export class LlmError extends Error {
|
||||
* StreamChunk) and one provider's wire format. Adapters register themselves
|
||||
* via `ctx.llm.registerAdapter(models, adapter)`.
|
||||
*
|
||||
* TODO: the first real adapter (DeepSeek V4) lands in a later phase; until
|
||||
* then only mock adapters (tests, demo) exist.
|
||||
* Real implementations: `@deepseek-ai/dsh-llm-deepseek` (hand-rolled
|
||||
* fetch/SSE) and `@deepseek-ai/dsh-llm-pi-ai` (pi-ai-backed) — two
|
||||
* deliberately different internals over the same contract; see the
|
||||
* adapter contract documented on `StreamChunk` in `./types.ts`.
|
||||
*/
|
||||
export abstract class LlmAdapter {
|
||||
/** Stream one model call as raw chunks. The only required method. */
|
||||
|
||||
@@ -111,7 +111,14 @@ export interface FinishReasonMap {
|
||||
|
||||
export type FinishReason = FinishReasonMap[keyof FinishReasonMap]
|
||||
|
||||
/** Token accounting for one model call (cache fields are optional). */
|
||||
/**
|
||||
* Token accounting for one model call (cache fields are optional).
|
||||
*
|
||||
* Counts are DISJOINT: `inputTokens` is uncached input only; cached input is
|
||||
* reported separately as `cacheReadTokens`/`cacheWriteTokens` (billed input =
|
||||
* sum of the three). Adapters whose providers fold cache hits into a total
|
||||
* prompt count (DeepSeek's `prompt_tokens`) subtract them out.
|
||||
*/
|
||||
export interface TokenUsage {
|
||||
inputTokens: number
|
||||
outputTokens: number
|
||||
@@ -128,9 +135,19 @@ export interface TokenUsage {
|
||||
* carries the fully-assembled ContentBlock so consumers don't have to
|
||||
* re-assemble deltas themselves (use {@link BlockAssembler} when they do).
|
||||
*
|
||||
* TODO(review): this protocol needs careful review before the first real
|
||||
* adapter lands (DeepSeek V4 wire format, partial JSON arguments, interleaved
|
||||
* reasoning signatures, …).
|
||||
* Adapter contract — every adapter MUST obey these, and every consumer may
|
||||
* rely on them:
|
||||
* - Emit `usage` BEFORE `finish`, and nothing after `finish` (defer both to
|
||||
* the provider's end-of-stream marker so trailing usage-only chunks can't
|
||||
* violate this).
|
||||
* - Tool-call `arguments` stay RAW JSON strings end-to-end; partial fragments
|
||||
* stream via `argumentsDelta` (providers that hand back parsed objects
|
||||
* re-stringify at `block-end`).
|
||||
* - Failures may either THROW from `stream()` (transport/protocol errors) or
|
||||
* end the stream with `finish {kind:'error'|'aborted'}` (provider in-band
|
||||
* errors, for adapters that can't throw mid-stream); consumers must handle
|
||||
* both. The agent loop translates a finish-error/aborted into a turn error —
|
||||
* it never logs a normal completed assistant message for a failed step.
|
||||
*/
|
||||
export type StreamChunk =
|
||||
| { type: 'block-start'; index: number; blockType: ContentBlockType }
|
||||
@@ -168,6 +185,11 @@ export interface GenerateOptions {
|
||||
prefill?: ContentBlock[]
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Stop sequences: generation halts as soon as the model produces any one of
|
||||
* these strings (adapters map to the provider's stop field, e.g. OpenAI
|
||||
* `stop`). The stop string itself is not included in the output.
|
||||
*/
|
||||
stop?: string[]
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
@@ -11,6 +11,8 @@ const packages = [
|
||||
'packages/agent',
|
||||
'packages/agent-loop',
|
||||
'packages/bash',
|
||||
'packages/llm-deepseek',
|
||||
'packages/llm-pi-ai',
|
||||
'packages/bash-local',
|
||||
'packages/tool-bash',
|
||||
]
|
||||
|
||||
@@ -41,6 +41,8 @@
|
||||
"@deepseek-ai/dsh-agent": ["./packages/agent/src"],
|
||||
"@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"],
|
||||
"@deepseek-ai/dsh-bash": ["./packages/bash/src"],
|
||||
"@deepseek-ai/dsh-llm-deepseek": ["./packages/llm-deepseek/src"],
|
||||
"@deepseek-ai/dsh-llm-pi-ai": ["./packages/llm-pi-ai/src"],
|
||||
"@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"],
|
||||
"@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"]
|
||||
}
|
||||
|
||||
@@ -17,6 +17,8 @@
|
||||
{ "path": "./packages/tools" },
|
||||
{ "path": "./packages/agent-loop" },
|
||||
{ "path": "./packages/bash" },
|
||||
{ "path": "./packages/llm-deepseek" },
|
||||
{ "path": "./packages/llm-pi-ai" },
|
||||
{ "path": "./packages/bash-local" },
|
||||
{ "path": "./packages/tool-bash" }
|
||||
]
|
||||
|
||||
@@ -23,6 +23,8 @@
|
||||
"@deepseek-ai/dsh-agent": ["./packages/agent/src"],
|
||||
"@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"],
|
||||
"@deepseek-ai/dsh-bash": ["./packages/bash/src"],
|
||||
"@deepseek-ai/dsh-llm-deepseek": ["./packages/llm-deepseek/src"],
|
||||
"@deepseek-ai/dsh-llm-pi-ai": ["./packages/llm-pi-ai/src"],
|
||||
"@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"],
|
||||
"@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import tsconfigPaths from 'vite-tsconfig-paths'
|
||||
import { defineConfig } from 'vitest/config'
|
||||
|
||||
// Real-API end-to-end tests: `yarn test:e2e`, file pattern *.e2e.ts.
|
||||
// Separate from the default suite (`yarn test`, *.spec.ts) on purpose —
|
||||
// these hit the live DeepSeek API, spend tokens, and need a key.
|
||||
//
|
||||
// Secrets: tests gate themselves with
|
||||
// `describe.skipIf(!process.env.DEEPSEEK_API_KEY)`, so the suite passes
|
||||
// (all-skipped) without credentials — CI has none and stays green. Put the
|
||||
// key in the environment or in a gitignored `.env` at the repo root:
|
||||
//
|
||||
// DEEPSEEK_API_KEY=sk-…
|
||||
// DEEPSEEK_BASE_URL=https://… # optional, defaults to the public API
|
||||
try {
|
||||
// Node >= 21.7 native; throws when the file does not exist.
|
||||
process.loadEnvFile(new URL('.env', import.meta.url).pathname)
|
||||
} catch {
|
||||
// No .env — fine, the environment may already carry the variables.
|
||||
}
|
||||
|
||||
export default defineConfig({
|
||||
// Same resolution note as vitest.config.ts: bare workspace names resolve
|
||||
// through the root tsconfig paths map; the native option cannot do this.
|
||||
plugins: [tsconfigPaths({ projects: ['./tsconfig.test.json'] })],
|
||||
test: {
|
||||
include: ['packages/*/tests/**/*.e2e.ts', 'examples/*/tests/**/*.e2e.ts'],
|
||||
// Real model calls: generous timeouts, one retry for transient flakes,
|
||||
// no coverage (unit suites own the coverage gate).
|
||||
testTimeout: 120_000,
|
||||
hookTimeout: 30_000,
|
||||
retry: 1,
|
||||
},
|
||||
})
|
||||
Reference in New Issue
Block a user