fix(llm): stop exposing the mock server binary
This commit is contained in:
Yichen Jiang
+3
-3
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-25-scriptable-llm-wire-fault-server.md: 0795f71f0eab1a107740aaa8cba6fa04b1fbd306
|
||||
2026-07-25-scriptable-llm-wire-fault-server.zh.md: a0e3c98d729adcc74e6d98933ab2beb538f71cb3
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-25-scriptable-llm-wire-fault-server.md
|
||||
2026-07-25-scriptable-llm-wire-fault-server.md: b8e64d92db224c199d0f8c547f0caa588d0ca65e
|
||||
2026-07-25-scriptable-llm-wire-fault-server.zh.md: 35b27efa99b625fa3c815bbe58fef6a138477e55
|
||||
@@ -12,7 +12,7 @@ Connection refusal, a reset before the first event, clean EOF without `[DONE]`,
|
||||
|
||||
## Decision
|
||||
|
||||
`@deepseek-ai/dsh-llm-mock-server` is a support package with an importable Node HTTP server and a standalone CLI. It accepts OpenAI-compatible root and `/v1` chat-completions paths, validates an optional bearer token, captures requests, and consumes one explicit behavior per accepted request. Script exhaustion fails loud; repetition requires `repeatLast`.
|
||||
`@deepseek-ai/dsh-llm-mock-server` is a private support package with an importable Node HTTP server. The repository-local `pnpm run mock:llm` source entry provides a standalone process for manual fault injection; the package exposes no installable binary. It accepts OpenAI-compatible root and `/v1` chat-completions paths, validates an optional bearer token, captures requests, and consumes one explicit behavior per accepted request. Script exhaustion fails loud; repetition requires `repeatLast`.
|
||||
|
||||
Request behaviors cover socket reset, post-header disconnect, partial disconnect, stall, valid empty completion, clean truncated streams, malformed payloads, representative HTTP failures, complete text/reasoning/tool-call responses, slow streaming, and max-token completion. A true `connection_refused` is a CLI listener-lifecycle phase because a bound request handler cannot refuse its own TCP connection.
|
||||
|
||||
@@ -32,10 +32,12 @@ Package tests exercise every request behavior, split UTF-8 request decoding, HTT
|
||||
|
||||
**Use only an in-process `LlmAdapter` mock** — rejected because it bypasses fetch, HTTP status/header parsing, SSE framing, socket termination, and the adapter idle watchdog: the exact boundaries this test infrastructure exists to exercise.
|
||||
|
||||
**Expose an installable workspace binary** — rejected because pnpm links dependency binaries before repository build outputs exist, coupling clean installs to a test-only artifact. The repository-local source command supports the same manual fault injection without adding a package installation surface.
|
||||
|
||||
**Change retry defaults with the server** — rejected because the server reveals existing semantics rather than deciding policy. Extending recovery to `STREAM_CLOSED` requires a separate decision with its own cost, latency, and duplicate-generation trade-offs.
|
||||
|
||||
## Consequences
|
||||
|
||||
Developers can reproduce fault sequences by changing only provider URL/key configuration, and automated tests can keep socket-level failures deterministic through explicit scripts and seeds. The same wire fixture now exposes gaps between hard resets, clean truncation, and recovered empty completions without splicing attempts or modifying model history.
|
||||
|
||||
The server adds a support package, executable build entry, and behavior vocabulary that must remain compatible with both direct tests and CLI examples. Arrival-ordered scripts are intentionally shared across clients, random defaults are stress weights rather than operational truth, and exact connection refusal requires coordinating the client attempt with the pre-listen interval.
|
||||
The server adds a private support package and behavior vocabulary that must remain compatible with both direct tests and repository-local CLI examples. Arrival-ordered scripts are intentionally shared across clients, random defaults are stress weights rather than operational truth, and exact connection refusal requires coordinating the client attempt with the pre-listen interval.
|
||||
@@ -12,9 +12,9 @@ Status: implemented
|
||||
|
||||
## 决策
|
||||
|
||||
`@deepseek-ai/dsh-llm-mock-server` 是一个支持包（package），提供可导入的 Node HTTP 服务器和独立 CLI（命令行界面）。它接受兼容 OpenAI 的根路径和 `/v1` chat-completions 路径，校验可选的 bearer token，捕获请求，并对每个已接受请求消耗一个显式行为。脚本耗尽时快速失败；只有设置 `repeatLast` 才会重复最后一个行为。
|
||||
`@deepseek-ai/dsh-llm-mock-server` 是一个私有支持包（package），提供可导入的 Node HTTP 服务器。仓库内的 `pnpm run mock:llm` 源码入口提供一个用于手动故障注入的独立进程；该包不公开可安装的二进制命令。它接受兼容 OpenAI 的根路径和 `/v1` chat-completions 路径，校验可选的 bearer token，捕获请求，并对每个已接受请求消耗一个显式行为。脚本耗尽时快速失败；只有设置 `repeatLast` 才会重复最后一个行为。
|
||||
|
||||
请求行为覆盖 socket 重置、发送 header 后断开、发送部分内容后断开、停滞、合法空完成、正常关闭但被截断的流、畸形 payload、典型 HTTP 故障、完整的文本/推理/工具调用响应、慢速流式输出以及达到 token 上限的完成。真正的 `connection_refused` 由 CLI 的监听器生命周期阶段实现，因为已经绑定端口的请求处理器无法拒绝自身的 TCP 连接。
|
||||
请求行为覆盖 socket 重置、发送 header 后断开、发送部分内容后断开、停滞、合法空完成、正常关闭但被截断的流、畸形 payload、典型 HTTP 故障、完整的文本/推理/工具调用响应、慢速流式输出以及达到 token 上限的完成。真正的 `connection_refused` 由 CLI（命令行界面）的监听器生命周期阶段实现，因为已经绑定端口的请求处理器无法拒绝自身的 TCP 连接。
|
||||
|
||||
脚本项 `random` 会为每个请求重新执行一次加权选择。服务器公开并记录其无符号 32 位 seed，允许调用方提供相对权重，并内置一套偏重成功结果的压力测试配置，将传输、协议、提供方、超时和语义空结果混合在一起。该配置用于提供可调的测试压力，并非对生产事故发生频率的估算；`connection_refused` 仍不进入请求级随机池。
|
||||
|
||||
@@ -32,10 +32,12 @@ Status: implemented
|
||||
|
||||
**仅使用进程内的 `LlmAdapter` mock**：不予采纳。它会绕过 fetch、HTTP 状态与 header 解析、SSE 分帧、socket 终止以及适配器的空闲看门狗，而这正是这套测试基础设施要覆盖的边界。
|
||||
|
||||
**公开可安装的 workspace 二进制命令**：不予采纳。pnpm 会在仓库构建产物存在之前链接依赖项的二进制命令，从而让干净安装与仅供测试的产物产生耦合。仓库内的源码命令支持相同的手动故障注入，而不会新增包安装接口。
|
||||
|
||||
**随服务器一起修改默认重试策略**：不予采纳。服务器用于揭示既有语义，而非决定策略。是否将恢复能力扩展到 `STREAM_CLOSED`，需要单独决策，并权衡成本、延迟和重复生成风险。
|
||||
|
||||
## 后果
|
||||
|
||||
开发者只需修改提供方 URL/key 配置即可复现故障序列；自动化测试则可通过显式脚本和 seed，让 socket 层故障保持确定性。同一套协议 fixture 现在可以暴露硬重置、正常截断与恢复后的空完成之间的差异，而不会拼接多次尝试的内容或修改模型历史。
|
||||
|
||||
服务器新增了一个支持包、可执行构建入口和行为词汇，三者必须同时兼容直接测试与 CLI 示例。按请求到达顺序执行的脚本有意由所有客户端共享；随机模式的默认值代表压力测试权重，而非实际运行规律；精确模拟连接遭拒时，需要让客户端尝试与监听开始前的时间区间协调一致。
|
||||
服务器新增了一个私有支持包和一套行为词汇，二者必须同时兼容直接测试与仓库内的 CLI 示例。按请求到达顺序执行的脚本有意由所有客户端共享；随机模式的默认值代表压力测试权重，而非实际运行规律；精确模拟连接遭拒时，需要让客户端尝试与监听开始前的时间区间协调一致。
|
||||
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
README.md: 77a5c5e35fe0b4b1c92968eecea85b6059c889fc
|
||||
README.zh.md: bf84a1c5f5428e845a6917a82287733d82405144
|
||||
# pnpm run verify-translation-pairing --write packages/support/llm-mock-server/README.md
|
||||
README.md: a535c086bf688ad48b1a3bb19c7b81da21cbad92
|
||||
README.zh.md: e013cc47399da7fdde42c10dfe086ffab9105785
|
||||
@@ -26,7 +26,7 @@ DEEPSEEK_API_KEY=mock-key \
|
||||
pnpm run demo:headless "test provider recovery"
|
||||
```
|
||||
|
||||
The built package also exposes `dsh-llm-mock-server`. Stdout is JSONL: a `ready` record carries the `/v1` base URL and random seed, followed by request/result records that name both the scripted behavior and the concrete selected behavior.
|
||||
The repository script writes JSONL to stdout: a `ready` record carries the `/v1` base URL and random seed, followed by request/result records that name both the scripted behavior and the concrete selected behavior. The private support package exposes no installable binary.
|
||||
|
||||
## Behavior script
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ DEEPSEEK_API_KEY=mock-key \
|
||||
pnpm run demo:headless "test provider recovery"
|
||||
```
|
||||
|
||||
构建包还公开 `dsh-llm-mock-server`。Stdout 是 JSONL：`ready` 记录携带 `/v1` base URL 和随机种子，后续请求/结果记录同时命名脚本行为和实际选中的具体行为。
|
||||
仓库脚本将 JSONL 写入 stdout：`ready` 记录携带 `/v1` base URL 和随机种子，后续请求/结果记录同时命名脚本行为和实际选中的具体行为。这个私有支持包（package）不公开可安装的二进制命令。
|
||||
|
||||
## 行为脚本
|
||||
|
||||
|
||||
@@ -6,9 +6,6 @@
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"bin": {
|
||||
"dsh-llm-mock-server": "lib/bin.js"
|
||||
},
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
@@ -18,17 +15,12 @@
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./bin": {
|
||||
"types": "./lib/types/bin.d.ts",
|
||||
"default": "./lib/bin.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/bin.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Standalone process wrapper for the scriptable mock LLM server.
|
||||
* @module @deepseek-ai/dsh-llm-mock-server/bin
|
||||
* @module @deepseek-ai/dsh-llm-mock-server/src/bin
|
||||
*/
|
||||
|
||||
import { setTimeout as delay } from 'node:timers/promises'
|
||||
|
||||
@@ -10,8 +10,4 @@ export default defineConfig([
|
||||
entry: ['lib/types/invariant.js'], outDir: 'lib', format: ['esm'], platform: 'node', target: 'es2024',
|
||||
fixedExtension: false, outputOptions: { codeSplitting: false }, dts: false, clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/bin.js'], outDir: 'lib', format: ['esm'], platform: 'node', target: 'es2024',
|
||||
fixedExtension: false, outputOptions: { codeSplitting: false }, dts: false, clean: false,
|
||||
},
|
||||
])
|
||||
Reference in New Issue
Block a user