fix(pkg): support worker-backed tools in single exe
pkg stores application files in a virtual filesystem, and its worker_threads hook discovers worker entry points only when they are passed as filesystem strings. Convert the code-runtime entry with fileURLToPath() and return the workflow built entry as a string while retaining its source-mode data URL bootstrap. Emit worker entry bundles as CommonJS .cjs files. pkg executes a VFS-backed string-path worker through Module._compile, so an ESM-only entry can be present in the executable yet still fail when launched. Keep the public hosts ESM, adapt worker startup accordingly, and align exports, package file lists, workspace constraints, documentation, and built-worker tests with the actual artifact format. Expand the custom-config executable smoke to load the Code Mode and workflow plugins and script real run_code and zero-agent workflow calls. Require both tools to return 42 from workers launched inside the pkg VFS, turning worker support from an asset-presence assumption into an end-to-end runtime contract. Update the implemented RFC and verification gates to describe and exercise the supported built-worker path. This adds no tool or JSON-RPC protocol shape; it fixes how existing worker-backed capabilities are located and executed in the single-file distribution.
This commit is contained in:
Tianyi Cui
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 08fd6f530d1f564ca169a168c0f7d77bff6bf29f
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 050e4f9a6e8bb03e59835f2c06dd3c701aa34eed
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 511f5f2cf8b43aac5960dfe4630f489ec3dc5086
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 6774fc7ec2d9439154305ba0ae35dceeff922a4a
|
||||
+3
-3
@@ -58,11 +58,11 @@ The exe's "must be explicitly configured" hard semantic is unchanged; the zero-c
|
||||
|
||||
## Disposition of worker-style plugins
|
||||
|
||||
`dsh-workflow-workerthread` and `dsh-code-runtime-worker` depend on on-disk sibling files via `new Worker(new URL('./worker.js', import.meta.url))`. PoC measurement: pkg's Worker patch intercepts **string paths only**; the URL-object form finds no file inside the VFS. The fix is already known (`fileURLToPath()` to a string), but this round's review decision is **do not verify, do not promise, do not handle**: they compile into the exe with the full set, and behavior when an external `cordis.yml` references them is undefined.
|
||||
`dsh-workflow-workerthread` and `dsh-code-runtime-worker` are supported inside the exe. Their built entries convert the sibling worker URL with `fileURLToPath()` and pass the resulting filesystem string to `Worker`, which is the form pkg's Worker hook resolves inside the VFS. The workflow engine keeps its data-URL bootstrap for unbuilt source execution; only its built sibling entry uses the filesystem string. The custom-config executable smoke loads both backends, invokes a real `run_code` call and a zero-agent `workflow` call, and requires each worker to return `42` from inside pkg's VFS.
|
||||
|
||||
## Testing
|
||||
|
||||
The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, and the direct binary protocol, with final text and JSONL checked; its platform wheel is then installed in a clean venv and run without `runtime_bin`. The JSON-RPC protocol is not part of the ACP snapshot system, so there is no snapshot tier (an explicitly named gap, not an oversight).
|
||||
The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, and the direct binary protocol, with final text and JSONL checked. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. Its platform wheel is then installed in a clean venv and run without `runtime_bin`. The JSON-RPC protocol is not part of the ACP snapshot system, so there is no snapshot tier (an explicitly named gap, not an oversight).
|
||||
|
||||
Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disposes immediately, so a short-lived pipe aborts an in-flight turn — pipe-driven runs must keep stdin open until the turn ends.
|
||||
|
||||
@@ -82,4 +82,4 @@ Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disp
|
||||
|
||||
**Bought**: zero-dependency single-file distribution on target platforms; plugin semantics strictly identical to running from source (the same real package tree, no transpilation, no registry); the serving surface, the plugin set, and the configuration all converge on two sources of truth — `cordis.yml` plus one dependency manifest; the exe and node carriers share one tree and one semantics, so development verification never waits for packaging; official Node binaries remove the patched-binary supply-chain concern.
|
||||
|
||||
**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (the build script pins `@yao-pkg/pkg@6.21.0`; upgrading is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial); worker-style plugins have undefined behavior inside the exe.
|
||||
**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (the build script pins `@yao-pkg/pkg@6.21.0`; upgrading is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial).
|
||||
+4
-4
@@ -56,13 +56,13 @@ exe「必须显式配置」的硬语义不变；零配置体验由 wrapper 恢
|
||||
|
||||
`@deepseek-ai/dsh-jsonrpc-agent`（包）→ `dsh-jsonrpc-agent`（bin）→ `dsh-jsonrpc-agent-pkg`（闭包清单；无 scope 前缀，刻意避开 constraints 对 `@deepseek-ai/dsh-*` 的包形状规则）→ `dsh-jsonrpc-agent-pkg-<platform>-<arch>`（exe 产物）。wire `serverInfo.name` 保持 `deepseek-harness-sdk-runtime`（协议稳定值）；Python dist 名为 `deepseek-harness` / `deepseek-harness-runtime-bin`。
|
||||
|
||||
## worker 类插件的处置
|
||||
## 工作线程插件
|
||||
|
||||
`dsh-workflow-workerthread` 与 `dsh-code-runtime-worker` 以 `new Worker(new URL('./worker.js', import.meta.url))` 依赖磁盘兄弟文件。PoC 实测：pkg 的 Worker 补丁只拦截**字符串路径**，URL 对象形态在 VFS 内找不到文件——修法已明确（`fileURLToPath()` 转字符串），但本期评审决策是**不验证、不承诺、不处理**：它们随全量集合编译进 exe，外部 `cordis.yml` 引用时行为未定义。
|
||||
exe 内支持 `dsh-workflow-workerthread` 与 `dsh-code-runtime-worker`。两个后端的构建入口都通过 `fileURLToPath()` 转换相邻 worker 的 URL，再将所得文件系统字符串传给 `Worker`；pkg 的 Worker 钩子可以用这种形式解析 VFS 内文件。工作流引擎在未构建的源码执行中仍保留 data URL 引导程序，只有构建后的相邻入口使用文件系统字符串。自定义配置的可执行文件冒烟测试会加载两个后端，实际调用 `run_code` 与不启动 agent 的 `workflow`，并要求两个 worker 都从 pkg 的 VFS 内返回 `42`。
|
||||
|
||||
## 测试
|
||||
|
||||
验证面分三层。机制层：`--sea` 链路的实测结论内嵌在「决策」各节（VFS 内 ESM 动态 import、cordis 单实例、fail-loud 配置链路、`node:sqlite`、macOS ad-hoc 签名可运行）。SDK 层：完整的 keyless pytest 套件以假运行时对端覆盖客户端协议、子进程清理、绝对 cwd 传递、双载体启动与载体解析；根 CI 在 Python 3.10 上运行全部用例。端到端层：每个平台构建都通过默认 SDK 路径、自定义配置和直接二进制协议对着 mock 端点完成一个轮次，并校验最终文本与 JSONL；随后把平台 wheel 安装进干净 venv，在不传 `runtime_bin` 的情况下运行。JSON-RPC 协议不在 ACP snapshot 体系内，无 snapshot 层（点名后的明确空缺，非遗漏）。
|
||||
验证面分三层。机制层：`--sea` 链路的实测结论内嵌在「决策」各节（VFS 内 ESM 动态 import、cordis 单实例、fail-loud 配置链路、`node:sqlite`、macOS ad-hoc 签名可运行）。SDK 层：完整的 keyless pytest 套件以假运行时对端覆盖客户端协议、子进程清理、绝对 cwd 传递、双载体启动与载体解析；根 CI 在 Python 3.10 上运行全部用例。端到端层：每个平台构建都通过默认 SDK 路径、自定义配置和直接二进制协议对着 mock 端点完成一个轮次，并校验最终文本与 JSONL。自定义配置还会通过打包进 VFS 的真实 worker 文件执行 `run_code` 和不启动 agent 的 `workflow`。随后把平台 wheel 安装进干净 venv，在不传 `runtime_bin` 的情况下运行。JSON-RPC 协议不在 ACP snapshot 体系内，无 snapshot 层（点名后的明确空缺，非遗漏）。
|
||||
|
||||
手工驱动注意：bin 视 stdin EOF 为「客户端已走」并立即 dispose，短命管道会中止在飞回合——管道驱动必须保持 stdin 打开到回合结束。
|
||||
|
||||
@@ -82,4 +82,4 @@ exe「必须显式配置」的硬语义不变；零配置体验由 wrapper 恢
|
||||
|
||||
**买到的**：目标平台零依赖单文件分发；插件语义与源码运行严格一致（同一棵真实包树，无转译无注册表）；serving 面、插件集、配置三者全部收敛到 `cordis.yml` + 一份依赖清单两个事实源；exe 与 node 双载体同树同语义，开发验证不必等打包；官方 Node 二进制消除了补丁二进制供应链顾虑。
|
||||
|
||||
**付出的**：产物 174MB 级且源码原样进 blob（无字节码混淆，闭源分发诉求需另行评估）；pkg 的 VFS/模块钩子层仍是社区维护（构建脚本钉死 `@yao-pkg/pkg@6.21.0`，升级走显式改动）；`--sea` 单 target 单次调用（与 CI 每平台一腿匹配，本地多平台构建串行）；worker 类插件在 exe 内行为未定义。
|
||||
**付出的**：产物 174MB 级且源码原样进 blob（无字节码混淆，闭源分发诉求需另行评估）；pkg 的 VFS/模块钩子层仍是社区维护（构建脚本钉死 `@yao-pkg/pkg@6.21.0`，升级走显式改动）；`--sea` 每个 target 调用一次（与 CI 每平台一腿匹配，本地多平台构建串行）。
|
||||
@@ -26,7 +26,7 @@ One deliberate strictness DIVERGENCE from CC: hook misuse — unknown or deferre
|
||||
|
||||
**Why node:worker_threads**: one run uses one unpooled worker because a workflow run is already heavyweight relative to thread startup. The script runs in a vm context inside the worker, keeping the script-visible surface to the hook contract instead of exposing a bare worker realm, while `agent()` bridges by message-port RPC to I/O-bound child loops on the host. This keeps `start()` from blocking the host on the script's synchronous slice, makes the post-cancel deadline end in a real `worker.terminate()`, and gives cross-thread values a serialization boundary by construction. isolated-vm was rejected for its maintenance state, required `--no-node-snapshot` consumer flag on Node ≥ 20, and node-gyp fallback.
|
||||
|
||||
Host-side meta validation and body pre-parsing preserve the seam's synchronous errors, and private enum-keyed payload maps define the wire protocol. Pending async starts, published child records, one host cancellation signal, worker-death reaping, result precedence, and disposal quiescence preserve the subagent run contract across that wire; the [agent-scope runtime-design RFC](../architecture/2026-07-12-agent-scope-runtime-design.md#workflow-children-are-pending-starts-or-published-records) owns those race algorithms. Coverage uses an in-process `MessageChannel` for worker-side logic that main-process V8 coverage cannot see and separately proves the built `lib/worker.js`—a second tsdown entry sanctioned by the `"./worker"` subpath export—under plain Node in the built-bin smoke gate.
|
||||
Host-side meta validation and body pre-parsing preserve the seam's synchronous errors, and private enum-keyed payload maps define the wire protocol. Pending async starts, published child records, one host cancellation signal, worker-death reaping, result precedence, and disposal quiescence preserve the subagent run contract across that wire; the [agent-scope runtime-design RFC](../architecture/2026-07-12-agent-scope-runtime-design.md#workflow-children-are-pending-starts-or-published-records) owns those race algorithms. Coverage uses an in-process `MessageChannel` for worker-side logic that main-process V8 coverage cannot see and separately proves the built `lib/worker.cjs`—a second tsdown entry sanctioned by the `"./worker"` subpath export—under plain Node in the built-bin smoke gate.
|
||||
|
||||
**Meta as data, never evaluated**: the meta block reaches the seam as a plain JSON request field (the tool's schema-validated `meta` parameter) and the engine only shape-validates it, every violation named. This is a host-isolation invariant, not a convenience: evaluating a meta literal host-side — even one contractually "pure", in an empty timed vm context — hands script-controlled getters a host stack with no timeout the moment the result is READ, defeating the exact spin isolation the worker thread buys.
|
||||
|
||||
|
||||
+1
-1
@@ -25,7 +25,7 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
|
||||
|
||||
- A plugin shipped via `cordis.yml` needs at least one test through the REAL Loader path: hand-built `ctx.plugin({...})` mounts bypass `unwrapExports` and cannot catch a broken export shape ([postmortem 0001](postmortem/0001-acp-default-export-drops-inject.md); export-shape rules in [packages/AGENTS.md](../packages/AGENTS.md)).
|
||||
- A guard only guards if the regression actually fails it. For a plugin without `inject` (bundle/composition plugins), a Loader smoke stays green under a broken export shape — add an explicit `expect('default' in mod).toBe(false)` plus an `unwrapExports` round-trip assertion, and prove it: introduce the regression, watch red, revert.
|
||||
- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). The same applies to any non-index runtime entry the built package resolves at run time (the worker-thread runtime's sibling `lib/worker.js`). Keep the built-artifact smokes green (`packages/ui/*/tests/built-bin.e2e.ts`, `packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts`), and assert a genuinely-missing config exits non-zero.
|
||||
- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). The same applies to any non-index runtime entry the built package resolves at run time (the worker-thread runtime's sibling `lib/worker.cjs`). Keep the built-artifact smokes green (`packages/ui/*/tests/built-bin.e2e.ts`, `packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts`), and assert a genuinely-missing config exits non-zero.
|
||||
- An e2e that spawns an example from a temp cwd sets `TSX_TSCONFIG_PATH` to the repo-root tsconfig, or it silently falls back to stale built `lib/` ([examples/AGENTS.md](../examples/AGENTS.md)).
|
||||
|
||||
## When a snapshot test is required
|
||||
|
||||
@@ -29,4 +29,4 @@ Every field is validated (positive numbers) and defaulted; there are no other tu
|
||||
|
||||
## The worker entry, unbuilt and built
|
||||
|
||||
`worker.ts` is deliberately erasable-only TypeScript with type-only cross-package imports: unbuilt (vitest/tsx), the host spawns `src/worker.ts` directly and Node's native type stripping loads it; built, the entry ships as the sibling bundle `lib/worker.js` (its own tsdown entry). The built path is pinned by `tests/built-lib.e2e.ts`, the real-load-path guard from [docs/testing.md](../../../docs/testing.md).
|
||||
`worker.ts` is deliberately erasable-only TypeScript with type-only cross-package imports: unbuilt (vitest/tsx), the host spawns `src/worker.ts` directly and Node's native type stripping loads it; built, the entry ships as the sibling CommonJS bundle `lib/worker.cjs` (its own tsdown entry). The CommonJS format is required because pkg's VFS Worker hook compiles filesystem-string entries as CommonJS. The host converts either entry URL to a filesystem string before constructing `Worker`, which works through both ordinary Node resolution and that pkg hook. The built path is pinned by `tests/built-lib.e2e.ts`, the real-load-path guard from [docs/testing.md](../../../docs/testing.md).
|
||||
@@ -13,14 +13,14 @@
|
||||
},
|
||||
"./worker": {
|
||||
"types": "./lib/types/worker.d.ts",
|
||||
"default": "./lib/worker.js"
|
||||
"default": "./lib/worker.cjs"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/worker.js",
|
||||
"lib/worker.cjs",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import { stripTypeScriptTypes } from 'node:module'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
|
||||
@@ -98,16 +99,19 @@ interface LiveRun {
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker entry module. Source runs unbuilt (`src/worker.ts`, loadable
|
||||
* The worker entry path. Source runs unbuilt (`src/worker.ts`, loadable
|
||||
* directly on this repo's Node range via native type stripping — the file
|
||||
* is erasable-only with type-only relative imports); the built package
|
||||
* ships it as a sibling bundle (`lib/worker.js`, its own tsdown entry).
|
||||
* ships it as a sibling CommonJS bundle (`lib/worker.cjs`, its own tsdown
|
||||
* entry) because pkg's VFS Worker hook compiles string-path entries as
|
||||
* CommonJS.
|
||||
* The URL *pathname*'s extension says which world this module is in —
|
||||
* pathname, because dev-time module runners (vitest) may suffix
|
||||
* `import.meta.url` with a query string; relative resolution drops it.
|
||||
* `import.meta.url` with a query string; relative resolution drops it. Worker
|
||||
* receives a filesystem string so pkg's VFS Worker hook can resolve it.
|
||||
*/
|
||||
/* v8 ignore next -- the './worker.js' arm is the built-lib world, unreachable unbuilt by construction; the built-lib e2e pins it. */
|
||||
const WORKER_URL = new URL(new URL(import.meta.url).pathname.endsWith('.ts') ? './worker.ts' : './worker.js', import.meta.url)
|
||||
/* v8 ignore next -- the './worker.cjs' arm is the built-lib world, unreachable unbuilt by construction; the built-lib e2e pins it. */
|
||||
const WORKER_PATH = fileURLToPath(new URL(new URL(import.meta.url).pathname.endsWith('.ts') ? './worker.ts' : './worker.cjs', import.meta.url))
|
||||
|
||||
/** Render an unknown thrown value as a message, `Error` or not. */
|
||||
function messageOf(error: unknown): string {
|
||||
@@ -273,7 +277,7 @@ export class WorkerCodeRuntime extends CodeRuntime {
|
||||
maxLogBytes: this.config.maxLogBytes,
|
||||
maxValueBytes: this.config.maxValueBytes,
|
||||
}
|
||||
const worker = new Worker(WORKER_URL, {
|
||||
const worker = new Worker(WORKER_PATH, {
|
||||
workerData: bootData,
|
||||
// Model code gets NO ambient environment — stronger than the scrubbed
|
||||
// env the defensive-patterns rule requires for spawned commands.
|
||||
|
||||
@@ -17,4 +17,4 @@ import type { WorkerBootData } from './protocol.ts'
|
||||
// A worker always has a parent port; guard loudly rather than run detached.
|
||||
if (!parentPort) throw new Error('dsh-code-runtime-worker: worker entry loaded outside a worker thread')
|
||||
|
||||
await runWorkerMain(parentPort, workerData as WorkerBootData, { stdout: process.stdout, stderr: process.stderr })
|
||||
void runWorkerMain(parentPort, workerData as WorkerBootData, { stdout: process.stdout, stderr: process.stderr })
|
||||
@@ -8,7 +8,7 @@ import { describe, expect, it } from 'vitest'
|
||||
* BUILT-ARTIFACT smoke for the published package (the real-load-path guard
|
||||
* from docs/testing.md): the unit suite runs `src/` under vitest, where the
|
||||
* worker entry resolves to `src/worker.ts` — a consumer runs `lib/index.js`
|
||||
* under plain `node`, where it must resolve the sibling `lib/worker.js`
|
||||
* under plain `node`, where it must resolve the sibling `lib/worker.cjs`
|
||||
* bundle instead. This spawns plain `node` (NOT tsx) from inside the package
|
||||
* directory and imports the package BY NAME, so resolution flows through the
|
||||
* real `exports` map exactly as it would from a downstream install; the
|
||||
@@ -21,11 +21,11 @@ import { describe, expect, it } from 'vitest'
|
||||
*/
|
||||
|
||||
const pkgDir = fileURLToPath(new URL('..', import.meta.url))
|
||||
const built = ['lib/index.js', 'lib/worker.js'].every(file => existsSync(join(pkgDir, file)))
|
||||
const built = ['lib/index.js', 'lib/worker.cjs'].every(file => existsSync(join(pkgDir, file)))
|
||||
&& existsSync(join(pkgDir, '../code-runtime/lib/index.js'))
|
||||
|
||||
describe.skipIf(!built)('built lib real load path (plain node)', () => {
|
||||
it('runs a TypeScript program with a binding through lib/index.js and its lib/worker.js entry', async () => {
|
||||
it('runs a TypeScript program with a binding through lib/index.js and its lib/worker.cjs entry', async () => {
|
||||
const script = `
|
||||
const { Context } = await import('cordis')
|
||||
const { WorkerCodeRuntime } = await import('@deepseek-ai/dsh-code-runtime-worker')
|
||||
|
||||
@@ -3,8 +3,10 @@ import { defineConfig } from 'tsdown'
|
||||
/**
|
||||
* Package-shape override (see the root tsdown.config.ts): besides the
|
||||
* default lib/index.js bundle, the worker BOOTSTRAP ships as its own
|
||||
* sibling entry — `new Worker(new URL('./worker.js', import.meta.url))`
|
||||
* loads it as a file, so it cannot be part of the index bundle. TWO
|
||||
* sibling CommonJS entry — `new Worker(fileURLToPath(new URL('./worker.cjs', import.meta.url)))`
|
||||
* loads it as a file, so it cannot be part of the index bundle. pkg's VFS
|
||||
* Worker hook compiles string-path entries as CommonJS, so an ESM worker is
|
||||
* not viable inside the executable. TWO
|
||||
* single-entry builds, not one two-entry build: a multi-entry build emits
|
||||
* the shared bootstrap module as a `lib/bootstrap-*.js` chunk both bundles
|
||||
* import, which the package.json `files` whitelist (deliberately exact)
|
||||
@@ -25,7 +27,7 @@ export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/worker.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
format: ['cjs'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
|
||||
@@ -32,7 +32,7 @@ Unknown options, malformed arguments, unsupported schemas, tripped caps, provide
|
||||
|
||||
## Run sequence
|
||||
|
||||
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. Source mode uses a data-URL bootstrap that installs the TypeScript transforms inside the worker; built mode passes the sibling CommonJS bundle `lib/worker.cjs` as a filesystem string. CommonJS is required because pkg's VFS Worker hook compiles filesystem-string entries in that format; the same entry also works under ordinary Node resolution. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
|
||||
For each `agent()` call:
|
||||
|
||||
|
||||
@@ -13,14 +13,14 @@
|
||||
},
|
||||
"./worker": {
|
||||
"types": "./lib/types/worker.d.ts",
|
||||
"default": "./lib/worker.js"
|
||||
"default": "./lib/worker.cjs"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/worker.js",
|
||||
"lib/worker.cjs",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
|
||||
@@ -43,6 +43,7 @@
|
||||
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import type { WorkerOptions } from 'node:worker_threads'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import type { Context } from 'cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { assertNever } from '@deepseek-ai/dsh-llm'
|
||||
@@ -87,12 +88,12 @@ interface ChildRecord {
|
||||
* AMBIENT channel only — an escapee still holds process-wide privileges
|
||||
* like fs access (the README's trust premise stands).
|
||||
* @param init - the run payload, passed as `workerData`.
|
||||
* @returns the entry URL and the Worker options to spawn it with.
|
||||
* @returns the entry path or URL and the Worker options to spawn it with.
|
||||
*/
|
||||
function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOptions } {
|
||||
function resolveWorkerSpawn(init: WorkerInit): { entry: string | URL; options: WorkerOptions } {
|
||||
/* v8 ignore next 3 -- the built-output arm: tests always run unbuilt (src/); the built-worker e2e exercises this shape for real */
|
||||
if (!import.meta.url.endsWith('.ts')) {
|
||||
return { entry: new URL('./worker.js', import.meta.url), options: { workerData: init, env: {}, execArgv: [] } }
|
||||
return { entry: fileURLToPath(new URL('./worker.cjs', import.meta.url)), options: { workerData: init, env: {}, execArgv: [] } }
|
||||
}
|
||||
// Resolve tsx lazily: only the unbuilt shape executes this arm, so a built
|
||||
// consumer never needs the dev-only loader installed. A JavaScript entry is
|
||||
|
||||
@@ -8,17 +8,17 @@ import { describe, expect, it } from 'vitest'
|
||||
|
||||
const packageRoot = fileURLToPath(new URL('..', import.meta.url))
|
||||
const builtIndex = join(packageRoot, 'lib', 'index.js')
|
||||
const builtWorker = join(packageRoot, 'lib', 'worker.js')
|
||||
const builtWorker = join(packageRoot, 'lib', 'worker.cjs')
|
||||
const run = promisify(execFile)
|
||||
|
||||
/**
|
||||
* The BUILT-output guard for the worker entry: every other suite runs
|
||||
* unbuilt (src/ + tsx), so nothing else proves that `lib/index.js` resolves
|
||||
* its sibling `lib/worker.js` and that the bundle boots a worker under plain
|
||||
* its sibling `lib/worker.cjs` and that the bundle boots a worker under plain
|
||||
* node (no tsx loader). Keyless — a zero-agent script needs no provider —
|
||||
* and self-skips until `pnpm run build` has produced the bundles.
|
||||
*/
|
||||
describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.js)', () => {
|
||||
describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.cjs)', () => {
|
||||
it('the built engine spawns its built worker under plain node and completes a run', async () => {
|
||||
// ESM resolves bare specifiers from the IMPORTING FILE's location, so the
|
||||
// driver must live inside the package for its node_modules to apply — a
|
||||
|
||||
@@ -6,7 +6,9 @@ import { defineConfig } from 'tsdown'
|
||||
* entries are JS emitted by tsc under lib/types and are bundled as two
|
||||
* single-entry passes so shared modules (realm, runtime, session) are inlined
|
||||
* into each instead of split into a hash-named chunk (the worker entry must
|
||||
* be a self-contained file the Worker constructor can load by path).
|
||||
* be a self-contained file the Worker constructor can load by path). The
|
||||
* worker bundle is CommonJS because pkg's VFS Worker hook compiles
|
||||
* filesystem-string entries as CommonJS.
|
||||
*/
|
||||
export default defineConfig([
|
||||
{
|
||||
@@ -22,7 +24,7 @@ export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/worker.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
format: ['cjs'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
|
||||
@@ -110,7 +110,7 @@ const dshBinPackageFiles = [
|
||||
|
||||
const dshWorkerPackageFiles = [
|
||||
'lib/index.js',
|
||||
'lib/worker.js',
|
||||
'lib/worker.cjs',
|
||||
'lib/types/**/*.d.ts',
|
||||
'lib/types/**/*.d.ts.map',
|
||||
'src',
|
||||
|
||||
@@ -330,7 +330,7 @@ function builtBinSmokeGate(): Gate {
|
||||
'packages/ui/stdio-agent/tests/built-bin.e2e.ts',
|
||||
'packages/ui/acp-agent/tests/built-bin.e2e.ts',
|
||||
// The worker-entry packages' built bundles: the only automated proof
|
||||
// that lib/index.js resolves its sibling lib/worker.js under plain node
|
||||
// that lib/index.js resolves its sibling lib/worker.cjs under plain node
|
||||
// (the e2e lane runs unbuilt, so these files self-skip there).
|
||||
'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts',
|
||||
'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts',
|
||||
|
||||
+157
-15
@@ -17,11 +17,18 @@ from typing import Callable
|
||||
|
||||
|
||||
EXPECTED_TEXT = "runtime smoke ok"
|
||||
CODE_PROMPT = "Use run_code to compute the packaged worker smoke value."
|
||||
CODE_WORKER_TEXT = "code worker smoke ok"
|
||||
WORKFLOW_PROMPT = "Use workflow to compute the packaged worker smoke value without agents."
|
||||
WORKFLOW_WORKER_TEXT = "workflow worker smoke ok"
|
||||
CUSTOM_CORDIS = """\
|
||||
- id: jsonrpc
|
||||
name: '@deepseek-ai/dsh-jsonrpc'
|
||||
- id: agent-core
|
||||
name: '@deepseek-ai/dsh-agent-core'
|
||||
config:
|
||||
tools:
|
||||
mode: both
|
||||
- id: sessions
|
||||
name: '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
config:
|
||||
@@ -30,11 +37,21 @@ CUSTOM_CORDIS = """\
|
||||
name: '@deepseek-ai/dsh-bash-local'
|
||||
config:
|
||||
cwd: !!js process.env.DSH_CWD
|
||||
- id: code-runtime
|
||||
name: '@deepseek-ai/dsh-code-runtime-worker'
|
||||
- id: subagents
|
||||
name: '@deepseek-ai/dsh-subagent'
|
||||
- id: workflow-engine
|
||||
name: '@deepseek-ai/dsh-workflow-workerthread'
|
||||
config:
|
||||
provider: spawn
|
||||
- id: workflow-tool
|
||||
name: '@deepseek-ai/dsh-tool-workflow'
|
||||
"""
|
||||
|
||||
|
||||
class MockModelHandler(BaseHTTPRequestHandler):
|
||||
"""Return one deterministic OpenAI-compatible streaming completion."""
|
||||
"""Return deterministic text and worker-tool streaming completions."""
|
||||
|
||||
requests: list[dict[str, object]] = []
|
||||
|
||||
@@ -45,11 +62,7 @@ class MockModelHandler(BaseHTTPRequestHandler):
|
||||
self.send_response(200)
|
||||
self.send_header("content-type", "text/event-stream")
|
||||
self.end_headers()
|
||||
chunks = [
|
||||
{"choices": [{"delta": {"role": "assistant", "content": None, "reasoning_content": ""}}]},
|
||||
{"choices": [{"delta": {"content": EXPECTED_TEXT}}]},
|
||||
{"choices": [{"delta": {"content": ""}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 3, "completion_tokens": 3}},
|
||||
]
|
||||
chunks = completion_chunks(body)
|
||||
for chunk in chunks:
|
||||
self.wfile.write(f"data: {json.dumps(chunk)}\n\n".encode())
|
||||
self.wfile.write(b"data: [DONE]\n\n")
|
||||
@@ -59,6 +72,127 @@ class MockModelHandler(BaseHTTPRequestHandler):
|
||||
return
|
||||
|
||||
|
||||
def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]:
|
||||
"""Choose the next deterministic model response from request history."""
|
||||
messages = body.get("messages")
|
||||
if not isinstance(messages, list) or not messages:
|
||||
raise AssertionError(f"model request has no messages: {body}")
|
||||
latest = messages[-1]
|
||||
if not isinstance(latest, dict):
|
||||
raise AssertionError(f"model request has an invalid latest message: {body}")
|
||||
|
||||
if latest.get("role") == "tool":
|
||||
tool_name = latest_tool_name(messages)
|
||||
tool_text = json.dumps(latest.get("content"))
|
||||
if "42" not in tool_text:
|
||||
raise AssertionError(f"{tool_name} worker returned no expected value: {latest}")
|
||||
if tool_name == "run_code":
|
||||
return text_chunks(CODE_WORKER_TEXT)
|
||||
if tool_name == "workflow":
|
||||
return text_chunks(WORKFLOW_WORKER_TEXT)
|
||||
raise AssertionError(f"unexpected tool follow-up: {tool_name}")
|
||||
|
||||
prompt = message_text(latest.get("content"))
|
||||
if prompt == CODE_PROMPT:
|
||||
assert_advertised_tool(body, "run_code")
|
||||
return tool_call_chunks("call-code-worker", "run_code", {"code": "return 6 * 7"})
|
||||
if prompt == WORKFLOW_PROMPT:
|
||||
assert_advertised_tool(body, "workflow")
|
||||
return tool_call_chunks(
|
||||
"call-workflow-worker",
|
||||
"workflow",
|
||||
{
|
||||
"script": "return 6 * 7",
|
||||
"meta": {
|
||||
"name": "pkg-worker-smoke",
|
||||
"description": "exercise the packaged workflow worker",
|
||||
},
|
||||
},
|
||||
)
|
||||
return text_chunks(EXPECTED_TEXT)
|
||||
|
||||
|
||||
def text_chunks(text: str) -> list[dict[str, object]]:
|
||||
"""Build a complete streaming text response."""
|
||||
return [
|
||||
{"choices": [{"delta": {"role": "assistant", "content": None, "reasoning_content": ""}}]},
|
||||
{"choices": [{"delta": {"content": text}}]},
|
||||
{
|
||||
"choices": [{"delta": {"content": ""}, "finish_reason": "stop"}],
|
||||
"usage": {"prompt_tokens": 3, "completion_tokens": 3},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def tool_call_chunks(call_id: str, name: str, arguments: dict[str, object]) -> list[dict[str, object]]:
|
||||
"""Build a complete streaming function-call response."""
|
||||
return [
|
||||
{"choices": [{"delta": {"role": "assistant", "content": None, "reasoning_content": ""}}]},
|
||||
{
|
||||
"choices": [{
|
||||
"delta": {
|
||||
"tool_calls": [{
|
||||
"index": 0,
|
||||
"id": call_id,
|
||||
"type": "function",
|
||||
"function": {"name": name, "arguments": json.dumps(arguments)},
|
||||
}],
|
||||
},
|
||||
}],
|
||||
},
|
||||
{
|
||||
"choices": [{"delta": {"content": ""}, "finish_reason": "tool_calls"}],
|
||||
"usage": {"prompt_tokens": 3, "completion_tokens": 3},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def latest_tool_name(messages: list[object]) -> str:
|
||||
"""Find the assistant tool call paired with the latest tool result."""
|
||||
for message in reversed(messages[:-1]):
|
||||
if not isinstance(message, dict):
|
||||
continue
|
||||
calls = message.get("tool_calls")
|
||||
if not isinstance(calls, list):
|
||||
continue
|
||||
for call in reversed(calls):
|
||||
if not isinstance(call, dict):
|
||||
continue
|
||||
function = call.get("function")
|
||||
if isinstance(function, dict) and isinstance(function.get("name"), str):
|
||||
return function["name"]
|
||||
raise AssertionError(f"tool result has no preceding assistant tool call: {messages}")
|
||||
|
||||
|
||||
def message_text(content: object) -> str:
|
||||
"""Read OpenAI text content in either string or block-list form."""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
return "".join(
|
||||
block.get("text", "")
|
||||
for block in content
|
||||
if isinstance(block, dict) and isinstance(block.get("text"), str)
|
||||
)
|
||||
return ""
|
||||
|
||||
|
||||
def assert_advertised_tool(body: dict[str, object], expected: str) -> None:
|
||||
"""Require the packaged deployment to expose the requested tool."""
|
||||
tools = body.get("tools")
|
||||
if not isinstance(tools, list):
|
||||
raise AssertionError(f"model request advertised no tools: {body}")
|
||||
names: set[str] = set()
|
||||
for tool in tools:
|
||||
if not isinstance(tool, dict):
|
||||
continue
|
||||
function = tool.get("function")
|
||||
if isinstance(function, dict) and isinstance(function.get("name"), str):
|
||||
names.add(function["name"])
|
||||
if expected not in names:
|
||||
raise AssertionError(f"model request did not advertise {expected}: {names}")
|
||||
|
||||
|
||||
class MockModel:
|
||||
def __enter__(self) -> "MockModel":
|
||||
MockModelHandler.requests.clear()
|
||||
@@ -116,7 +250,7 @@ def smoke_sdk_default(base_url: str) -> None:
|
||||
result = harness.run("reply with the smoke text", session_id="default-smoke")
|
||||
assert result.status == "ok", result
|
||||
assert result.final_response == EXPECTED_TEXT, result.final_response
|
||||
assert_session_log(sessions, root)
|
||||
assert_session_log(sessions, root, EXPECTED_TEXT)
|
||||
|
||||
|
||||
def smoke_sdk_custom(base_url: str, executable: Path) -> None:
|
||||
@@ -137,10 +271,16 @@ def smoke_sdk_custom(base_url: str, executable: Path) -> None:
|
||||
base_url=base_url,
|
||||
request_timeout_seconds=60,
|
||||
) as harness:
|
||||
result = harness.run("reply with the smoke text", session_id="custom-smoke")
|
||||
assert result.status == "ok", result
|
||||
assert result.final_response == EXPECTED_TEXT, result.final_response
|
||||
assert_session_log(sessions, root)
|
||||
text_result = harness.run("reply with the smoke text", session_id="custom-smoke")
|
||||
code_result = harness.run(CODE_PROMPT, session_id="custom-smoke")
|
||||
workflow_result = harness.run(WORKFLOW_PROMPT, session_id="custom-smoke")
|
||||
assert text_result.status == "ok", text_result
|
||||
assert text_result.final_response == EXPECTED_TEXT, text_result.final_response
|
||||
assert code_result.status == "ok", code_result
|
||||
assert code_result.final_response == CODE_WORKER_TEXT, code_result.final_response
|
||||
assert workflow_result.status == "ok", workflow_result
|
||||
assert workflow_result.final_response == WORKFLOW_WORKER_TEXT, workflow_result.final_response
|
||||
assert_session_log(sessions, root, EXPECTED_TEXT, CODE_WORKER_TEXT, WORKFLOW_WORKER_TEXT)
|
||||
|
||||
|
||||
def smoke_direct(base_url: str, executable: Path) -> None:
|
||||
@@ -177,7 +317,7 @@ def smoke_direct(base_url: str, executable: Path) -> None:
|
||||
peer.read_until(lambda message: message.get("id") == "shutdown")
|
||||
finally:
|
||||
peer.close()
|
||||
assert_session_log(sessions, root)
|
||||
assert_session_log(sessions, root, EXPECTED_TEXT)
|
||||
|
||||
|
||||
class RuntimePeer:
|
||||
@@ -245,7 +385,7 @@ class RuntimePeer:
|
||||
self.stderr.extend(self.process.stderr)
|
||||
|
||||
|
||||
def assert_session_log(sessions: Path, cwd: Path) -> None:
|
||||
def assert_session_log(sessions: Path, cwd: Path, *expected_texts: str) -> None:
|
||||
logs = list(sessions.rglob("*.jsonl"))
|
||||
if len(logs) != 1:
|
||||
raise AssertionError(f"expected one JSONL session log under {sessions}, found {logs}")
|
||||
@@ -253,8 +393,10 @@ def assert_session_log(sessions: Path, cwd: Path) -> None:
|
||||
header = json.loads(lines[0])
|
||||
if header.get("cwd") != str(cwd):
|
||||
raise AssertionError(f"session header cwd is not absolute/canonical: {header}")
|
||||
if EXPECTED_TEXT not in "\n".join(lines):
|
||||
raise AssertionError(f"session log has no final response: {logs[0]}")
|
||||
rendered = "\n".join(lines)
|
||||
for expected in expected_texts:
|
||||
if expected not in rendered:
|
||||
raise AssertionError(f"session log has no {expected!r} response: {logs[0]}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user