diff --git a/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml b/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml index f53d8a5d3b..9440824276 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml +++ b/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-10-single-file-executable-sdk-runtime-distribution.md: 08fd6f530d1f564ca169a168c0f7d77bff6bf29f -2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 050e4f9a6e8bb03e59835f2c06dd3c701aa34eed +2026-07-10-single-file-executable-sdk-runtime-distribution.md: 511f5f2cf8b43aac5960dfe4630f489ec3dc5086 +2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 6774fc7ec2d9439154305ba0ae35dceeff922a4a diff --git a/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md b/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md index 08fd6f530d..511f5f2cf8 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md +++ b/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md @@ -58,11 +58,11 @@ The exe's "must be explicitly configured" hard semantic is unchanged; the zero-c ## Disposition of worker-style plugins -`dsh-workflow-workerthread` and `dsh-code-runtime-worker` depend on on-disk sibling files via `new Worker(new URL('./worker.js', import.meta.url))`. PoC measurement: pkg's Worker patch intercepts **string paths only**; the URL-object form finds no file inside the VFS. The fix is already known (`fileURLToPath()` to a string), but this round's review decision is **do not verify, do not promise, do not handle**: they compile into the exe with the full set, and behavior when an external `cordis.yml` references them is undefined. +`dsh-workflow-workerthread` and `dsh-code-runtime-worker` are supported inside the exe. Their built entries convert the sibling worker URL with `fileURLToPath()` and pass the resulting filesystem string to `Worker`, which is the form pkg's Worker hook resolves inside the VFS. The workflow engine keeps its data-URL bootstrap for unbuilt source execution; only its built sibling entry uses the filesystem string. The custom-config executable smoke loads both backends, invokes a real `run_code` call and a zero-agent `workflow` call, and requires each worker to return `42` from inside pkg's VFS. ## Testing -The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, and the direct binary protocol, with final text and JSONL checked; its platform wheel is then installed in a clean venv and run without `runtime_bin`. The JSON-RPC protocol is not part of the ACP snapshot system, so there is no snapshot tier (an explicitly named gap, not an oversight). +The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, and the direct binary protocol, with final text and JSONL checked. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. Its platform wheel is then installed in a clean venv and run without `runtime_bin`. The JSON-RPC protocol is not part of the ACP snapshot system, so there is no snapshot tier (an explicitly named gap, not an oversight). Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disposes immediately, so a short-lived pipe aborts an in-flight turn — pipe-driven runs must keep stdin open until the turn ends. @@ -82,4 +82,4 @@ Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disp **Bought**: zero-dependency single-file distribution on target platforms; plugin semantics strictly identical to running from source (the same real package tree, no transpilation, no registry); the serving surface, the plugin set, and the configuration all converge on two sources of truth — `cordis.yml` plus one dependency manifest; the exe and node carriers share one tree and one semantics, so development verification never waits for packaging; official Node binaries remove the patched-binary supply-chain concern. -**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (the build script pins `@yao-pkg/pkg@6.21.0`; upgrading is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial); worker-style plugins have undefined behavior inside the exe. +**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (the build script pins `@yao-pkg/pkg@6.21.0`; upgrading is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial). diff --git a/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md b/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md index 050e4f9a6e..6774fc7ec2 100644 --- a/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md +++ b/docs/rfc/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md @@ -56,13 +56,13 @@ exe「必须显式配置」的硬语义不变;零配置体验由 wrapper 恢 `@deepseek-ai/dsh-jsonrpc-agent`(包)→ `dsh-jsonrpc-agent`(bin)→ `dsh-jsonrpc-agent-pkg`(闭包清单;无 scope 前缀,刻意避开 constraints 对 `@deepseek-ai/dsh-*` 的包形状规则)→ `dsh-jsonrpc-agent-pkg--`(exe 产物)。wire `serverInfo.name` 保持 `deepseek-harness-sdk-runtime`(协议稳定值);Python dist 名为 `deepseek-harness` / `deepseek-harness-runtime-bin`。 -## worker 类插件的处置 +## 工作线程插件 -`dsh-workflow-workerthread` 与 `dsh-code-runtime-worker` 以 `new Worker(new URL('./worker.js', import.meta.url))` 依赖磁盘兄弟文件。PoC 实测:pkg 的 Worker 补丁只拦截**字符串路径**,URL 对象形态在 VFS 内找不到文件——修法已明确(`fileURLToPath()` 转字符串),但本期评审决策是**不验证、不承诺、不处理**:它们随全量集合编译进 exe,外部 `cordis.yml` 引用时行为未定义。 +exe 内支持 `dsh-workflow-workerthread` 与 `dsh-code-runtime-worker`。两个后端的构建入口都通过 `fileURLToPath()` 转换相邻 worker 的 URL,再将所得文件系统字符串传给 `Worker`;pkg 的 Worker 钩子可以用这种形式解析 VFS 内文件。工作流引擎在未构建的源码执行中仍保留 data URL 引导程序,只有构建后的相邻入口使用文件系统字符串。自定义配置的可执行文件冒烟测试会加载两个后端,实际调用 `run_code` 与不启动 agent 的 `workflow`,并要求两个 worker 都从 pkg 的 VFS 内返回 `42`。 ## 测试 -验证面分三层。机制层:`--sea` 链路的实测结论内嵌在「决策」各节(VFS 内 ESM 动态 import、cordis 单实例、fail-loud 配置链路、`node:sqlite`、macOS ad-hoc 签名可运行)。SDK 层:完整的 keyless pytest 套件以假运行时对端覆盖客户端协议、子进程清理、绝对 cwd 传递、双载体启动与载体解析;根 CI 在 Python 3.10 上运行全部用例。端到端层:每个平台构建都通过默认 SDK 路径、自定义配置和直接二进制协议对着 mock 端点完成一个轮次,并校验最终文本与 JSONL;随后把平台 wheel 安装进干净 venv,在不传 `runtime_bin` 的情况下运行。JSON-RPC 协议不在 ACP snapshot 体系内,无 snapshot 层(点名后的明确空缺,非遗漏)。 +验证面分三层。机制层:`--sea` 链路的实测结论内嵌在「决策」各节(VFS 内 ESM 动态 import、cordis 单实例、fail-loud 配置链路、`node:sqlite`、macOS ad-hoc 签名可运行)。SDK 层:完整的 keyless pytest 套件以假运行时对端覆盖客户端协议、子进程清理、绝对 cwd 传递、双载体启动与载体解析;根 CI 在 Python 3.10 上运行全部用例。端到端层:每个平台构建都通过默认 SDK 路径、自定义配置和直接二进制协议对着 mock 端点完成一个轮次,并校验最终文本与 JSONL。自定义配置还会通过打包进 VFS 的真实 worker 文件执行 `run_code` 和不启动 agent 的 `workflow`。随后把平台 wheel 安装进干净 venv,在不传 `runtime_bin` 的情况下运行。JSON-RPC 协议不在 ACP snapshot 体系内,无 snapshot 层(点名后的明确空缺,非遗漏)。 手工驱动注意:bin 视 stdin EOF 为「客户端已走」并立即 dispose,短命管道会中止在飞回合——管道驱动必须保持 stdin 打开到回合结束。 @@ -82,4 +82,4 @@ exe「必须显式配置」的硬语义不变;零配置体验由 wrapper 恢 **买到的**:目标平台零依赖单文件分发;插件语义与源码运行严格一致(同一棵真实包树,无转译无注册表);serving 面、插件集、配置三者全部收敛到 `cordis.yml` + 一份依赖清单两个事实源;exe 与 node 双载体同树同语义,开发验证不必等打包;官方 Node 二进制消除了补丁二进制供应链顾虑。 -**付出的**:产物 174MB 级且源码原样进 blob(无字节码混淆,闭源分发诉求需另行评估);pkg 的 VFS/模块钩子层仍是社区维护(构建脚本钉死 `@yao-pkg/pkg@6.21.0`,升级走显式改动);`--sea` 单 target 单次调用(与 CI 每平台一腿匹配,本地多平台构建串行);worker 类插件在 exe 内行为未定义。 +**付出的**:产物 174MB 级且源码原样进 blob(无字节码混淆,闭源分发诉求需另行评估);pkg 的 VFS/模块钩子层仍是社区维护(构建脚本钉死 `@yao-pkg/pkg@6.21.0`,升级走显式改动);`--sea` 每个 target 调用一次(与 CI 每平台一腿匹配,本地多平台构建串行)。 diff --git a/docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md b/docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md index 9fef336d9c..11e52eb8de 100644 --- a/docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md +++ b/docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md @@ -26,7 +26,7 @@ One deliberate strictness DIVERGENCE from CC: hook misuse — unknown or deferre **Why node:worker_threads**: one run uses one unpooled worker because a workflow run is already heavyweight relative to thread startup. The script runs in a vm context inside the worker, keeping the script-visible surface to the hook contract instead of exposing a bare worker realm, while `agent()` bridges by message-port RPC to I/O-bound child loops on the host. This keeps `start()` from blocking the host on the script's synchronous slice, makes the post-cancel deadline end in a real `worker.terminate()`, and gives cross-thread values a serialization boundary by construction. isolated-vm was rejected for its maintenance state, required `--no-node-snapshot` consumer flag on Node ≥ 20, and node-gyp fallback. -Host-side meta validation and body pre-parsing preserve the seam's synchronous errors, and private enum-keyed payload maps define the wire protocol. Pending async starts, published child records, one host cancellation signal, worker-death reaping, result precedence, and disposal quiescence preserve the subagent run contract across that wire; the [agent-scope runtime-design RFC](../architecture/2026-07-12-agent-scope-runtime-design.md#workflow-children-are-pending-starts-or-published-records) owns those race algorithms. Coverage uses an in-process `MessageChannel` for worker-side logic that main-process V8 coverage cannot see and separately proves the built `lib/worker.js`—a second tsdown entry sanctioned by the `"./worker"` subpath export—under plain Node in the built-bin smoke gate. +Host-side meta validation and body pre-parsing preserve the seam's synchronous errors, and private enum-keyed payload maps define the wire protocol. Pending async starts, published child records, one host cancellation signal, worker-death reaping, result precedence, and disposal quiescence preserve the subagent run contract across that wire; the [agent-scope runtime-design RFC](../architecture/2026-07-12-agent-scope-runtime-design.md#workflow-children-are-pending-starts-or-published-records) owns those race algorithms. Coverage uses an in-process `MessageChannel` for worker-side logic that main-process V8 coverage cannot see and separately proves the built `lib/worker.cjs`—a second tsdown entry sanctioned by the `"./worker"` subpath export—under plain Node in the built-bin smoke gate. **Meta as data, never evaluated**: the meta block reaches the seam as a plain JSON request field (the tool's schema-validated `meta` parameter) and the engine only shape-validates it, every violation named. This is a host-isolation invariant, not a convenience: evaluating a meta literal host-side — even one contractually "pure", in an empty timed vm context — hands script-controlled getters a host stack with no timeout the moment the result is READ, defeating the exact spin isolation the worker thread buys. diff --git a/docs/testing.md b/docs/testing.md index 62989a1096..23cca651fe 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -25,7 +25,7 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword - A plugin shipped via `cordis.yml` needs at least one test through the REAL Loader path: hand-built `ctx.plugin({...})` mounts bypass `unwrapExports` and cannot catch a broken export shape ([postmortem 0001](postmortem/0001-acp-default-export-drops-inject.md); export-shape rules in [packages/AGENTS.md](../packages/AGENTS.md)). - A guard only guards if the regression actually fails it. For a plugin without `inject` (bundle/composition plugins), a Loader smoke stays green under a broken export shape — add an explicit `expect('default' in mod).toBe(false)` plus an `unwrapExports` round-trip assertion, and prove it: introduce the regression, watch red, revert. -- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). The same applies to any non-index runtime entry the built package resolves at run time (the worker-thread runtime's sibling `lib/worker.js`). Keep the built-artifact smokes green (`packages/ui/*/tests/built-bin.e2e.ts`, `packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts`), and assert a genuinely-missing config exits non-zero. +- "Real entry path" means the published artifact: the package `bin` points at built `lib/bin.js` under plain `node`, which tsx masks (settle races, module resolution, a swallowed load failure exiting 0). The same applies to any non-index runtime entry the built package resolves at run time (the worker-thread runtime's sibling `lib/worker.cjs`). Keep the built-artifact smokes green (`packages/ui/*/tests/built-bin.e2e.ts`, `packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts`), and assert a genuinely-missing config exits non-zero. - An e2e that spawns an example from a temp cwd sets `TSX_TSCONFIG_PATH` to the repo-root tsconfig, or it silently falls back to stale built `lib/` ([examples/AGENTS.md](../examples/AGENTS.md)). ## When a snapshot test is required diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index f691904e88..8138f01539 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -29,4 +29,4 @@ Every field is validated (positive numbers) and defaulted; there are no other tu ## The worker entry, unbuilt and built -`worker.ts` is deliberately erasable-only TypeScript with type-only cross-package imports: unbuilt (vitest/tsx), the host spawns `src/worker.ts` directly and Node's native type stripping loads it; built, the entry ships as the sibling bundle `lib/worker.js` (its own tsdown entry). The built path is pinned by `tests/built-lib.e2e.ts`, the real-load-path guard from [docs/testing.md](../../../docs/testing.md). +`worker.ts` is deliberately erasable-only TypeScript with type-only cross-package imports: unbuilt (vitest/tsx), the host spawns `src/worker.ts` directly and Node's native type stripping loads it; built, the entry ships as the sibling CommonJS bundle `lib/worker.cjs` (its own tsdown entry). The CommonJS format is required because pkg's VFS Worker hook compiles filesystem-string entries as CommonJS. The host converts either entry URL to a filesystem string before constructing `Worker`, which works through both ordinary Node resolution and that pkg hook. The built path is pinned by `tests/built-lib.e2e.ts`, the real-load-path guard from [docs/testing.md](../../../docs/testing.md). diff --git a/packages/code-runtime/code-runtime-worker/package.json b/packages/code-runtime/code-runtime-worker/package.json index 027089337b..91075243d2 100644 --- a/packages/code-runtime/code-runtime-worker/package.json +++ b/packages/code-runtime/code-runtime-worker/package.json @@ -13,14 +13,14 @@ }, "./worker": { "types": "./lib/types/worker.d.ts", - "default": "./lib/worker.js" + "default": "./lib/worker.cjs" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ "lib/index.js", - "lib/worker.js", + "lib/worker.cjs", "lib/types/**/*.d.ts", "lib/types/**/*.d.ts.map", "src" diff --git a/packages/code-runtime/code-runtime-worker/src/index.ts b/packages/code-runtime/code-runtime-worker/src/index.ts index f78f06cb0b..7a8024cb3d 100644 --- a/packages/code-runtime/code-runtime-worker/src/index.ts +++ b/packages/code-runtime/code-runtime-worker/src/index.ts @@ -14,6 +14,7 @@ import { Worker } from 'node:worker_threads' import { stripTypeScriptTypes } from 'node:module' +import { fileURLToPath } from 'node:url' import { Context } from 'cordis' import z from 'schemastery' import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime' @@ -98,16 +99,19 @@ interface LiveRun { } /** - * The worker entry module. Source runs unbuilt (`src/worker.ts`, loadable + * The worker entry path. Source runs unbuilt (`src/worker.ts`, loadable * directly on this repo's Node range via native type stripping — the file * is erasable-only with type-only relative imports); the built package - * ships it as a sibling bundle (`lib/worker.js`, its own tsdown entry). + * ships it as a sibling CommonJS bundle (`lib/worker.cjs`, its own tsdown + * entry) because pkg's VFS Worker hook compiles string-path entries as + * CommonJS. * The URL *pathname*'s extension says which world this module is in — * pathname, because dev-time module runners (vitest) may suffix - * `import.meta.url` with a query string; relative resolution drops it. + * `import.meta.url` with a query string; relative resolution drops it. Worker + * receives a filesystem string so pkg's VFS Worker hook can resolve it. */ -/* v8 ignore next -- the './worker.js' arm is the built-lib world, unreachable unbuilt by construction; the built-lib e2e pins it. */ -const WORKER_URL = new URL(new URL(import.meta.url).pathname.endsWith('.ts') ? './worker.ts' : './worker.js', import.meta.url) +/* v8 ignore next -- the './worker.cjs' arm is the built-lib world, unreachable unbuilt by construction; the built-lib e2e pins it. */ +const WORKER_PATH = fileURLToPath(new URL(new URL(import.meta.url).pathname.endsWith('.ts') ? './worker.ts' : './worker.cjs', import.meta.url)) /** Render an unknown thrown value as a message, `Error` or not. */ function messageOf(error: unknown): string { @@ -273,7 +277,7 @@ export class WorkerCodeRuntime extends CodeRuntime { maxLogBytes: this.config.maxLogBytes, maxValueBytes: this.config.maxValueBytes, } - const worker = new Worker(WORKER_URL, { + const worker = new Worker(WORKER_PATH, { workerData: bootData, // Model code gets NO ambient environment — stronger than the scrubbed // env the defensive-patterns rule requires for spawned commands. diff --git a/packages/code-runtime/code-runtime-worker/src/worker.ts b/packages/code-runtime/code-runtime-worker/src/worker.ts index efaafdb038..9c9b009b4e 100644 --- a/packages/code-runtime/code-runtime-worker/src/worker.ts +++ b/packages/code-runtime/code-runtime-worker/src/worker.ts @@ -17,4 +17,4 @@ import type { WorkerBootData } from './protocol.ts' // A worker always has a parent port; guard loudly rather than run detached. if (!parentPort) throw new Error('dsh-code-runtime-worker: worker entry loaded outside a worker thread') -await runWorkerMain(parentPort, workerData as WorkerBootData, { stdout: process.stdout, stderr: process.stderr }) +void runWorkerMain(parentPort, workerData as WorkerBootData, { stdout: process.stdout, stderr: process.stderr }) diff --git a/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts b/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts index 66ce1830b6..aac0ece8a1 100644 --- a/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts +++ b/packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts @@ -8,7 +8,7 @@ import { describe, expect, it } from 'vitest' * BUILT-ARTIFACT smoke for the published package (the real-load-path guard * from docs/testing.md): the unit suite runs `src/` under vitest, where the * worker entry resolves to `src/worker.ts` — a consumer runs `lib/index.js` - * under plain `node`, where it must resolve the sibling `lib/worker.js` + * under plain `node`, where it must resolve the sibling `lib/worker.cjs` * bundle instead. This spawns plain `node` (NOT tsx) from inside the package * directory and imports the package BY NAME, so resolution flows through the * real `exports` map exactly as it would from a downstream install; the @@ -21,11 +21,11 @@ import { describe, expect, it } from 'vitest' */ const pkgDir = fileURLToPath(new URL('..', import.meta.url)) -const built = ['lib/index.js', 'lib/worker.js'].every(file => existsSync(join(pkgDir, file))) +const built = ['lib/index.js', 'lib/worker.cjs'].every(file => existsSync(join(pkgDir, file))) && existsSync(join(pkgDir, '../code-runtime/lib/index.js')) describe.skipIf(!built)('built lib real load path (plain node)', () => { - it('runs a TypeScript program with a binding through lib/index.js and its lib/worker.js entry', async () => { + it('runs a TypeScript program with a binding through lib/index.js and its lib/worker.cjs entry', async () => { const script = ` const { Context } = await import('cordis') const { WorkerCodeRuntime } = await import('@deepseek-ai/dsh-code-runtime-worker') diff --git a/packages/code-runtime/code-runtime-worker/tsdown.config.ts b/packages/code-runtime/code-runtime-worker/tsdown.config.ts index 5af39d936a..3eec074119 100644 --- a/packages/code-runtime/code-runtime-worker/tsdown.config.ts +++ b/packages/code-runtime/code-runtime-worker/tsdown.config.ts @@ -3,8 +3,10 @@ import { defineConfig } from 'tsdown' /** * Package-shape override (see the root tsdown.config.ts): besides the * default lib/index.js bundle, the worker BOOTSTRAP ships as its own - * sibling entry — `new Worker(new URL('./worker.js', import.meta.url))` - * loads it as a file, so it cannot be part of the index bundle. TWO + * sibling CommonJS entry — `new Worker(fileURLToPath(new URL('./worker.cjs', import.meta.url)))` + * loads it as a file, so it cannot be part of the index bundle. pkg's VFS + * Worker hook compiles string-path entries as CommonJS, so an ESM worker is + * not viable inside the executable. TWO * single-entry builds, not one two-entry build: a multi-entry build emits * the shared bootstrap module as a `lib/bootstrap-*.js` chunk both bundles * import, which the package.json `files` whitelist (deliberately exact) @@ -25,7 +27,7 @@ export default defineConfig([ { entry: ['lib/types/worker.js'], outDir: 'lib', - format: ['esm'], + format: ['cjs'], platform: 'node', target: 'es2024', fixedExtension: false, diff --git a/packages/workflow/workflow-workerthread/README.md b/packages/workflow/workflow-workerthread/README.md index b9786ae1e2..023b562b42 100644 --- a/packages/workflow/workflow-workerthread/README.md +++ b/packages/workflow/workflow-workerthread/README.md @@ -32,7 +32,7 @@ Unknown options, malformed arguments, unsupported schemas, tripped caps, provide ## Run sequence -`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice. +`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. Source mode uses a data-URL bootstrap that installs the TypeScript transforms inside the worker; built mode passes the sibling CommonJS bundle `lib/worker.cjs` as a filesystem string. CommonJS is required because pkg's VFS Worker hook compiles filesystem-string entries in that format; the same entry also works under ordinary Node resolution. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice. For each `agent()` call: diff --git a/packages/workflow/workflow-workerthread/package.json b/packages/workflow/workflow-workerthread/package.json index ed934cd0cc..afaf840f78 100644 --- a/packages/workflow/workflow-workerthread/package.json +++ b/packages/workflow/workflow-workerthread/package.json @@ -13,14 +13,14 @@ }, "./worker": { "types": "./lib/types/worker.d.ts", - "default": "./lib/worker.js" + "default": "./lib/worker.cjs" }, "./src/*": "./src/*", "./package.json": "./package.json" }, "files": [ "lib/index.js", - "lib/worker.js", + "lib/worker.cjs", "lib/types/**/*.d.ts", "lib/types/**/*.d.ts.map", "src" diff --git a/packages/workflow/workflow-workerthread/src/host.ts b/packages/workflow/workflow-workerthread/src/host.ts index 95c07f0d2d..9ce754c0c6 100644 --- a/packages/workflow/workflow-workerthread/src/host.ts +++ b/packages/workflow/workflow-workerthread/src/host.ts @@ -43,6 +43,7 @@ import { Worker } from 'node:worker_threads' import type { WorkerOptions } from 'node:worker_threads' +import { fileURLToPath } from 'node:url' import type { Context } from 'cordis' import type { Agent } from '@deepseek-ai/dsh-agent' import { assertNever } from '@deepseek-ai/dsh-llm' @@ -87,12 +88,12 @@ interface ChildRecord { * AMBIENT channel only — an escapee still holds process-wide privileges * like fs access (the README's trust premise stands). * @param init - the run payload, passed as `workerData`. - * @returns the entry URL and the Worker options to spawn it with. + * @returns the entry path or URL and the Worker options to spawn it with. */ -function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOptions } { +function resolveWorkerSpawn(init: WorkerInit): { entry: string | URL; options: WorkerOptions } { /* v8 ignore next 3 -- the built-output arm: tests always run unbuilt (src/); the built-worker e2e exercises this shape for real */ if (!import.meta.url.endsWith('.ts')) { - return { entry: new URL('./worker.js', import.meta.url), options: { workerData: init, env: {}, execArgv: [] } } + return { entry: fileURLToPath(new URL('./worker.cjs', import.meta.url)), options: { workerData: init, env: {}, execArgv: [] } } } // Resolve tsx lazily: only the unbuilt shape executes this arm, so a built // consumer never needs the dev-only loader installed. A JavaScript entry is diff --git a/packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts b/packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts index 545be831f6..779dd50600 100644 --- a/packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts +++ b/packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts @@ -8,17 +8,17 @@ import { describe, expect, it } from 'vitest' const packageRoot = fileURLToPath(new URL('..', import.meta.url)) const builtIndex = join(packageRoot, 'lib', 'index.js') -const builtWorker = join(packageRoot, 'lib', 'worker.js') +const builtWorker = join(packageRoot, 'lib', 'worker.cjs') const run = promisify(execFile) /** * The BUILT-output guard for the worker entry: every other suite runs * unbuilt (src/ + tsx), so nothing else proves that `lib/index.js` resolves - * its sibling `lib/worker.js` and that the bundle boots a worker under plain + * its sibling `lib/worker.cjs` and that the bundle boots a worker under plain * node (no tsx loader). Keyless — a zero-agent script needs no provider — * and self-skips until `pnpm run build` has produced the bundles. */ -describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.js)', () => { +describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.cjs)', () => { it('the built engine spawns its built worker under plain node and completes a run', async () => { // ESM resolves bare specifiers from the IMPORTING FILE's location, so the // driver must live inside the package for its node_modules to apply — a diff --git a/packages/workflow/workflow-workerthread/tsdown.config.ts b/packages/workflow/workflow-workerthread/tsdown.config.ts index 3102a36c1c..c163fe4369 100644 --- a/packages/workflow/workflow-workerthread/tsdown.config.ts +++ b/packages/workflow/workflow-workerthread/tsdown.config.ts @@ -6,7 +6,9 @@ import { defineConfig } from 'tsdown' * entries are JS emitted by tsc under lib/types and are bundled as two * single-entry passes so shared modules (realm, runtime, session) are inlined * into each instead of split into a hash-named chunk (the worker entry must - * be a self-contained file the Worker constructor can load by path). + * be a self-contained file the Worker constructor can load by path). The + * worker bundle is CommonJS because pkg's VFS Worker hook compiles + * filesystem-string entries as CommonJS. */ export default defineConfig([ { @@ -22,7 +24,7 @@ export default defineConfig([ { entry: ['lib/types/worker.js'], outDir: 'lib', - format: ['esm'], + format: ['cjs'], platform: 'node', target: 'es2024', fixedExtension: false, diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index d7d52aa421..a1cbfccd09 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -110,7 +110,7 @@ const dshBinPackageFiles = [ const dshWorkerPackageFiles = [ 'lib/index.js', - 'lib/worker.js', + 'lib/worker.cjs', 'lib/types/**/*.d.ts', 'lib/types/**/*.d.ts.map', 'src', diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 94cf174468..c8a0e0d359 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -330,7 +330,7 @@ function builtBinSmokeGate(): Gate { 'packages/ui/stdio-agent/tests/built-bin.e2e.ts', 'packages/ui/acp-agent/tests/built-bin.e2e.ts', // The worker-entry packages' built bundles: the only automated proof - // that lib/index.js resolves its sibling lib/worker.js under plain node + // that lib/index.js resolves its sibling lib/worker.cjs under plain node // (the e2e lane runs unbuilt, so these files self-skip there). 'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts', 'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts', diff --git a/scripts/smoke-python-runtime.py b/scripts/smoke-python-runtime.py index c0d95b40d5..47de59dd2e 100644 --- a/scripts/smoke-python-runtime.py +++ b/scripts/smoke-python-runtime.py @@ -17,11 +17,18 @@ from typing import Callable EXPECTED_TEXT = "runtime smoke ok" +CODE_PROMPT = "Use run_code to compute the packaged worker smoke value." +CODE_WORKER_TEXT = "code worker smoke ok" +WORKFLOW_PROMPT = "Use workflow to compute the packaged worker smoke value without agents." +WORKFLOW_WORKER_TEXT = "workflow worker smoke ok" CUSTOM_CORDIS = """\ - id: jsonrpc name: '@deepseek-ai/dsh-jsonrpc' - id: agent-core name: '@deepseek-ai/dsh-agent-core' + config: + tools: + mode: both - id: sessions name: '@deepseek-ai/dsh-session-persistence-jsonl' config: @@ -30,11 +37,21 @@ CUSTOM_CORDIS = """\ name: '@deepseek-ai/dsh-bash-local' config: cwd: !!js process.env.DSH_CWD +- id: code-runtime + name: '@deepseek-ai/dsh-code-runtime-worker' +- id: subagents + name: '@deepseek-ai/dsh-subagent' +- id: workflow-engine + name: '@deepseek-ai/dsh-workflow-workerthread' + config: + provider: spawn +- id: workflow-tool + name: '@deepseek-ai/dsh-tool-workflow' """ class MockModelHandler(BaseHTTPRequestHandler): - """Return one deterministic OpenAI-compatible streaming completion.""" + """Return deterministic text and worker-tool streaming completions.""" requests: list[dict[str, object]] = [] @@ -45,11 +62,7 @@ class MockModelHandler(BaseHTTPRequestHandler): self.send_response(200) self.send_header("content-type", "text/event-stream") self.end_headers() - chunks = [ - {"choices": [{"delta": {"role": "assistant", "content": None, "reasoning_content": ""}}]}, - {"choices": [{"delta": {"content": EXPECTED_TEXT}}]}, - {"choices": [{"delta": {"content": ""}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 3, "completion_tokens": 3}}, - ] + chunks = completion_chunks(body) for chunk in chunks: self.wfile.write(f"data: {json.dumps(chunk)}\n\n".encode()) self.wfile.write(b"data: [DONE]\n\n") @@ -59,6 +72,127 @@ class MockModelHandler(BaseHTTPRequestHandler): return +def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]: + """Choose the next deterministic model response from request history.""" + messages = body.get("messages") + if not isinstance(messages, list) or not messages: + raise AssertionError(f"model request has no messages: {body}") + latest = messages[-1] + if not isinstance(latest, dict): + raise AssertionError(f"model request has an invalid latest message: {body}") + + if latest.get("role") == "tool": + tool_name = latest_tool_name(messages) + tool_text = json.dumps(latest.get("content")) + if "42" not in tool_text: + raise AssertionError(f"{tool_name} worker returned no expected value: {latest}") + if tool_name == "run_code": + return text_chunks(CODE_WORKER_TEXT) + if tool_name == "workflow": + return text_chunks(WORKFLOW_WORKER_TEXT) + raise AssertionError(f"unexpected tool follow-up: {tool_name}") + + prompt = message_text(latest.get("content")) + if prompt == CODE_PROMPT: + assert_advertised_tool(body, "run_code") + return tool_call_chunks("call-code-worker", "run_code", {"code": "return 6 * 7"}) + if prompt == WORKFLOW_PROMPT: + assert_advertised_tool(body, "workflow") + return tool_call_chunks( + "call-workflow-worker", + "workflow", + { + "script": "return 6 * 7", + "meta": { + "name": "pkg-worker-smoke", + "description": "exercise the packaged workflow worker", + }, + }, + ) + return text_chunks(EXPECTED_TEXT) + + +def text_chunks(text: str) -> list[dict[str, object]]: + """Build a complete streaming text response.""" + return [ + {"choices": [{"delta": {"role": "assistant", "content": None, "reasoning_content": ""}}]}, + {"choices": [{"delta": {"content": text}}]}, + { + "choices": [{"delta": {"content": ""}, "finish_reason": "stop"}], + "usage": {"prompt_tokens": 3, "completion_tokens": 3}, + }, + ] + + +def tool_call_chunks(call_id: str, name: str, arguments: dict[str, object]) -> list[dict[str, object]]: + """Build a complete streaming function-call response.""" + return [ + {"choices": [{"delta": {"role": "assistant", "content": None, "reasoning_content": ""}}]}, + { + "choices": [{ + "delta": { + "tool_calls": [{ + "index": 0, + "id": call_id, + "type": "function", + "function": {"name": name, "arguments": json.dumps(arguments)}, + }], + }, + }], + }, + { + "choices": [{"delta": {"content": ""}, "finish_reason": "tool_calls"}], + "usage": {"prompt_tokens": 3, "completion_tokens": 3}, + }, + ] + + +def latest_tool_name(messages: list[object]) -> str: + """Find the assistant tool call paired with the latest tool result.""" + for message in reversed(messages[:-1]): + if not isinstance(message, dict): + continue + calls = message.get("tool_calls") + if not isinstance(calls, list): + continue + for call in reversed(calls): + if not isinstance(call, dict): + continue + function = call.get("function") + if isinstance(function, dict) and isinstance(function.get("name"), str): + return function["name"] + raise AssertionError(f"tool result has no preceding assistant tool call: {messages}") + + +def message_text(content: object) -> str: + """Read OpenAI text content in either string or block-list form.""" + if isinstance(content, str): + return content + if isinstance(content, list): + return "".join( + block.get("text", "") + for block in content + if isinstance(block, dict) and isinstance(block.get("text"), str) + ) + return "" + + +def assert_advertised_tool(body: dict[str, object], expected: str) -> None: + """Require the packaged deployment to expose the requested tool.""" + tools = body.get("tools") + if not isinstance(tools, list): + raise AssertionError(f"model request advertised no tools: {body}") + names: set[str] = set() + for tool in tools: + if not isinstance(tool, dict): + continue + function = tool.get("function") + if isinstance(function, dict) and isinstance(function.get("name"), str): + names.add(function["name"]) + if expected not in names: + raise AssertionError(f"model request did not advertise {expected}: {names}") + + class MockModel: def __enter__(self) -> "MockModel": MockModelHandler.requests.clear() @@ -116,7 +250,7 @@ def smoke_sdk_default(base_url: str) -> None: result = harness.run("reply with the smoke text", session_id="default-smoke") assert result.status == "ok", result assert result.final_response == EXPECTED_TEXT, result.final_response - assert_session_log(sessions, root) + assert_session_log(sessions, root, EXPECTED_TEXT) def smoke_sdk_custom(base_url: str, executable: Path) -> None: @@ -137,10 +271,16 @@ def smoke_sdk_custom(base_url: str, executable: Path) -> None: base_url=base_url, request_timeout_seconds=60, ) as harness: - result = harness.run("reply with the smoke text", session_id="custom-smoke") - assert result.status == "ok", result - assert result.final_response == EXPECTED_TEXT, result.final_response - assert_session_log(sessions, root) + text_result = harness.run("reply with the smoke text", session_id="custom-smoke") + code_result = harness.run(CODE_PROMPT, session_id="custom-smoke") + workflow_result = harness.run(WORKFLOW_PROMPT, session_id="custom-smoke") + assert text_result.status == "ok", text_result + assert text_result.final_response == EXPECTED_TEXT, text_result.final_response + assert code_result.status == "ok", code_result + assert code_result.final_response == CODE_WORKER_TEXT, code_result.final_response + assert workflow_result.status == "ok", workflow_result + assert workflow_result.final_response == WORKFLOW_WORKER_TEXT, workflow_result.final_response + assert_session_log(sessions, root, EXPECTED_TEXT, CODE_WORKER_TEXT, WORKFLOW_WORKER_TEXT) def smoke_direct(base_url: str, executable: Path) -> None: @@ -177,7 +317,7 @@ def smoke_direct(base_url: str, executable: Path) -> None: peer.read_until(lambda message: message.get("id") == "shutdown") finally: peer.close() - assert_session_log(sessions, root) + assert_session_log(sessions, root, EXPECTED_TEXT) class RuntimePeer: @@ -245,7 +385,7 @@ class RuntimePeer: self.stderr.extend(self.process.stderr) -def assert_session_log(sessions: Path, cwd: Path) -> None: +def assert_session_log(sessions: Path, cwd: Path, *expected_texts: str) -> None: logs = list(sessions.rglob("*.jsonl")) if len(logs) != 1: raise AssertionError(f"expected one JSONL session log under {sessions}, found {logs}") @@ -253,8 +393,10 @@ def assert_session_log(sessions: Path, cwd: Path) -> None: header = json.loads(lines[0]) if header.get("cwd") != str(cwd): raise AssertionError(f"session header cwd is not absolute/canonical: {header}") - if EXPECTED_TEXT not in "\n".join(lines): - raise AssertionError(f"session log has no final response: {logs[0]}") + rendered = "\n".join(lines) + for expected in expected_texts: + if expected not in rendered: + raise AssertionError(f"session log has no {expected!r} response: {logs[0]}") if __name__ == "__main__":