diff --git a/docs/development.i18n.yaml b/docs/development.i18n.yaml index c1e557170d..e2dd772eb0 100644 --- a/docs/development.i18n.yaml +++ b/docs/development.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -development.md: bd6f6b561480419abea7a42a44b4078e2c59b1cb -development.zh.md: 54bf19765d2b4dc419e6b71684dbcfcd28230541 +development.md: ea5f2e5d08acbaf1dfce4661530218dbf1a051b9 +development.zh.md: 50877796f2ff47ad46cc67b35f3cd7b5704c8315 diff --git a/docs/development.md b/docs/development.md index bd6f6b5614..ea5f2e5d08 100644 --- a/docs/development.md +++ b/docs/development.md @@ -67,7 +67,7 @@ These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests withou ## CI gates -The keyless GitHub workflow has eight jobs: five Node 24 lanes run static gates, lint, coverage, snapshot replay, and artifact gates separately, and three compatibility jobs run `pnpm run check:node-compat` on Node 22.19, 24, and 26. The lane schedulers fan out independent gates from `package.json`: constraints, typecheck, lint, coverage, snapshot replay, `doc-sync` members, module-graph freshness, `knip`, and the echo-agent smoke test. +The keyless GitHub workflow has eight jobs: five Node 24 lanes run static gates, lint, coverage, snapshot replay, and artifact gates separately, and three compatibility jobs run `pnpm run check:node-compat` on Node 22.19, 24, and 26. The compatibility command runs the TypeScript typecheck and a keyless workflow-workerthread source-launch smoke on every runtime, so the matrix proves that the source graph typechecks and that a real unbuilt Worker loader path executes; the other lane schedulers fan out independent gates from `package.json`: constraints, lint, coverage, snapshot replay, `doc-sync` members, module-graph freshness, `knip`, and the echo-agent smoke test. `pnpm run build` feeds the artifact lane, and `publint`, `verify-node-next-types`, and built-bin smoke tests wait for build output. The separate real-API workflow runs `pnpm run test:e2e` with a secret and `DSH_E2E_MAX_WORKERS=14`. diff --git a/docs/development.zh.md b/docs/development.zh.md index 54bf19765d..50877796f2 100644 --- a/docs/development.zh.md +++ b/docs/development.zh.md @@ -67,7 +67,7 @@ vendor manifest 守卫检查 `vendor/*/src` 下的改动是否连同对应的 `v ## CI 门禁 -keyless GitHub 工作流有八个 job:五个 Node 24 lane 分别运行 static gates、lint、coverage、snapshot replay 和 artifact gates,三个兼容性 job 在 Node 22.19、24 和 26 上运行 `pnpm run check:node-compat`。各 lane 调度器并发运行来自 `package.json` 的独立门禁:constraints、typecheck、lint、coverage、snapshot replay、`doc-sync` 成员、module graph 新鲜度、`knip` 和 echo-agent 冒烟测试。 +keyless GitHub 工作流有八个 job:五个 Node 24 lane 分别运行 static gates、lint、coverage、snapshot replay 和 artifact gates,三个兼容性 job 在 Node 22.19、24 和 26 上运行 `pnpm run check:node-compat`。兼容性命令会在每个运行时上运行 TypeScript 类型检查和 keyless 的 workflow-workerthread 源码启动冒烟测试,因此该矩阵既证明源码图能通过类型检查,也会实际执行一条未构建的 Worker loader 路径;其他 lane 调度器并发运行来自 `package.json` 的独立门禁:constraints、lint、coverage、snapshot replay、`doc-sync` 成员、module graph 新鲜度、`knip` 和 echo-agent 冒烟测试。 `pnpm run build` 供给 artifact lane,`publint`、`verify-node-next-types` 和 built-bin 冒烟测试等待 build 输出。单独的真实 API 工作流带密钥运行 `pnpm run test:e2e`,并设置 `DSH_E2E_MAX_WORKERS=14`。 diff --git a/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md b/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md index 72c9f09eda..51328a5a1a 100644 --- a/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md +++ b/docs/rfc/implemented/process/2026-07-06-node-engine-floor.md @@ -8,7 +8,7 @@ The Node 22 branch of the root `engines.node` range is a contract for the instal ## Decision -Set `engines.node` to `^22.19.0 || >=24.0.0` and test the keyless CI compatibility matrix on `['22.19', 24, 26]`. The real-API e2e workflow stays on Node 24 because it exercises API integration rather than the runtime floor. +Set `engines.node` to `^22.19.0 || >=24.0.0` and test the keyless CI compatibility matrix on `['22.19', 24, 26]`. Every matrix leg runs the TypeScript typecheck plus a keyless source-mode worker smoke, so the floor is exercised through both a complete source typecheck and a real unbuilt runtime path. The real-API e2e workflow stays on Node 24 because it exercises API integration rather than the runtime floor. Two Node features gate the source runtime: @@ -22,7 +22,7 @@ Those source features clear on the 22.x line at **22.18**, but the installed Pi ## Consequences - The advertised LTS branch no longer undercuts the Pi adapter dependency floor. -- CI proves the Node 22 LTS floor directly with Node 22.19, keeps the Node 24 branch on `node: 24`, and keeps Node 26 for the next even line. +- CI proves the Node 22 LTS floor directly with Node 22.19, keeps the Node 24 branch on `node: 24`, and keeps Node 26 for the next even line; each leg typechecks the source graph and launches the unbuilt workflow worker for real. - The built-bin smoke needs no version-conditional flag: at 22.19 type-stripping is already the default, so the test stays the plain `node lib/bin.js` path it documents. - A future dependency or source API that raises the runtime floor must move `engines.node`, the compatibility matrix, and this RFC in the same change. diff --git a/docs/rfc/implemented/process/2026-07-06-parallel-github-ci-gates.md b/docs/rfc/implemented/process/2026-07-06-parallel-github-ci-gates.md index d63a8ad713..061d534b9b 100644 --- a/docs/rfc/implemented/process/2026-07-06-parallel-github-ci-gates.md +++ b/docs/rfc/implemented/process/2026-07-06-parallel-github-ci-gates.md @@ -10,9 +10,9 @@ The hard part is the artifact boundary. `publint`, `verify-node-next-types`, and ## Decision -[CI](../../../../.github/workflows/ci.yml) keeps the keyless workflow to a few broad jobs instead of one job per gate. The Node 24 matrix has five lanes: static gates (`pnpm run check:ci:static`), lint (`pnpm run check:ci:lint`), coverage (`pnpm run check:ci:coverage`), snapshot replay (`pnpm run check:ci:snapshot`), and artifact gates (`pnpm run check:ci:artifacts`). The Node 26 compatibility job installs once and runs `pnpm run check:node-compat`. +[CI](../../../../.github/workflows/ci.yml) keeps the keyless workflow to a few broad jobs instead of one job per gate. The Node 24 matrix has five lanes: static gates (`pnpm run check:ci:static`), lint (`pnpm run check:ci:lint`), coverage (`pnpm run check:ci:coverage`), snapshot replay (`pnpm run check:ci:snapshot`), and artifact gates (`pnpm run check:ci:artifacts`). The compatibility matrix has Node 22.19, 24, and 26 jobs; each installs once and runs `pnpm run check:node-compat`. -Each lane delegates to [scripts/run-gates.ts](../../../../scripts/run-gates.ts), an in-process scheduler with bounded concurrency (`DSH_GATE_CONCURRENCY`). The static lane fans out constraints, the echo-agent demo smoke, `doc-sync` leaf gates, module-graph freshness, and `knip`; the lint lane runs ESLint with its own Node heap cap and a content-strategy ESLint cache; the coverage lane runs Vitest coverage with bounded file workers (`DSH_COVERAGE_MAX_WORKERS`); the snapshot lane isolates replay; the artifact lane builds once and then fans out the artifact consumers; the Node 26 compatibility job owns the TypeScript typecheck. The scheduler buffers each gate's output and prints a named result block with duration, so independent failures stay attributable inside each broad job log. +Each lane delegates to [scripts/run-gates.ts](../../../../scripts/run-gates.ts), an in-process scheduler with bounded concurrency (`DSH_GATE_CONCURRENCY`). The static lane fans out constraints, the echo-agent demo smoke, `doc-sync` leaf gates, module-graph freshness, and `knip`; the lint lane runs ESLint with its own Node heap cap and a content-strategy ESLint cache; the coverage lane runs Vitest coverage with bounded file workers (`DSH_COVERAGE_MAX_WORKERS`); the snapshot lane isolates replay; the artifact lane builds once and then fans out the artifact consumers. Every compatibility job runs the TypeScript typecheck and a keyless workflow-workerthread source-launch smoke, which starts a real unbuilt worker and therefore catches Node-version-specific loader/runtime failures that typechecking cannot. The scheduler buffers each gate's output and prints a named result block with duration, so independent failures stay attributable inside each broad job log. Generated `.sessions/` logs and `.doc-typecheck-*` temp directories are ignored by lint. The aggregate local CI mode still runs demo smoke after lint, while the split GitHub static lane can run demo smoke directly because lint is isolated in its own lane. @@ -36,4 +36,4 @@ The broad-lane split repeats checkout, setup, and install more often than a sing The split introduces a maintenance obligation: when `package.json` adds or removes a gate that belongs in CI, [scripts/run-gates.ts](../../../../scripts/run-gates.ts) needs the matching leaf. That obligation is intentional because the runner is the parallel execution plan for the same gate vocabulary, not a separate quality policy. -The Node 26 signal is narrower than the primary Node 24 signal. It proves the source graph on the newer runtime without doubling documentation, coverage, publication, snapshot, and smoke checks whose failures are not expected to vary by Node minor version. +The compatibility signal is narrower than the primary Node 24 signal. It proves that the source graph typechecks and that the real unbuilt workflow-worker launch path executes on every advertised runtime line without doubling documentation, coverage, publication, snapshot replay, and unrelated smoke checks whose failures are not expected to vary by Node version. diff --git a/packages/workflow/workflow-workerthread/README.md b/packages/workflow/workflow-workerthread/README.md index 2ea1a78c39..0aba0eb680 100644 --- a/packages/workflow/workflow-workerthread/README.md +++ b/packages/workflow/workflow-workerthread/README.md @@ -21,7 +21,7 @@ What the seam guarantees regardless, because benign scripts hit these constantly ## How a run executes -`start()` shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (`new vm.Script`, discarded), preserving the seam's synchronous `META_INVALID`/`SCRIPT_PARSE` throws; one redundant parse per run is the deliberate price. It then spawns the worker (`src/worker.ts` unbuilt via an explicit tsx `execArgv`; the sibling `lib/worker.js` bundle when built) with the meta, body, `args`, and worker-side limits as `workerData`. +`start()` shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (`new vm.Script`, discarded), preserving the seam's synchronous `META_INVALID`/`SCRIPT_PARSE` throws; one redundant parse per run is the deliberate price. It then spawns the worker (unbuilt: a JavaScript data-URL bootstrap registers tsx's ESM and CommonJS transforms inside the worker before importing `src/worker.ts`, giving the whole mixed-module source graph full TypeScript and tsconfig-path transformation on every supported Node line; built: the sibling `lib/worker.js` bundle) with the meta, body, `args`, and worker-side limits as `workerData`. Inside the worker, `runWorkerSession` builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a **child port**. `agent()` sends `child-start`, and the host starts the child on `ctx.subagents` with parent attribution, the shared per-run abort signal, and `outputSchema`/`model` pass-through. diff --git a/packages/workflow/workflow-workerthread/src/host.ts b/packages/workflow/workflow-workerthread/src/host.ts index e1c1a6542f..56c0e455f8 100644 --- a/packages/workflow/workflow-workerthread/src/host.ts +++ b/packages/workflow/workflow-workerthread/src/host.ts @@ -41,7 +41,6 @@ * @module @deepseek-ai/dsh-workflow-workerthread/host */ -import { fileURLToPath } from 'node:url' import { Worker } from 'node:worker_threads' import type { WorkerOptions } from 'node:worker_threads' import type { Context } from 'cordis' @@ -59,13 +58,16 @@ import type { ChildResult, ChildStartRequest, WorkerInit } from './types.ts' /** * Resolve the worker entry and spawn options for the current runtime shape. * Unbuilt (tsx demos, vitest — `import.meta.url` points into `src/`), the - * entry is the TypeScript sibling and the worker needs the tsx loader - * registered explicitly: a worker thread inherits no transform pipeline from - * vitest (vite transforms in-process, not via a node loader), and passing - * execArgv explicitly also shields the worker from any loader flags the - * parent was started with. Built (`lib/index.js`), the entry is the sibling - * bundle the package tsdown config emits and no loader is needed (execArgv - * pinned empty — hermetic, like the environment). + * entry is a JavaScript data-URL bootstrap. That bootstrap runs INSIDE the + * user worker, registers tsx's ESM AND CommonJS transforms there, and only + * then imports the TypeScript sibling. The whole mixed-module source graph + * therefore receives TypeScript transformation and the tsconfig paths map in + * the worker's own module-loader realm. A worker inherits no + * transform pipeline from vitest (vite transforms in-process), and a parent + * `--import tsx` registration is not a contract that user workers share on + * every supported Node line. Built (`lib/index.js`), the entry is the sibling + * bundle the package tsdown config emits and no loader is needed (`execArgv` + * pinned empty in both shapes — hermetic, like the environment). * * Both shapes spawn with an EMPTY environment (`env: {}`): the documented vm * escape reaches `process`, and the harness's ambient credentials @@ -85,19 +87,31 @@ function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOpti if (!import.meta.url.endsWith('.ts')) { return { entry: new URL('./worker.js', import.meta.url), options: { workerData: init, env: {}, execArgv: [] } } } - // Lazy tsx resolution: only the unbuilt shape needs it, so the built - // bundle never requires tsx to be installed. TSX_TSCONFIG_PATH is the one - // variable forwarded through the scrub: tsx finds a tsconfig by searching - // UP from the worker's cwd, and a parent running with its cwd outside the - // repo (the ACP snapshot harness pins the tsconfig through this exact - // variable) would otherwise lose the dsh-* paths map and resolve workspace - // imports to unbuilt lib/ bundles. Loader plumbing, not a secret. + // Resolve tsx lazily: only the unbuilt shape executes this arm, so a built + // consumer never needs the dev-only loader installed. A JavaScript entry is + // essential — it can install tsx's ESM and CommonJS hooks from INSIDE the + // user worker before any TypeScript enters Node's native strip-only parser. + // Both hooks are load-bearing because the source graph crosses both module + // shapes on supported Node lines. TSX_TSCONFIG_PATH is + // the one variable forwarded through the scrub: a parent running outside + // the repo cwd (the ACP snapshot harness is the real case) pins the paths + // map through it. Loader plumbing, not a secret. + const workerEntry = new URL('./worker.ts', import.meta.url) + const tsxEsmApiEntry = import.meta.resolve('tsx/esm/api') + const tsxCjsApiEntry = import.meta.resolve('tsx/cjs/api') + const bootstrap = [ + `import { register as registerEsm } from ${JSON.stringify(tsxEsmApiEntry)}`, + `import { register as registerCjs } from ${JSON.stringify(tsxCjsApiEntry)}`, + 'registerCjs()', + 'registerEsm()', + `await import(${JSON.stringify(workerEntry.href)})`, + ].join('\n') return { - entry: new URL('./worker.ts', import.meta.url), + entry: new URL(`data:text/javascript,${encodeURIComponent(bootstrap)}`), options: { workerData: init, env: process.env.TSX_TSCONFIG_PATH === undefined ? {} : { TSX_TSCONFIG_PATH: process.env.TSX_TSCONFIG_PATH }, - execArgv: ['--import', fileURLToPath(import.meta.resolve('tsx'))], + execArgv: [], }, } } diff --git a/packages/workflow/workflow-workerthread/tests/source-worker.compat.spec.ts b/packages/workflow/workflow-workerthread/tests/source-worker.compat.spec.ts new file mode 100644 index 0000000000..7f34396d52 --- /dev/null +++ b/packages/workflow/workflow-workerthread/tests/source-worker.compat.spec.ts @@ -0,0 +1,38 @@ +/** + * Keyless runtime smoke for the source-mode workflow worker. The Node + * compatibility matrix runs this WHOLE file, so renaming or removing its test + * cannot turn the runtime proof into a successful zero-match filter. + */ + +import { expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import { AgentId } from '@deepseek-ai/dsh-agent' +import type { Agent } from '@deepseek-ai/dsh-agent' +import SubagentService from '@deepseek-ai/dsh-subagent' +import WorkerWorkflowEngine from '../src/index.ts' + +// A fresh thread compiles the source runtime. Leave contention headroom on +// shared CI runners without weakening any engine-level timeout assertion. +vi.setConfig({ testTimeout: 30_000 }) + +it('runs the default config through the source worker', async () => { + const ctx = new Context() + const subagents = await ctx.plugin(SubagentService) + const engine = await ctx.plugin(WorkerWorkflowEngine, {}) + const parent = { id: AgentId('workflow-compat-parent'), options: {} } as unknown as Agent + try { + const run = ctx.workflows.start({ + script: 'return 6 * 7', + meta: { name: 'source-worker-compat', description: 'exercise the unbuilt worker entry' }, + parent, + }) + try { + await expect(run.result).resolves.toMatchObject({ value: 42, stopReason: 'completed', agentsStarted: 0 }) + } finally { + await run.dispose() + } + } finally { + await engine.dispose() + await subagents.dispose() + } +}) diff --git a/packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts b/packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts index d5b16a8e2f..9ccbb17a4e 100644 --- a/packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts +++ b/packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts @@ -1226,15 +1226,11 @@ describe('dsh-workflow-workerthread', () => { await second.dispose() }) - it('unregisters ctx.workflows when the engine fiber is disposed (HMR safety), and default config runs (auto concurrency)', async () => { + it('unregisters ctx.workflows when the engine fiber is disposed (HMR safety)', async () => { const ctx = new Context() await ctx.plugin(SubagentService) const fiber = await ctx.plugin(WorkerWorkflowEngine, {}) expect(ctx.get('workflows')).toBeDefined() - // A zero-agent run through the DEFAULT config exercises the auto - // concurrency resolution (cores - 2, capped) in start(). - const result = await run(ctx, fakeParent(), scripted('return 6 * 7')) - expect(result.value).toBe(42) await fiber.dispose() expect(ctx.get('workflows')).toBeUndefined() }) diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 2f98c74eea..b26e13a56c 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -143,6 +143,11 @@ function gatesForMode(selected: Mode): Gate[] { case 'node-compat': return [ pnpmScript('typecheck', 'typecheck'), + pnpmExec('source-worker-smoke', [ + 'vitest', + 'run', + 'packages/workflow/workflow-workerthread/tests/source-worker.compat.spec.ts', + ], { label: 'source worker smoke' }), ] case 'pre-push': return [