fix(dev-infra): harden gate runner execution

2026-07-28 00:12:16 +08:00
parent 3e6fbccffa
commit 55d6ab5220
5 changed files with 103 additions and 32 deletions
@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-27-replayable-gate-plans.md
-2026-07-27-replayable-gate-plans.md: 8a42ae3c89a3f75bb248f78583f6ef825af3241e
-2026-07-27-replayable-gate-plans.zh.md: 572ef1ce131a0ced8d723e1caa822fe47556fcee
+2026-07-27-replayable-gate-plans.md: fc4f883d74d765d173a6d1419c2e75f44bd1966f
+2026-07-27-replayable-gate-plans.zh.md: d6c6eb9649120c7dfbfb6cd0c935e915c62f40ef
@@ -8,13 +8,13 @@ English | [中文](2026-07-27-replayable-gate-plans.zh.md)

 Repository aggregates need to fail before execution when their dependency graph is invalid. Without validation, an empty aggregate can succeed, duplicate gate IDs can overwrite scheduler state, and missing or cyclic dependencies can appear as generic skips after unrelated work has already run.

-Operators also need the scheduler-owned environment and dependency context for a failed command. The Node 24 consumer job instead owned a separate shell process pool, duplicating commands, concurrency, environment, and failure collection while allowing restored build artifacts to be consumed before any command established that the download was complete.
+Operators also need the scheduler-owned environment and dependency context for a failed command. The Node 24 consumer job instead owned a separate shell process pool, duplicating commands, concurrency, environment, and failure collection while allowing later commands to consume restored artifacts before publint and built-package invariant checks established their public and runtime-closure contracts.

 ## Decision

 [`scripts/run-gates.ts`](../../../../scripts/run-gates.ts) constructs a complete `GatePlan` before execution and validates that it is non-empty, every ID is unique and replay-safe, every dependency exists, and the graph is acyclic. `executeGatePlan()` repeats validation at the process boundary, so an invalid injected plan cannot start a child. The empty `pre-push` mode is absent; Git hooks retain their separate narrow contract.

-Every mode supports deterministic `--list` output and a versioned stable `--list --json` object. Machine consumers invoke `pnpm --silent run <owning-script> -- --list --json`; `--silent` removes pnpm's outer command banner so stdout is exactly one JSON object. Both views expose canonical gate order, IDs, display commands, dependencies, blocking disposition, the plan-owned worker ceiling, and scheduler-owned environment operations. Environment overrides remain declarative until spawn and support only the forms current plans use: setting a value or appending one with a space. Inspection therefore never enumerates or bakes in inherited values; values under secret-like names are redacted.
+Every mode supports deterministic `--list` output and a versioned stable `--list --json` object. Machine consumers invoke `pnpm --silent run <owning-script> -- --list --json`; `--silent` removes pnpm's outer command banner so stdout is exactly one JSON object. Both views expose canonical gate order, IDs, display commands, dependencies, blocking disposition, the plan-owned worker ceiling, and scheduler-owned environment operations. Gate-level spawn overrides remain declarative until spawn and support only the forms current plans use: setting a value or appending one with a space. Inspection serializes those operations without resolving them against inherited values; values under secret-like declared names are redacted.

 `--only <gate-id>` runs the named gate with its complete transitive dependency closure in canonical plan order. Its banner identifies the run as partial diagnostic evidence and names the complete owning package script. Every failed or skipped gate prints the cross-platform replay command `pnpm run <owning-script> -- --only <gate-id>`, which restores dependency and environment semantics through the scheduler.

@@ -24,7 +24,7 @@ The `check:ci:consumers` mode owns the Node 24 consumer job's seven top-level co

 ## Verification

-[`scripts/run-gates.spec.ts`](../../../../scripts/run-gates.spec.ts) proves invalid plans cannot reach the injected executor, dependency closure is complete, list order and JSON fields are stable, the silent package-script entry emits one parseable JSON object, replay text is portable, environment resolution is deferred to spawn, signal termination remains distinct from exit status, and a settled failure is observed before an unrelated gate finishes. Its consumer-plan case pins the seven-command inventory, worker default and ceiling, and restored-build validation dependencies. [`scripts/publint-all.spec.ts`](../../../../scripts/publint-all.spec.ts) proves a missing public export fails the first stage. The CI workflow invokes only `pnpm run check:ci:consumers` for that process pool.
+[`scripts/run-gates.spec.ts`](../../../../scripts/run-gates.spec.ts) proves invalid plans cannot reach the injected executor, dependency closure is complete, list order and JSON fields are stable, direct and symlinked entries emit one parseable JSON object, replay text is portable, environment resolution is deferred to spawn, signal termination remains distinct from exit status, and a settled failure is observed before an unrelated gate finishes. Its consumer-plan case pins the seven-command inventory, worker default and ceiling, and restored-build validation dependencies. [`scripts/publint-all.spec.ts`](../../../../scripts/publint-all.spec.ts) proves a missing public export fails the first stage. The CI workflow invokes only `pnpm run check:ci:consumers` for that process pool.

 ## Alternatives considered

@@ -8,15 +8,15 @@ Status: implemented

 仓库聚合任务的依赖图无效时，必须在开始执行前失败。若不验证，空聚合任务可能成功退出，重复的门禁 ID 可能覆盖调度器状态，缺失或成环的依赖则可能在无关任务已经运行后，以笼统的跳过状态出现。

-故障排查者还需要失败命令的依赖上下文和由调度器掌管的环境设置。Node 24 消费方作业却曾自行管理一套独立的 shell 进程池，造成命令、并发度、环境和失败收集重复维护，并允许在任何命令确认下载产物完整之前消费恢复后的构建产物。
+故障排查者还需要失败命令的依赖上下文和由调度器掌管的环境设置。Node 24 消费方作业却曾自行管理一套独立的 shell 进程池，造成命令、并发度、环境和失败收集重复维护，并允许后续命令在 publint 和已构建包（package）不变式检查确立恢复后产物的公开契约与运行时闭包契约之前，就消费这些产物。

 ## 决策

 [`scripts/run-gates.ts`](../../../../scripts/run-gates.ts) 在执行前构造完整的 `GatePlan`，并验证计划不为空、每个 ID 唯一且可安全用于回放、每项依赖都存在、依赖图无环。`executeGatePlan()` 在进程边界再次执行验证，因此注入的无效计划无法启动子进程。空的 `pre-push` 模式不存在；Git 钩子仍遵循独立的狭窄契约。

-每种模式都支持确定性的 `--list` 输出，以及带版本标识且保持稳定的 `--list --json` 对象。机器消费方使用 `pnpm --silent run <owning-script> -- --list --json`；`--silent` 会去除 pnpm 外层的命令横幅，使 stdout 恰好只包含一个 JSON 对象。两种视图都公开规范的门禁顺序、ID、显示命令、依赖、阻塞属性、计划掌管的工作进程上限，以及由调度器掌管的环境操作。环境覆盖在 spawn 之前保持声明式，并且只支持当前计划使用的两种形式：设置值，或以空格分隔后追加值。检查结果因此不会枚举或固化继承值；名称疑似机密项的值会被脱敏。
+每种模式都支持确定性的 `--list` 输出，以及带版本标识且保持稳定的 `--list --json` 对象。机器消费方使用 `pnpm --silent run <owning-script> -- --list --json`；`--silent` 会去除 pnpm 外层的命令横幅，使 stdout 恰好只包含一个 JSON 对象。两种视图都公开规范的门禁顺序、ID、显示命令、依赖、阻塞属性、计划掌管的工作进程上限，以及由调度器掌管的环境操作。门禁级 spawn 覆盖在 spawn 之前保持声明式，并且只支持当前计划使用的两种形式：设置值，或以空格分隔后追加值。检查会序列化这些操作，而不会结合继承值进行解析；声明的名称若疑似机密，其值会被脱敏。

-`--only <gate-id>` 按规范的计划顺序运行指定门禁及其完整的传递依赖闭包。启动横幅明确标记本次运行只构成局部诊断证据，并给出所属的完整包（package）脚本。每个失败或跳过的门禁都打印跨平台回放命令 `pnpm run <owning-script> -- --only <gate-id>`，该命令通过调度器还原依赖与环境语义。
+`--only <gate-id>` 按规范的计划顺序运行指定门禁及其完整的传递依赖闭包。启动横幅明确标记本次运行只构成局部诊断证据，并给出所属的完整包脚本。每个失败或跳过的门禁都打印跨平台回放命令 `pnpm run <owning-script> -- --only <gate-id>`，该命令通过调度器还原依赖与环境语义。

 调度器会宣告每项门禁开始运行，将子进程的 stdout 和 stderr 缓冲到该门禁结束，再在无关门禁仍继续运行时输出一项归属明确的结果。失败块包含显示命令、经过脱敏且由调度器掌管的环境操作、彼此独立的退出码和信号结果、完整的子进程输出，以及回放命令；成功运行的子进程输出默认仍不显示，只有设置 `DSH_GATE_VERBOSE=1` 时才会输出。子进程输出不会持久化。

@@ -24,7 +24,7 @@ Status: implemented

 ## 验证

-[`scripts/run-gates.spec.ts`](../../../../scripts/run-gates.spec.ts) 证明无效计划无法触达注入的执行器、依赖闭包完整、列表顺序与 JSON 字段稳定、静默的包脚本入口只输出一个可解析的 JSON 对象、回放文本可跨平台使用、环境解析推迟到 spawn 时进行、信号终止与退出状态彼此独立，而且某项门禁失败结束后，无须等待无关门禁完成即可观察到该失败。消费方计划用例固定了 7 条命令的清单、工作进程默认值与上限，以及恢复后构建产物验证的依赖关系。[`scripts/publint-all.spec.ts`](../../../../scripts/publint-all.spec.ts) 证明缺失公开导出时第一阶段会失败。CI 工作流只为该进程池调用 `pnpm run check:ci:consumers`。
+[`scripts/run-gates.spec.ts`](../../../../scripts/run-gates.spec.ts) 证明无效计划无法触达注入的执行器、依赖闭包完整、列表顺序与 JSON 字段稳定、直接入口和符号链接入口只输出一个可解析的 JSON 对象、回放文本可跨平台使用、环境解析推迟到 spawn 时进行、信号终止与退出状态彼此独立，而且某项门禁失败结束后，无须等待无关门禁完成即可观察到该失败。消费方计划用例固定了 7 条命令的清单、工作进程默认值与上限，以及恢复后构建产物验证的依赖关系。[`scripts/publint-all.spec.ts`](../../../../scripts/publint-all.spec.ts) 证明缺失公开导出时第一阶段会失败。CI 工作流只为该进程池调用 `pnpm run check:ci:consumers`。

 ## 曾考虑的替代方案

@@ -1,4 +1,6 @@
 import { spawnSync } from 'node:child_process'
+import { mkdtempSync, rmSync, symlinkSync } from 'node:fs'
+import { tmpdir } from 'node:os'
 import { join } from 'node:path'
 import { afterEach, describe, expect, it, vi } from 'vitest'
 import {
@@ -12,7 +14,6 @@ import {
  listedGatePlan,
  parseCliRequest,
  replayCommand,
-  resolveGateEnvironment,
  resolvePlanConcurrency,
  runGate,
  validateGatePlan,
@@ -89,6 +90,7 @@ describe('gate plan validation', () => {
  it.each([
    ['empty', plan([]), /plan has no gates/],
    ['duplicate ids', plan([gate('same'), gate('same')]), /duplicate gate id "same"/],
+    ['unsafe ids', plan([gate('unsafe id')]), /gate id "unsafe id" must contain only lowercase letters/],
    ['unknown dependencies', plan([gate('subject', { needs: ['missing'] })]), /depends on unknown gate "missing"/],
    ['cycles', plan([gate('first', { needs: ['second'] }), gate('second', { needs: ['first'] })]), /dependency cycle: first -> second -> first/],
  ])('rejects %s before starting a child', async (_label, invalid, message) => {
@@ -138,6 +140,27 @@ describe('gate plan validation', () => {
    expect(observed).toEqual(['first:failed', 'second:passed'])
  })

+  it('propagates dependency skips in causal order', async () => {
+    const leaf = gate('leaf', { needs: ['middle'] })
+    const middle = gate('middle', { needs: ['root'] })
+    const rootGate = gate('root')
+    const execute = vi.fn(async (subject: Gate) => resultFor(subject, 'failed'))
+    const observed: string[] = []
+
+    const results = await executeGatePlan(
+      plan([leaf, middle, rootGate]),
+      1,
+      execute,
+      result => observed.push(`${result.gate.id}:${result.status}`),
+    )
+
+    expect(execute).toHaveBeenCalledOnce()
+    expect(execute).toHaveBeenCalledWith(rootGate)
+    expect(observed).toEqual(['root:failed', 'middle:skipped', 'leaf:skipped'])
+    expect(results.find(result => result.gate === middle)?.error).toBe('dependency failed or skipped: root')
+    expect(results.find(result => result.gate === leaf)?.error).toBe('dependency failed or skipped: middle')
+  })
+
  it('selects a target with its transitive dependencies in canonical plan order', () => {
    const subject = plan([
      gate('prepare'),
@@ -235,6 +258,32 @@ describe('gate plan inspection and replay', () => {
    })
  })

+  it.skipIf(process.platform === 'win32')('executes when the script entry path is a symlink', () => {
+    const temporary = mkdtempSync(join(tmpdir(), 'dsh-run-gates-entry-'))
+    const entry = join(temporary, 'run-gates.ts')
+    try {
+      symlinkSync(join(repositoryRoot, 'scripts/run-gates.ts'), entry)
+      const result = spawnSync(process.execPath, [
+        '--import',
+        'tsx',
+        entry,
+        'ci-consumers',
+        '--list',
+        '--json',
+      ], {
+        cwd: repositoryRoot,
+        encoding: 'utf8',
+        env: { ...process.env, npm_execpath: process.env.npm_execpath ?? '/private/pnpm.cjs' },
+        timeout: 10_000,
+      })
+
+      expect(result.status, result.stderr).toBe(0)
+      expect(JSON.parse(result.stdout)).toMatchObject({ mode: 'ci-consumers', maxWorkers: 7 })
+    } finally {
+      rmSync(temporary, { recursive: true, force: true })
+    }
+  })
+
  it('renders a cross-platform scheduler replay and labels focused evidence', () => {
    const subject = plan([gate('snapshot')])
    expect(replayCommand(subject, 'snapshot')).toBe('pnpm run check:all -- --only snapshot')
@@ -243,17 +292,22 @@ describe('gate plan inspection and replay', () => {
    )
  })

-  it('resolves append and set operations only when spawning', () => {
-    const resolved = resolveGateEnvironment(gate('subject', {
+  it('applies append and set operations through the child spawn environment', async () => {
+    vi.stubEnv('NODE_OPTIONS', '--trace-warnings')
+    vi.stubEnv('INHERITED', 'kept')
+    const result = await runGate(gate('subject', {
+      args: ['-e', 'process.stdout.write(JSON.stringify({ nodeOptions: process.env.NODE_OPTIONS, mode: process.env.MODE, inherited: process.env.INHERITED }))'],
      env: {
        NODE_OPTIONS: { operation: 'append', value: '--max-old-space-size=8192' },
        MODE: { operation: 'set', value: 'lib' },
      },
-    }), { NODE_OPTIONS: '--trace-warnings', INHERITED: 'kept' })
-    expect(resolved).toEqual({
-      NODE_OPTIONS: '--trace-warnings --max-old-space-size=8192',
-      MODE: 'lib',
-      INHERITED: 'kept',
+    }))
+
+    expect(result.status).toBe('passed')
+    expect(JSON.parse(result.stdout)).toEqual({
+      nodeOptions: '--trace-warnings --max-old-space-size=8192',
+      mode: 'lib',
+      inherited: 'kept',
    })
  })

@@ -9,7 +9,6 @@ import { spawn } from 'node:child_process'
 import { availableParallelism } from 'node:os'
 import { resolve } from 'node:path'
 import { performance } from 'node:perf_hooks'
-import { pathToFileURL } from 'node:url'

 const MODE_SCRIPTS = {
  'ci-primary': 'check:ci',
@@ -121,8 +120,7 @@ type GateExecutor = (gate: Gate) => Promise<GateResult>
 type ResultObserver = (result: GateResult) => void

 const root = resolve(import.meta.dirname, '..')
-const entry = process.argv[1]
-if (entry !== undefined && import.meta.url === pathToFileURL(resolve(entry)).href) {
+if (import.meta.main) {
  process.exitCode = await main(process.argv.slice(2))
 }

@@ -833,21 +831,31 @@ export function formatOnlyNotice(plan: GatePlan, gateId: string): string {
 * @param inherited - environment inherited by the runner.
 * @returns the child environment without mutating the inherited object.
 */
-export function resolveGateEnvironment(gate: Gate, inherited: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
+function resolveGateEnvironment(gate: Gate, inherited: NodeJS.ProcessEnv): NodeJS.ProcessEnv {
  const resolved = { ...inherited }
  for (const [name, override] of Object.entries(gate.env ?? {})) {
-    if (override.operation === 'set') {
-      resolved[name] = override.value
-    } else {
-      const current = resolved[name]
-      resolved[name] = current === undefined || current === ''
-        ? override.value
-        : `${current} ${override.value}`
+    switch (override.operation) {
+      case 'set':
+        resolved[name] = override.value
+        break
+      case 'append': {
+        const current = resolved[name]
+        resolved[name] = current === undefined || current === ''
+          ? override.value
+          : `${current} ${override.value}`
+        break
+      }
+      default:
+        assertNever(override)
    }
  }
  return resolved
 }

+function assertNever(value: never): never {
+  throw new Error(`run-gates: unreachable value ${JSON.stringify(value)}.`)
+}
+
 /**
 * Run a validated plan; invalid input rejects before the injected executor can start a child.
 * @param plan - complete or diagnostic plan to execute.
@@ -894,9 +902,17 @@ async function runGates(
    }

    if (running.length === 0) {
-      const pending = allGates.filter(gate => states.get(gate.id) === 'pending')
-      for (const gate of pending) {
-        const failedDeps = (gate.needs ?? []).filter(id => states.get(id) !== 'passed')
+      let pending = allGates.filter(gate => states.get(gate.id) === 'pending')
+      while (pending.length > 0) {
+        const gate = pending.find(item => (item.needs ?? []).some((id) => {
+          const state = states.get(id)
+          return state === 'failed' || state === 'skipped'
+        }))
+        if (gate === undefined) throw new Error('run-gates: validated plan stalled without a failed dependency.')
+        const failedDeps = (gate.needs ?? []).filter((id) => {
+          const state = states.get(id)
+          return state === 'failed' || state === 'skipped'
+        })
        const result: GateResult = {
          gate,
          status: 'skipped',
@@ -911,6 +927,7 @@ async function runGates(
        states.set(gate.id, 'skipped')
        results.set(gate.id, result)
        observe(result)
+        pending = pending.filter(item => item !== gate)
      }
      break
    }
@@ -1031,10 +1048,10 @@ function printResult(plan: GatePlan, result: GateResult): void {
    const environment = listedGate(result.gate).env
    console.error(`command: ${result.gate.displayCommand}`)
    if (Object.keys(environment).length > 0) console.error(`scheduler environment: ${JSON.stringify(environment)}`)
+    console.error(`outcome: ${formatGateResultReason(result)}`)
    console.error(`replay: ${replayCommand(plan, result.gate.id)}`)
  }
  printOutput(result.output)
-  if (result.error !== undefined) console.error(result.error)
 }

 function printSummary(plan: GatePlan, results: GateResult[], durationMs: number): void {