ci: add serial cross-platform reference runs

2026-07-21 23:03:52 +08:00
parent 821e9b6bef
commit 06f3b246b5
5 changed files with 193 additions and 11 deletions
@@ -12,13 +12,13 @@ The artifact boundary remains load-bearing. `publint`, `verify-node-next-types`,

 ## Decision

-[CI](../../../../.github/workflows/ci.yml) bounds every non-Windows job to one minute and every Windows job to three minutes. The timeout is an executable regression ceiling; the lane design leaves headroom below it rather than treating a timeout as normal control flow.
+[CI](../../../../.github/workflows/ci.yml) treats one minute for non-Windows jobs and three minutes for Windows jobs as observed performance targets, not cancellation deadlines. Hosted-runner variance should leave complete timing evidence and useful failure logs instead of cancelling an otherwise-correct gate. The [serial cross-platform CI reference](2026-07-21-serial-cross-platform-ci-reference.md) independently runs the complete unsharded primary Node aggregate on Linux, macOS, and Windows so the optimized lane inventory is not its own completeness oracle.

 [scripts/run-gates.ts](../../../../scripts/run-gates.ts) remains the common bounded scheduler, but GitHub supplies explicit shard names for the expensive gate families. [scripts/static-shards.ts](../../../../scripts/static-shards.ts) partitions static gates into foundation, documentation-type, API-contract, catalog, prose, documentation-projection, and documentation-build lanes and rejects a missing or duplicate gate assignment. Lint uses disjoint package-source, package-test, and repository-complement lanes; the complement still starts from `.` so a new top-level lint target cannot disappear between shards, and it owns the single cross-file duplication run. [scripts/coverage-shards.ts](../../../../scripts/coverage-shards.ts) assigns every workspace package to exactly one source-coverage lane; its test expands the live package tree, so a new package makes CI red until it has an owner. Each coverage lane includes only its owned source files, repeats the exhaustive companion topology test, and runs without a preceding build because the complete coverage suite passes from a tree with every generated `lib/` removed.

 Snapshot replay uses two explicit multi-file lanes and six scenario partitions of the large ACP file. [scripts/snapshot-shards.ts](../../../../scripts/snapshot-shards.ts) owns that inventory, and its test discovers every file admitted by the snapshot config so a new file cannot land outside CI. Each snapshot job builds the shipped runtime while its Linux runner installs bubblewrap from the hosted image's existing package index, then runs only its assigned replay surface. CI explicitly retains the suite's bounded concurrency of five subprocesses instead of clamping it to the runner's two logical CPUs, because replay spends most of its time waiting on child protocol I/O. Fixture guards still inspect the complete ACP scenario table in every partition. Static, lint, coverage, and snapshot sharding changes only GitHub scheduling: the ordinary local package scripts still run their complete suites.

-Cold standalone documentation typechecking rebuilds the complete project-reference graph, so a dedicated documentation-type lane builds once and checks Markdown blocks against those declarations. The Linux documentation lane uses VitePress's MPA build to retain page rendering and dead-link validation inside the one-minute budget; the blocking Windows build also produces the normal production SPA, preserving the shipped-site check under its three-minute budget.
+Cold standalone documentation typechecking rebuilds the complete project-reference graph, so a dedicated documentation-type lane builds once and checks Markdown blocks against those declarations. The Linux documentation lane uses VitePress's MPA build to retain page rendering and dead-link validation within the observed non-Windows target; the blocking Windows build also produces the normal production SPA, preserving the shipped-site check within the observed Windows target.

 Artifacts use two lanes: one metadata lane for `publint`, NodeNext declarations, and compiled invariant loading, plus one built-bin smoke lane. Each lane produces its own build before its consumers. Repeating the short build costs runner minutes but avoids an upload/download dependency and keeps each job's critical path bounded.

@@ -26,11 +26,11 @@ Artifacts use two lanes: one metadata lane for `publint`, NodeNext declarations,

 Compatibility lanes run the source worker and Zstandard runtime smokes on every advertised Node line. TypeScript checks the source graph once in a dedicated primary Node 24 lane; repeating the same compiler analysis in runtime compatibility jobs added time without runtime-specific signal.

-The workflow caches the pnpm store, preserves native PowerShell for Windows measurements, and retains one aggregate `all checks passed` status for branch protection. Windows reuses the three exhaustive lint partitions and groups foundation/catalog/prose plus documentation-type/API-contract gates behind shared runner setups; only scheduling differs from the Linux partitions. Windows build remains blocking; the wider Windows static, lint, and artifact matrix remains observational while carrying the same three-minute ceiling.
+The workflow caches the pnpm store, preserves native PowerShell for Windows measurements, and retains one aggregate `all checks passed` status for branch protection. Windows reuses the three exhaustive lint partitions and groups foundation/catalog/prose plus documentation-type/API-contract gates behind shared runner setups; only scheduling differs from the Linux partitions. Windows build remains blocking, while the wider Windows static, lint, and artifact matrix remains observational.

 ## Alternatives considered

- **Keep the broad lanes and raise timeouts** - minimizes workflow YAML, but it preserves the measured multi-minute feedback loop and offers no regression budget.
+- **Keep the broad lanes** - minimizes workflow YAML, but it preserves the measured multi-minute feedback loop.
 - **Run every leaf gate as a separate GitHub job** - maximizes fan-out, but short generators and prose checks would spend more time preparing a runner than checking the repository.
 - **Upload one build to artifact consumers** - avoids repeated compilation, but upload/download and dependency scheduling lengthen wall time; the clean build is short enough to repeat inside bounded lanes.
 - **Keep package-manager packing in both publication gates** - delegates inventory selection to pnpm, but repeats more than 200 package-manager processes. The manifest structural gate plus publication-view fixtures make the optimized inventory contract explicit and fail on an on-disk but unpublished dependency.
@@ -0,0 +1,6 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority;
+# after editing either side, bring the other along and re-record with:
+#   pnpm run verify-translation-pairing --write
+2026-07-21-serial-cross-platform-ci-reference.md: e2cb9dbc5f8778b5a28236cb135f6dcbb979d38b
+2026-07-21-serial-cross-platform-ci-reference.zh.md: 85c3403f1acb78e60e3a811503b5b264f84242c7
@@ -0,0 +1,39 @@
+# Agent Note: Serial cross-platform CI reference
+
+Status: implemented
+
+English | [中文](2026-07-21-serial-cross-platform-ci-reference.zh.md)
+
+## Problem
+
+The pull-request workflow reaches its latency targets by partitioning static checks, lint, coverage, snapshot replay, and artifact validation across explicit GitHub jobs. Those partitions are exhaustively checked in code, but the optimized workflow still should not be its own only completeness oracle: a defect shared by shard selection and its inventory test could omit work while every optimized lane stays green.
+
+Encoding the one-minute non-Windows target and three-minute Windows target as job timeouts creates a separate failure mode. Hosted-runner startup and performance vary, so a correct gate can be cancelled at the target boundary before it emits useful diagnostics. The performance objective needs measurement against GitHub timestamps, while correctness needs enough time to finish.
+
+Reviewers also need a direct answer to a simpler question: what happens when the repository's complete primary Node CI aggregate runs without matrix selection, shard variables, or concurrent gates on each selected hosted operating system?
+
+## Decision
+
+[CI](../../../../.github/workflows/ci.yml) accepts `workflow_dispatch` in addition to its normal push and pull-request events. A manual dispatch skips the optimized and compatibility jobs and exposes three explicit jobs named `serial / linux`, `serial / macos`, and `serial / windows`. They intentionally duplicate their short checkout, runtime setup, and immutable install sequences instead of hiding the operating systems behind a matrix or reusable workflow.
+
+Each reference job runs `pnpm run check:ci` without any shard selector. `DSH_GATE_CONCURRENCY=1` makes the top-level aggregate execute one ready gate at a time; coverage, snapshot replay, built-bin smoke, and publication validation also receive worker counts of one. The three operating-system jobs may run beside one another, but each host's repository gates are serial and complete. Linux installs bubblewrap before replaying snapshots, and Windows enables Developer Mode before installing the symlinked workspace.
+
+Manual reference jobs are diagnostic and do not participate in the required `all checks passed` result. Pull-request and push events continue to run only the optimized lanes. The one-minute non-Windows and three-minute Windows objectives are evaluated from completed hosted-job timestamps and reported as measurements; they are not `timeout-minutes` values.
+
+The portable reference uses GitHub's standard `ubuntu-latest`, `macos-latest`, and `windows-2025` labels. A higher-core hosted runner remains a possible future benchmark, but it is not the default: larger runners require organization-owned labels and provisioning, while a reference oracle should remain runnable without repository-external runner configuration. Provisioning one later can change the performance experiment without changing this correctness baseline.
+
+## Alternatives considered
+
+- **Set each timeout equal to its latency target** - rejected because scheduling variance would cancel correct work and suppress the evidence needed to diagnose a regression.
+- **Trust only the optimized shard inventory** - rejected because selection and validation share implementation assumptions; an unsharded aggregate is an independent completeness check.
+- **Run the serial references on every pull request** - rejected because they deliberately trade wall time and runner consumption for simplicity and are not needed in the fast feedback loop.
+- **Use one operating-system matrix** - rejected because three named jobs make the reference surface visible without another selection mechanism.
+- **Move the fast workflow to larger runners now** - rejected as the portable default because it would couple ordinary CI to organization-specific runner capacity. It remains an opt-in experiment after such capacity has an owned label and budget.
+
+## Consequences
+
+The workflow contains duplicated setup steps and a manual reference run can take much longer than the optimized pull-request path. That duplication is deliberate: reviewers can inspect each operating system's complete command without resolving a matrix or shard inventory.
+
+The reference may expose platform failures that the optimized blocking set does not yet claim to support, especially on Windows. Such a failure is evidence about current cross-platform behavior rather than a reason to weaken or silently skip the aggregate.
+
+Removing strict duration timeouts means a latency regression is observed rather than automatically cancelled. Hosted measurements must therefore accompany performance changes, while the completed logs retain the information needed to optimize the slow lane.
@@ -0,0 +1,39 @@
+# Agent Note: 跨平台串行 CI 参考流程
+
+Status: implemented
+
+[English](2026-07-21-serial-cross-platform-ci-reference.md) | 中文
+
+## 问题
+
+拉取请求工作流通过把静态检查、lint、覆盖率、快照回放和产物验证拆分到显式的 GitHub 作业中来达到延迟目标。这些分区由代码穷举校验，但优化工作流仍不应成为自身唯一的完整性判定基准：如果分片选择逻辑及其清单测试存在同一缺陷，即使所有优化通道都保持绿灯，也可能漏掉部分工作。
+
+将非 Windows 作业的 1 分钟目标和 Windows 作业的 3 分钟目标写成作业超时，会引入另一种失败模式。托管运行器的启动时间和性能会波动，因此即使门禁本身正确，也可能在到达目标时间边界时被取消，来不及输出有用的诊断信息。性能目标需要根据 GitHub 时间戳衡量，而正确性验证需要给门禁留足完成时间。
+
+评审人还需要直接回答一个更简单的问题：在每个选定的托管操作系统上，如果仓库完整的主 Node CI 聚合流程不使用矩阵选择、分片变量或并发门禁，运行结果会怎样？
+
+## 决策
+
+[CI](../../../../.github/workflows/ci.yml) 除正常的 push 和拉取请求事件外，也接受 `workflow_dispatch`。手动触发时会跳过优化作业和兼容性作业，只提供三个名为 `serial / linux`、`serial / macos` 和 `serial / windows` 的显式作业。这些作业有意分别重复简短的代码检出、运行时设置和依赖锁定的安装步骤，不用矩阵或可复用工作流把操作系统差异隐藏起来。
+
+每个参考作业均在不设置任何分片选择器的情况下运行 `pnpm run check:ci`。`DSH_GATE_CONCURRENCY=1` 使顶层聚合每次只执行一个已经就绪的门禁；覆盖率、快照回放、built-bin 冒烟测试和发布验证的并发数也设为 1。三种操作系统的作业可以彼此并行，但每台主机上的仓库门禁都串行运行且完整执行。Linux 在回放快照前安装 bubblewrap，Windows 则在安装采用符号链接的工作区前启用开发人员模式。
+
+手动参考作业仅用于诊断，不参与必需的 `all checks passed` 结果。拉取请求和 push 事件仍只运行优化通道。系统根据已完成托管作业的时间戳评估非 Windows 作业的 1 分钟目标和 Windows 作业的 3 分钟目标，并将其报告为测量结果，而不是写成 `timeout-minutes` 值。
+
+可移植的参考流程使用 GitHub 标准的 `ubuntu-latest`、`macos-latest` 和 `windows-2025` 标签。仍可将更高核心数的托管运行器作为未来的基准测试，但不将其设为默认选择：更大型运行器需要组织自有的标签和预配，而参考判定基准应无需仓库外部的运行器配置即可运行。日后完成这类预配，可以改变性能实验而无需改变该正确性基线。
+
+## 曾考虑的替代方案
+
+- **将每个超时值设为相应延迟目标**：不予采纳，因为调度波动会中止原本正确的执行，并使诊断回归所需的证据无法产生。
+- **仅信任优化分片清单**：不予采纳，因为选择逻辑与校验逻辑共享实现假设；无分片的聚合流程是一项独立的完整性检查。
+- **在每个拉取请求上运行串行参考作业**：不予采纳，因为这些作业有意以更长的总耗时和更多运行器用量换取简单性，快速反馈循环不需要它们。
+- **使用一个操作系统矩阵**：不予采纳，因为三个具名作业无需另一套选择机制，就能让参考流程的构成清晰可见。
+- **立即把快速工作流迁移到更大型运行器**：不作为可移植的默认方案，因为这会让常规 CI 与特定组织的运行器容量耦合。等到这类容量拥有明确归属的标签和预算后，仍可将其作为一项可选实验。
+
+## 后果
+
+工作流包含重复的设置步骤，手动参考运行也可能比优化后的拉取请求路径耗时长得多。这些重复是有意保留的：评审人无需解析矩阵或分片清单，就能直接检查每种操作系统执行的完整命令。
+
+参考流程可能暴露某些平台上的故障，而优化后的阻塞门禁集合尚未声明支持这些平台，Windows 尤其如此。这类失败反映了当前的跨平台行为，不应成为削弱或静默跳过该聚合流程的理由。
+
+移除严格的时长超时后，系统会观测到延迟回归，而不是在发生回归时自动取消运行。因此，性能改动必须附带托管环境测量结果，已完成的日志则保留优化最慢通道所需的信息。
@@ -4,6 +4,7 @@ on:
  push:
    branches: [main, master]
  pull_request:
+  workflow_dispatch:

 concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
@@ -17,9 +18,9 @@ env:

 jobs:
  node-24:
+    if: github.event_name != 'workflow_dispatch'
    runs-on: ubuntu-latest
    name: node 24 / ${{ matrix.lane }}
-    timeout-minutes: 1
    env:
      DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
      DSH_PUBLINT_CONCURRENCY: ${{ matrix.publint_concurrency }}
@@ -248,9 +249,9 @@ jobs:
        run: ${{ matrix.command }}

  node-compat:
+    if: github.event_name != 'workflow_dispatch'
    runs-on: ubuntu-latest
    name: node ${{ matrix.node }}
-    timeout-minutes: 1
    env:
      DSH_GATE_CONCURRENCY: '2'
      DSH_NODE_COMPAT_SKIP_TYPECHECK: ${{ matrix.skip_typecheck }}
@@ -292,9 +293,9 @@ jobs:
        run: pnpm run check:node-compat

  python-sdk:
+    if: github.event_name != 'workflow_dispatch'
    runs-on: ubuntu-latest
    name: python 3.10 / keyless SDK
-    timeout-minutes: 1
    steps:
      - uses: actions/checkout@v6

@@ -313,9 +314,9 @@ jobs:
  # while the broader observational gate matrix below exposes the remaining
  # portability work without blocking mainline merges.
  windows-build:
+    if: github.event_name != 'workflow_dispatch'
    runs-on: windows-2025
    name: windows / build
-    timeout-minutes: 3
    steps:
      - uses: actions/checkout@v6

@@ -341,10 +342,10 @@ jobs:
  # would change the environment being measured. This job intentionally stays
  # out of all-checks-passed.needs.
  windows-gates:
+    if: github.event_name != 'workflow_dispatch'
    continue-on-error: true
    runs-on: windows-2025
    name: windows node 24 / ${{ matrix.lane }}
-    timeout-minutes: 3
    env:
      DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
      DSH_PUBLINT_CONCURRENCY: ${{ matrix.publint_concurrency }}
@@ -433,6 +434,104 @@ jobs:
        shell: pwsh
        run: ${{ matrix.command }}

+  # Manual reference runs deliberately avoid the optimized matrices above.
+  # Each host executes the complete, unsharded primary Node aggregate with one
+  # gate worker, giving reviewers a simple cross-platform oracle for completeness
+  # and timing.
+  serial-linux:
+    if: github.event_name == 'workflow_dispatch'
+    name: serial / linux
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: actions/setup-node@v6
+        with:
+          node-version: ${{ env.PRIMARY_NODE_VERSION }}
+
+      - name: Enable corepack (pnpm)
+        run: corepack enable
+
+      - name: Install (immutable)
+        run: pnpm install --frozen-lockfile
+
+      - name: Install bubblewrap (unrestrict userns)
+        run: |
+          sudo apt-get update -q
+          sudo apt-get install -yq --no-install-recommends bubblewrap
+          sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 \
+            || echo "apparmor userns knob absent — the functional probe decides"
+
+      - name: Run complete unsharded primary Node CI serially
+        env:
+          DSH_COVERAGE_MAX_WORKERS: '1'
+          DSH_E2E_MAX_WORKERS: '1'
+          DSH_GATE_CONCURRENCY: '1'
+          DSH_PUBLINT_CONCURRENCY: '1'
+          DSH_SNAPSHOT_MAX_CONCURRENCY: '1'
+        run: pnpm run check:ci
+
+  serial-macos:
+    if: github.event_name == 'workflow_dispatch'
+    name: serial / macos
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: actions/setup-node@v6
+        with:
+          node-version: ${{ env.PRIMARY_NODE_VERSION }}
+
+      - name: Enable corepack (pnpm)
+        run: corepack enable
+
+      - name: Install (immutable)
+        run: pnpm install --frozen-lockfile
+
+      - name: Run complete unsharded primary Node CI serially
+        env:
+          DSH_COVERAGE_MAX_WORKERS: '1'
+          DSH_E2E_MAX_WORKERS: '1'
+          DSH_GATE_CONCURRENCY: '1'
+          DSH_PUBLINT_CONCURRENCY: '1'
+          DSH_SNAPSHOT_MAX_CONCURRENCY: '1'
+        run: pnpm run check:ci
+
+  serial-windows:
+    if: github.event_name == 'workflow_dispatch'
+    name: serial / windows
+    runs-on: windows-2025
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Enable Developer Mode (symlink support)
+        shell: pwsh
+        run: >-
+          reg add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock"
+          /t REG_DWORD /f /v "AllowDevelopmentWithoutDevLicense" /d "1"
+
+      - uses: actions/setup-node@v6
+        with:
+          node-version: ${{ env.PRIMARY_NODE_VERSION }}
+
+      - name: Enable corepack (pnpm)
+        shell: pwsh
+        run: corepack enable
+
+      - name: Install (immutable)
+        shell: pwsh
+        run: pnpm install --frozen-lockfile
+
+      - name: Run complete unsharded primary Node CI serially
+        shell: pwsh
+        env:
+          DSH_COVERAGE_MAX_WORKERS: '1'
+          DSH_E2E_MAX_WORKERS: '1'
+          DSH_GATE_CONCURRENCY: '1'
+          DSH_PUBLINT_CONCURRENCY: '1'
+          DSH_SNAPSHOT_MAX_CONCURRENCY: '1'
+        run: pnpm run check:ci
+
  # Single stable required check for branch protection: require "all checks
  # passed" instead of enumerating matrix legs whose names change as lanes and
  # node versions evolve. Every blocking job in THIS workflow must be listed in
@@ -445,9 +544,8 @@ jobs:
  all-checks-passed:
    name: all checks passed
    runs-on: ubuntu-latest
-    timeout-minutes: 1
    needs: [node-24, node-compat, python-sdk, windows-build]
-    if: always()
+    if: always() && github.event_name != 'workflow_dispatch'
    steps:
      - name: Fail if any needed job did not succeed
        if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'skipped')