diff --git a/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.i18n.yaml b/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.i18n.yaml index 5e7792d08d..509cff58e2 100644 --- a/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write -2026-07-22-evidence-based-larger-hosted-runners.md: 5390cce001f5d493fc4fb3eaee9203c34af1175d -2026-07-22-evidence-based-larger-hosted-runners.zh.md: 04a32f9ac1e0b131a695695b8f6cece3a8ffd3fa +2026-07-22-evidence-based-larger-hosted-runners.md: 81de62fa15623b96ac4913ee6c6fc15e886222b0 +2026-07-22-evidence-based-larger-hosted-runners.zh.md: 4e31937284b221d92552957ecc0434d9fc2e2914 diff --git a/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.md b/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.md index 5390cce001..81de62fa15 100644 --- a/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.md +++ b/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.md @@ -17,7 +17,7 @@ The organization keeps twelve x64 larger-runner pools in the repo-restricted `ds Production CI assigns each of the six Linux pool sizes exactly once, assigns one 32-core Windows pool, and keeps only the final aggregator on a standard runner. The version and language jobs are environment contracts rather than slices of one gate inventory; the primary Node work has three coarse lanes instead of a gate-level shard matrix: - `node 24 / core` uses the 96-core Linux pool. One checkout, setup, cache restore, and install feeds 36 unsharded static, lint, documentation, hygiene, build, and artifact gates. `run-gates` starts up to 32 independent gates and ESLint uses 32 workers. Build starts eagerly; its artifact consumers still wait for emitted output. -- `node 24 / cpu` uses the 64-core Linux pool for six CPU- or dependency-critical gates: typecheck, coverage, build followed by snapshot replay, and two Node 24 compatibility smokes. Coverage and snapshot each use at most 16 workers. This lane builds separately so snapshot replay consumes same-lane output. Coverage stays below 32 forks because that setting twice caused Node 24's CJS lexer to terminate a Vitest worker and invalidate coverage. +- `node 24 / cpu` uses the 64-core Linux pool for six CPU- or dependency-critical gates: typecheck, coverage, build followed by snapshot replay, and two Node 24 compatibility smokes. Coverage uses at most 12 workers and snapshot uses at most 16. This lane builds separately so snapshot replay consumes same-lane output. Coverage stays at 12 forks because 32 forks crashed Node 24's CJS lexer twice and a later 16-fork run reproduced the same worker loss and invalid coverage. - `node 24 / production site` uses the 16-core Linux pool for the longest independent primary gate. This is one coarse split, not a shard matrix: the job performs one setup and one production VitePress build. - Node 22.19 compatibility, Python 3.10, and Node 26 compatibility use the 4-, 8-, and 32-core Linux pools respectively. Distinct labels avoid both standard-runner setup outliers and the delayed second allocation observed when two jobs shared one pool. - `windows node 24 / complete` uses the 32-core Windows pool. One setup feeds the required package build, the required production site build, and the complete observational portability inventory. The outer scheduler has 32 slots. Required failures fail the job; observational failures are printed as non-blocking and preserve their former advisory status. ESLint itself stays single-threaded because 16 ESLint worker threads increased full-lint time to 174.54 seconds; outer gate concurrency uses the runner without multiplying Windows worker startup and TypeScript project loading. @@ -26,6 +26,8 @@ The Windows shape followed two cold-path observations. A first candidate used tw Two later runs set the Linux boundaries. A [standard-runner validation](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902209492) took 67 seconds for Node 26 even though repository work took five seconds, because GitHub spent 36 seconds in `Set up job`. Moving the environment contracts to distinct larger pools removed that lottery. The next [all-larger-runner validation](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902541203) took 68 seconds on the 96-core primary job: repository work remained 26 seconds, but setup, cache, install, and finalization consumed 42 seconds. Moving typecheck, coverage, and the build-to-snapshot dependency chain to one coarse 64-core lane reduced the 96-core lane's repository critical path to 14.81 seconds without returning to per-gate shards. +A [documentation-head repeat](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903735616) showed that 16 coverage forks still admitted the CJS-lexer crash. The coverage process completed its remaining tests in 23.73 seconds, but the dead worker left one file below threshold and correctly failed the lane. Twelve forks completed the same gate in 26.06 seconds in the final run, keeping the CPU lane below one minute while restoring process headroom. + The workflow retains four manual diagnostics. `suite=larger-runner-benchmark` compares isolated critical lanes across every size, `suite=consolidated-runner-benchmark` compares whole aggregates, `suite=sharded-reference` preserves the former production shard topology, and `suite=serial-reference` remains the unsharded cross-platform completeness oracle. `suite=optimized-larger-runners` runs the exact production topology against a branch ref when a pull request cannot form a merge commit. The first [twelve-size critical-lane benchmark](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29895295659) used a workflow-only commit on top of the standard-runner [baseline](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29850033610), so the code, lockfile, and commands were identical: @@ -44,19 +46,19 @@ Those isolated results showed that setup dominated but did not identify the prod The Linux 32-core failure was the first CJS-lexer worker crash. The 96-core aggregate was the only successful all-size result at the one-minute boundary. Although Windows repository work gained little above 16 cores, the 32-core pool can start the complete outer inventory together and, more importantly, removes an entire paid setup from production. -The exact [all-pool validation run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903067274) passed every job at the tested branch head: +The exact [all-pool validation run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29904080103) passed every job at the tested branch head: | Production job (pool) | Active time | Repository work | Result | |---|---:|---:|---:| -| Node 22.19 compatibility (Linux 4) | 26 s | compatibility smokes | passed | -| Python 3.10 (Linux 8) | 22 s | complete keyless SDK suite | passed | -| Production site (Linux 16) | 39 s | VitePress in 22.57 s | passed | -| Node 26 compatibility (Linux 32) | 24 s | compatibility smokes | passed | -| Primary CPU (Linux 64) | 53 s | 6 gates in 23.70 s | passed | -| Primary core (Linux 96) | 47 s | 36 gates in 14.81 s | passed | -| Windows complete (Windows 32) | 109 s | 37 gates in 32.71 s | passed | +| Node 22.19 compatibility (Linux 4) | 30 s | compatibility smokes | passed | +| Python 3.10 (Linux 8) | 31 s | complete keyless SDK suite | passed | +| Production site (Linux 16) | 51 s | VitePress in 24.23 s | passed | +| Node 26 compatibility (Linux 32) | 27 s | compatibility smokes | passed | +| Primary CPU (Linux 64) | 56 s | 6 gates in 26.07 s | passed | +| Primary core (Linux 96) | 49 s | 36 gates in 15.58 s | passed | +| Windows complete (Windows 32) | 102 s | 37 gates in 29.98 s | passed | -All seven paid jobs began in the same second. The slowest non-Windows job finished in 53 seconds. The Windows job spent 20 seconds restoring its pnpm cache and 15 seconds installing dependencies, so its 109-second active time measures hosted setup variance as well as repository work. Every non-Windows job stays below one minute and the sole Windows job stays below three minutes. +All seven paid jobs began within one second. The slowest non-Windows job finished in 56 seconds. The Windows job spent 21 seconds restoring its pnpm cache and 14 seconds installing dependencies, so its 102-second active time measures hosted setup variance as well as repository work. Every non-Windows job stays below one minute and the sole Windows job stays below three minutes. ## Alternatives considered diff --git a/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.zh.md b/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.zh.md index 04a32f9ac1..4e31937284 100644 --- a/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.zh.md +++ b/.agents/notes/implemented/process/2026-07-22-evidence-based-larger-hosted-runners.zh.md @@ -17,7 +17,7 @@ Status: implemented 生产 CI 对 6 种 Linux 池规格各使用一次，另使用一个 32 核 Windows 池，只有最终聚合作业继续使用标准运行器。版本和语言作业属于环境契约，并非从同一门禁清单切出的分片；主 Node 工作采用 3 个粗粒度通道，而非门禁级分片矩阵： - `node 24 / core` 使用 96 核 Linux 池。只需执行一次代码检出、设置、缓存恢复和安装，即可供 36 项未分片的静态、lint、文档、hygiene、构建和产物门禁使用。`run-gates` 最多同时启动 32 项相互独立的门禁，ESLint 使用 32 个工作线程。构建会尽早启动；其产物消费方仍会等待生成的输出。 -- `node 24 / cpu` 使用 64 核 Linux 池运行 6 项 CPU 或依赖关键门禁：类型检查、覆盖率、构建后回放快照，以及两项 Node 24 兼容性冒烟测试。覆盖率和快照各最多使用 16 个工作进程。该通道单独构建，使快照回放使用同一通道生成的输出。覆盖率运行的 fork 数保持低于 32，因为将其设为 32 曾两次导致 Node 24 的 CJS 词法分析器终止一个 Vitest 工作进程，使覆盖率结果失效。 +- `node 24 / cpu` 使用 64 核 Linux 池运行 6 项 CPU 或依赖关键门禁：类型检查、覆盖率、构建后回放快照，以及两项 Node 24 兼容性冒烟测试。覆盖率最多使用 12 个工作进程，快照最多使用 16 个。该通道单独构建，使快照回放使用同一通道生成的输出。覆盖率运行固定使用 12 个 fork，因为 32 个 fork 曾两次导致 Node 24 的 CJS 词法分析器崩溃，而后续一次使用 16 个 fork 的运行又复现了相同的工作进程丢失，并使覆盖率结果无效。 - `node 24 / production site` 使用 16 核 Linux 池运行耗时最长的独立主门禁。这是一项粗粒度拆分，而非分片矩阵：该作业执行一次设置和一次生产版 VitePress 构建。 - Node 22.19 兼容性、Python 3.10 和 Node 26 兼容性分别使用 4 核、8 核和 32 核 Linux 池。不同标签既避免了标准运行器设置耗时的异常值，也避免了两个作业共享一个池时曾出现的第二次分配延迟。 - `windows node 24 / complete` 使用 32 核 Windows 池。一次设置供必需的包构建、必需的生产网站构建以及完整的观测性可移植性清单共用。外层调度器设有 32 个槽位。任何必需项失败都会使作业失败；观测项失败会明确打印为非阻塞，并保留其原有的建议性状态。ESLint 本身仍采用单线程，因为启用 16 个 ESLint 工作线程会把完整 lint 的耗时增加至 174.54 秒；外层门禁并发能够利用该运行器，同时不会成倍增加 Windows 工作线程的启动和 TypeScript 项目加载开销。 @@ -26,6 +26,8 @@ Windows 结构由两项冷路径观测结果确定。首个候选方案使用两 随后两次运行确定了 Linux 的边界。一次[标准运行器验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902209492)中，Node 26 作业耗时 67 秒，而仓库工作仅耗时 5 秒，因为 GitHub 在 `Set up job` 上花费了 36 秒。把各项环境契约迁移到不同的大型运行器池后，这种随机波动随之消失。接下来的[全大型运行器验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902541203)中，96 核主作业耗时 68 秒：仓库工作仍为 26 秒，但设置、缓存、安装和收尾共占用 42 秒。把类型检查、覆盖率以及构建至快照的依赖链迁移到一个粗粒度 64 核通道后，96 核通道的仓库关键路径缩短至 14.81 秒，且无需恢复按门禁分片。 +一次[文档分支头重复验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903735616)表明，覆盖率使用 16 个 fork 时仍未能阻止 CJS 词法分析器崩溃。覆盖率进程在 23.73 秒内完成了余下测试，但已终止的工作进程让一个文件的覆盖率低于阈值，因此该通道按预期失败。最终运行使用 12 个 fork，在 26.06 秒内完成同一门禁，既让 CPU 通道保持在 1 分钟以内，也恢复了进程余量。 + 工作流保留 4 项手动诊断。`suite=larger-runner-benchmark` 比较所有规格下相互独立的关键通道，`suite=consolidated-runner-benchmark` 比较完整聚合流程，`suite=sharded-reference` 保留原生产分片拓扑，`suite=serial-reference` 则继续作为未分片的跨平台完整性判定基准。当拉取请求无法生成合并提交时，`suite=optimized-larger-runners` 会直接针对分支引用运行与生产环境完全相同的拓扑。 首次[涵盖 12 种规格的关键通道基准测试](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29895295659)以标准运行器[基线](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29850033610)为基础，只叠加了一个仅修改工作流的提交，因此代码、锁文件和命令完全相同： @@ -44,19 +46,19 @@ Windows 结构由两项冷路径观测结果确定。首个候选方案使用两 Linux 32 核作业的失败是首次发生的 CJS 词法分析器工作进程崩溃。在所有规格的结果中，96 核聚合作业是唯一成功达到 1 分钟边界的结果。尽管 Windows 仓库工作在超过 16 核后收益很小，32 核池可以让完整的外层清单同时启动；更重要的是，它能从生产环境中消除整整一次付费设置。 -精确的[全运行器池验证运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903067274)在受测分支头通过了所有作业： +精确的[全运行器池验证运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29904080103)在受测分支头通过了所有作业： | 生产作业（运行器池） | 活动耗时 | 仓库工作 | 结果 | |---|---:|---:|---:| -| Node 22.19 兼容性（Linux 4） | 26 秒 | 兼容性冒烟测试 | 通过 | -| Python 3.10（Linux 8） | 22 秒 | 完整无密钥 SDK 套件 | 通过 | -| 生产网站（Linux 16） | 39 秒 | VitePress 耗时 22.57 秒 | 通过 | -| Node 26 兼容性（Linux 32） | 24 秒 | 兼容性冒烟测试 | 通过 | -| 主流程 CPU（Linux 64） | 53 秒 | 6 项门禁耗时 23.70 秒 | 通过 | -| 主流程核心（Linux 96） | 47 秒 | 36 项门禁耗时 14.81 秒 | 通过 | -| Windows 完整流程（Windows 32） | 109 秒 | 37 项门禁耗时 32.71 秒 | 通过 | +| Node 22.19 兼容性（Linux 4） | 30 秒 | 兼容性冒烟测试 | 通过 | +| Python 3.10（Linux 8） | 31 秒 | 完整无密钥 SDK 套件 | 通过 | +| 生产网站（Linux 16） | 51 秒 | VitePress 耗时 24.23 秒 | 通过 | +| Node 26 兼容性（Linux 32） | 27 秒 | 兼容性冒烟测试 | 通过 | +| 主流程 CPU（Linux 64） | 56 秒 | 6 项门禁耗时 26.07 秒 | 通过 | +| 主流程核心（Linux 96） | 49 秒 | 36 项门禁耗时 15.58 秒 | 通过 | +| Windows 完整流程（Windows 32） | 102 秒 | 37 项门禁耗时 29.98 秒 | 通过 | -7 个付费作业均在同一秒开始运行。最慢的非 Windows 作业耗时 53 秒。Windows 作业花费 20 秒恢复 pnpm 缓存、15 秒安装依赖，因此其 109 秒的活动耗时既反映托管环境的设置波动，也反映仓库工作耗时。每个非 Windows 作业均低于 1 分钟，唯一的 Windows 作业也低于 3 分钟。 +7 个付费作业均在 1 秒内开始运行。最慢的非 Windows 作业耗时 56 秒。Windows 作业花费 21 秒恢复 pnpm 缓存、14 秒安装依赖，因此其 102 秒的活动耗时既反映托管环境的设置波动，也反映仓库工作耗时。每个非 Windows 作业均低于 1 分钟，唯一的 Windows 作业也低于 3 分钟。 ## 曾考虑的替代方案