docs: record final larger-runner evidence

2026-07-22 16:55:39 +08:00
parent d7e0104a72
commit 11236c46fb
4 changed files with 28 additions and 22 deletions
@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write
-2026-07-22-evidence-based-larger-hosted-runners.md: 81de62fa15623b96ac4913ee6c6fc15e886222b0
-2026-07-22-evidence-based-larger-hosted-runners.zh.md: 4e31937284b221d92552957ecc0434d9fc2e2914
+2026-07-22-evidence-based-larger-hosted-runners.md: 30a8b0517690d70ad90f8042a3f1ba841cace76e
+2026-07-22-evidence-based-larger-hosted-runners.zh.md: 1335fc66bcbf24cdda3508210dd92786b52b9fff
@@ -28,6 +28,8 @@ Two later runs set the Linux boundaries. A [standard-runner validation](https://

 A [documentation-head repeat](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903735616) showed that 16 coverage forks still admitted the CJS-lexer crash. The coverage process completed its remaining tests in 23.73 seconds, but the dead worker left one file below threshold and correctly failed the lane. Twelve forks completed the same gate in 26.06 seconds in the final run, keeping the CPU lane below one minute while restoring process headroom.

+One later exact-head run exposed a host-image tax rather than a repository bottleneck: the CPU lane completed its six gates in 28 seconds but took 66 seconds overall because registering the 50 KB Bubblewrap package scanned the runner's 202,507-file package database and consumed 18 seconds. `scripts/prepare-ci-bubblewrap.sh` instead downloads the pinned Ubuntu 24.04 package payload, verifies its archive checksum, extracts it into the ephemeral runner directory, and runs the same functional confinement probe used by the provider. Dependency installation and this sub-second preparation still overlap. This preserves the real Bubblewrap coverage without mutating the hosted image or adding another shard.
+
 The workflow retains four manual diagnostics. `suite=larger-runner-benchmark` compares isolated critical lanes across every size, `suite=consolidated-runner-benchmark` compares whole aggregates, `suite=sharded-reference` preserves the former production shard topology, and `suite=serial-reference` remains the unsharded cross-platform completeness oracle. `suite=optimized-larger-runners` runs the exact production topology against a branch ref when a pull request cannot form a merge commit.

 The first [twelve-size critical-lane benchmark](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29895295659) used a workflow-only commit on top of the standard-runner [baseline](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29850033610), so the code, lockfile, and commands were identical:
@@ -46,19 +48,19 @@ Those isolated results showed that setup dominated but did not identify the prod

 The Linux 32-core failure was the first CJS-lexer worker crash. The 96-core aggregate was the only successful all-size result at the one-minute boundary. Although Windows repository work gained little above 16 cores, the 32-core pool can start the complete outer inventory together and, more importantly, removes an entire paid setup from production.

-The exact [all-pool validation run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29904080103) passed every job at the tested branch head:
+The exact [all-pool validation run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29905362252) passed every job at the tested branch head:

 | Production job (pool) | Active time | Repository work | Result |
 |---|---:|---:|---:|
-| Node 22.19 compatibility (Linux 4) | 30 s | compatibility smokes | passed |
-| Python 3.10 (Linux 8) | 31 s | complete keyless SDK suite | passed |
-| Production site (Linux 16) | 51 s | VitePress in 24.23 s | passed |
-| Node 26 compatibility (Linux 32) | 27 s | compatibility smokes | passed |
-| Primary CPU (Linux 64) | 56 s | 6 gates in 26.07 s | passed |
-| Primary core (Linux 96) | 49 s | 36 gates in 15.58 s | passed |
-| Windows complete (Windows 32) | 102 s | 37 gates in 29.98 s | passed |
+| Node 22.19 compatibility (Linux 4) | 26 s | compatibility smokes | passed |
+| Python 3.10 (Linux 8) | 23 s | complete keyless SDK suite | passed |
+| Production site (Linux 16) | 48 s | VitePress in 25 s | passed |
+| Node 26 compatibility (Linux 32) | 28 s | compatibility smokes | passed |
+| Primary CPU (Linux 64) | 46 s | 6 gates in 25.74 s | passed |
+| Primary core (Linux 96) | 42 s | 36 gates in 15.13 s | passed |
+| Windows complete (Windows 32) | 137 s | 37 gates in 37.74 s | passed |

-All seven paid jobs began within one second. The slowest non-Windows job finished in 56 seconds. The Windows job spent 21 seconds restoring its pnpm cache and 14 seconds installing dependencies, so its 102-second active time measures hosted setup variance as well as repository work. Every non-Windows job stays below one minute and the sole Windows job stays below three minutes.
+All seven paid jobs began within one second. The slowest non-Windows job finished in 48 seconds. The Windows job spent 25 seconds checking out, 22 seconds enabling Developer Mode, 19 seconds restoring its pnpm cache, and 16 seconds installing dependencies, so its 137-second active time measures hosted setup variance as well as repository work. Every non-Windows job stays below one minute and the sole Windows job stays below three minutes.

 ## Alternatives considered

@@ -80,7 +82,7 @@ All seven paid jobs began within one second. The slowest non-Windows job finishe

 ## Consequences

-The all-pool validation consumed one billed minute at each Linux size and two billed 32-core Windows minutes. At the configured larger-runner rates, its larger-runner cost was $0.896. The all-size critical benchmark cost $2.936. GitHub rounds each larger-runner job up to a whole minute, so reducing paid job count from 49 to seven matters as much as shortening repository work.
+The all-pool validation consumed one billed minute at each Linux size and three billed 32-core Windows minutes. At the configured larger-runner rates, its larger-runner cost was $1.058. A Windows run below 120 seconds costs $0.896 instead; both shapes remain inside the three-minute target. The all-size critical benchmark cost $2.936. GitHub rounds each larger-runner job up to a whole minute, so reducing paid job count from 49 to seven matters as much as shortening repository work.

 The existing zero-dollar Actions budget did not block larger-runner jobs. The repo-only runner group, bounded workflow topology, manual benchmark triggers, and job timeouts are the observed cost controls; the budget is not treated as an execution guard.

@@ -28,6 +28,8 @@ Windows 结构由两项冷路径观测结果确定。首个候选方案使用两

 一次[文档分支头重复验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903735616)表明，覆盖率使用 16 个 fork 时仍未能阻止 CJS 词法分析器崩溃。覆盖率进程在 23.73 秒内完成了余下测试，但已终止的工作进程让一个文件的覆盖率低于阈值，因此该通道按预期失败。最终运行使用 12 个 fork，在 26.06 秒内完成同一门禁，既让 CPU 通道保持在 1 分钟以内，也恢复了进程余量。

+后来一次分支头精确运行暴露的是主机映像的额外开销，而非仓库瓶颈：CPU 通道在 28 秒内完成了 6 项门禁，总耗时却达到 66 秒，因为注册 50 KB 的 Bubblewrap 包时扫描了运行器中记录 202,507 个文件的包数据库，耗时 18 秒。`scripts/prepare-ci-bubblewrap.sh` 改为下载固定版本的 Ubuntu 24.04 包内容，验证归档文件的校验和，将其解压到临时运行器目录，并执行与提供方相同的功能性隔离探针。依赖安装仍与这项不足 1 秒的准备工作重叠执行。这样既保留了对真实 Bubblewrap 的覆盖率，又无需修改托管映像或增加分片。
+
 工作流保留 4 项手动诊断。`suite=larger-runner-benchmark` 比较所有规格下相互独立的关键通道，`suite=consolidated-runner-benchmark` 比较完整聚合流程，`suite=sharded-reference` 保留原生产分片拓扑，`suite=serial-reference` 则继续作为未分片的跨平台完整性判定基准。当拉取请求无法生成合并提交时，`suite=optimized-larger-runners` 会直接针对分支引用运行与生产环境完全相同的拓扑。

 首次[涵盖 12 种规格的关键通道基准测试](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29895295659)以标准运行器[基线](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29850033610)为基础，只叠加了一个仅修改工作流的提交，因此代码、锁文件和命令完全相同：
@@ -46,19 +48,19 @@ Windows 结构由两项冷路径观测结果确定。首个候选方案使用两

 Linux 32 核作业的失败是首次发生的 CJS 词法分析器工作进程崩溃。在所有规格的结果中，96 核聚合作业是唯一成功达到 1 分钟边界的结果。尽管 Windows 仓库工作在超过 16 核后收益很小，32 核池可以让完整的外层清单同时启动；更重要的是，它能从生产环境中消除整整一次付费设置。

-精确的[全运行器池验证运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29904080103)在受测分支头通过了所有作业：
+精确的[全运行器池验证运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29905362252)在受测分支头通过了所有作业：

 | 生产作业（运行器池） | 活动耗时 | 仓库工作 | 结果 |
 |---|---:|---:|---:|
-| Node 22.19 兼容性（Linux 4） | 30 秒 | 兼容性冒烟测试 | 通过 |
-| Python 3.10（Linux 8） | 31 秒 | 完整无密钥 SDK 套件 | 通过 |
-| 生产网站（Linux 16） | 51 秒 | VitePress 耗时 24.23 秒 | 通过 |
-| Node 26 兼容性（Linux 32） | 27 秒 | 兼容性冒烟测试 | 通过 |
-| 主流程 CPU（Linux 64） | 56 秒 | 6 项门禁耗时 26.07 秒 | 通过 |
-| 主流程核心（Linux 96） | 49 秒 | 36 项门禁耗时 15.58 秒 | 通过 |
-| Windows 完整流程（Windows 32） | 102 秒 | 37 项门禁耗时 29.98 秒 | 通过 |
+| Node 22.19 兼容性（Linux 4） | 26 秒 | 兼容性冒烟测试 | 通过 |
+| Python 3.10（Linux 8） | 23 秒 | 完整无密钥 SDK 套件 | 通过 |
+| 生产网站（Linux 16） | 48 秒 | VitePress 耗时 25 秒 | 通过 |
+| Node 26 兼容性（Linux 32） | 28 秒 | 兼容性冒烟测试 | 通过 |
+| 主流程 CPU（Linux 64） | 46 秒 | 6 项门禁耗时 25.74 秒 | 通过 |
+| 主流程核心（Linux 96） | 42 秒 | 36 项门禁耗时 15.13 秒 | 通过 |
+| Windows 完整流程（Windows 32） | 137 秒 | 37 项门禁耗时 37.74 秒 | 通过 |

-7 个付费作业均在 1 秒内开始运行。最慢的非 Windows 作业耗时 56 秒。Windows 作业花费 21 秒恢复 pnpm 缓存、14 秒安装依赖，因此其 102 秒的活动耗时既反映托管环境的设置波动，也反映仓库工作耗时。每个非 Windows 作业均低于 1 分钟，唯一的 Windows 作业也低于 3 分钟。
+7 个付费作业均在 1 秒内开始运行。最慢的非 Windows 作业耗时 48 秒。Windows 作业花费 25 秒检出代码、22 秒启用开发人员模式、19 秒恢复 pnpm 缓存、16 秒安装依赖，因此其 137 秒的活动耗时既反映托管环境的设置波动，也反映仓库工作耗时。每个非 Windows 作业均低于 1 分钟，唯一的 Windows 作业也低于 3 分钟。

 ## 曾考虑的替代方案

@@ -80,7 +82,7 @@ Linux 32 核作业的失败是首次发生的 CJS 词法分析器工作进程崩

 ## 后果

-全运行器池验证中，每种 Linux 规格各计费 1 分钟，32 核 Windows 计费 2 分钟。按已配置的大型运行器费率计算，其大型运行器成本为 $0.896。全规格关键通道基准测试的成本为 $2.936。GitHub 会把每个大型运行器作业向上取整到整分钟计费，因此把付费作业数从 49 个减少到 7 个，与缩短仓库工作耗时同样重要。
+全运行器池验证中，每种 Linux 规格各计费 1 分钟，32 核 Windows 计费 3 分钟。按已配置的大型运行器费率计算，其大型运行器成本为 $1.058；如果 Windows 运行低于 120 秒，成本则为 $0.896。两种运行形态都保持在 3 分钟目标以内。全规格关键通道基准测试的成本为 $2.936。GitHub 会把每个大型运行器作业向上取整到整分钟计费，因此把付费作业数从 49 个减少到 7 个，与缩短仓库工作耗时同样重要。

 现有的零美元 Actions 预算并未阻止大型运行器作业。仅限本仓库的运行器组、有界的工作流拓扑、手动基准测试触发和作业超时限制才是经实测有效的成本控制机制；该预算不被视为执行防护措施。

@@ -28,3 +28,5 @@ printf '%s\n' "$root/usr/bin" >> "$GITHUB_PATH"
 sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 \
  || echo 'apparmor userns knob absent — the functional probe decides'
 "$root/usr/bin/bwrap" --version
+"$root/usr/bin/bwrap" --ro-bind / / --dev /dev --proc /proc --die-with-parent -- true
+echo 'bubblewrap functional probe passed'