ci: consolidate primary checks on one larger runner
This commit is contained in:
Tianyi Cui
@@ -12,13 +12,13 @@ The artifact boundary remains load-bearing. `publint`, `verify-node-next-types`,
|
||||
|
||||
## Decision
|
||||
|
||||
The production topology below was superseded by [Evidence-based larger hosted runners](2026-07-22-evidence-based-larger-hosted-runners.md). Its shard selectors and former job layout remain available only through manual `suite=sharded-reference`; this note preserves why that earlier topology was implemented.
|
||||
The production topology below is historical and is superseded by [Evidence-based larger hosted runners](2026-07-22-evidence-based-larger-hosted-runners.md). The larger-runner decision removes its shard selectors and workflow jobs; this note preserves why that earlier topology was implemented.
|
||||
|
||||
[CI](../../../../.github/workflows/ci.yml) treats one minute for non-Windows jobs and three minutes for Windows jobs as observed performance targets, not cancellation deadlines. Hosted-runner variance should leave complete timing evidence and useful failure logs instead of cancelling an otherwise-correct gate. The [serial cross-platform CI reference](2026-07-21-serial-cross-platform-ci-reference.md) independently runs the complete unsharded primary Node aggregate on Linux, macOS, and Windows so the optimized lane inventory is not its own completeness oracle.
|
||||
|
||||
[scripts/run-gates.ts](../../../../scripts/run-gates.ts) remains the common bounded scheduler, but GitHub supplies explicit shard names for the expensive gate families. [scripts/static-shards.ts](../../../../scripts/static-shards.ts) partitions static gates into foundation, documentation-type, API-contract, catalog, prose, documentation-projection, and documentation-build ownership and rejects a missing or duplicate gate assignment; low-cost ownership groups may share one hosted lane to stay below the runner-provisioning concurrency wave. Linux lint uses disjoint A-C, D-M, N-S, and T-Z package-source and package-test lanes, while the longer Windows target uses one complete package-source lane and one complete package-test lane; both topologies include a repository complement that still starts from `.` so a new top-level lint target cannot disappear between shards and owns the single cross-file duplication run. [scripts/coverage-shards.ts](../../../../scripts/coverage-shards.ts) assigns every workspace package to exactly one source-coverage lane; its test expands the live package tree, so a new package makes CI red until it has an owner. Directory filters retain a trailing separator because Vitest positional filters match substrings and would otherwise admit prefix-named sibling packages. Each coverage lane includes only its owned source files, repeats the exhaustive companion topology test, and runs without a preceding build because the complete coverage suite passes from a tree with every generated `lib/` removed.
|
||||
In that topology, [scripts/run-gates.ts](../../../../scripts/run-gates.ts) was the common bounded scheduler and GitHub supplied explicit shard names for the expensive gate families. `scripts/static-shards.ts` partitioned static gates into foundation, documentation-type, API-contract, catalog, prose, documentation-projection, and documentation-build ownership and rejected a missing or duplicate gate assignment. Linux lint used disjoint A-C, D-M, N-S, and T-Z package-source and package-test lanes, while Windows used complete package-source and package-test lanes; both included a repository complement starting from `.` so new top-level targets could not disappear between shards and owned the single cross-file duplication run. `scripts/coverage-shards.ts` assigned every workspace package to exactly one source-coverage lane. Directory filters retained a trailing separator because Vitest positional filters match substrings and would otherwise admit prefix-named siblings. Each coverage lane included only its owned source files, repeated the exhaustive companion topology test, and ran without a preceding build because the complete coverage suite passes from a tree with every generated `lib/` removed.
|
||||
|
||||
Snapshot replay uses two explicit multi-file lanes and eight scenario partitions of the large ACP file. [scripts/snapshot-shards.ts](../../../../scripts/snapshot-shards.ts) owns that inventory, and its test discovers every file admitted by the snapshot config so a new file cannot land outside CI. Each snapshot job installs dependencies while its Linux runner installs bubblewrap, refreshing APT indexes and retrying if the hosted image's initial package install fails, then builds the shipped runtime and runs only its assigned replay surface. CI explicitly retains the suite's bounded concurrency of five subprocesses instead of clamping it to the runner's two logical CPUs, because replay spends most of its time waiting on child protocol I/O. Fixture guards still inspect the complete ACP scenario table in every partition. Static, lint, coverage, and snapshot sharding changes only GitHub scheduling: the ordinary local package scripts still run their complete suites.
|
||||
Snapshot replay used two explicit multi-file lanes and eight scenario partitions of the large ACP file. `scripts/snapshot-shards.ts` owned that inventory, and its test discovered every file admitted by the snapshot config. Each snapshot job installed dependencies while its Linux runner prepared Bubblewrap, built the shipped runtime, and ran only its assigned replay surface. The suite retained bounded concurrency of five subprocesses because replay spent most of its time waiting on child protocol I/O. Fixture guards still inspected the complete ACP scenario table in every partition.
|
||||
|
||||
Cold standalone documentation typechecking rebuilds the complete project-reference graph, so a dedicated documentation-type lane builds once and checks Markdown blocks against those declarations. The Linux documentation lane uses VitePress's MPA build to retain page rendering and dead-link validation within the observed non-Windows target; separate blocking Windows build and production-site lanes preserve the emitted-package and shipped-site checks without putting both critical paths in one job.
|
||||
|
||||
@@ -41,9 +41,7 @@ The workflow caches the pnpm store, keys each immutable ESLint cache to its owni
|
||||
|
||||
## Consequences
|
||||
|
||||
The Actions UI contains more matrix checks and total runner time can exceed a serial workflow, but PR wall time is the slowest bounded lane instead of the sum of unrelated work. Repeated setup and builds are the deliberate price of sub-minute non-Windows feedback and sub-three-minute Windows feedback.
|
||||
|
||||
Shard inventories are repository contracts. Static selection validates the complete live gate list at runtime, coverage tests validate exhaustive package ownership, and Vitest owns deterministic file sharding for snapshots and built-bin smokes. Adding a gate or package therefore requires an explicit scheduling decision instead of silently lengthening an existing lane.
|
||||
The shard inventories and matrix jobs described above are not part of the current repository contract. The superseding larger-runner decision keeps the complete primary inventory in one process and uses the serial suite as its independent completeness oracle.
|
||||
|
||||
The optimized publication validators rely on the manifest `files` contract enforced by `verify-package-invariants`. If publication rules grow beyond that contract, the structural gate and both staged views must change together.
|
||||
|
||||
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-22-evidence-based-larger-hosted-runners.md: 30a8b0517690d70ad90f8042a3f1ba841cace76e
|
||||
2026-07-22-evidence-based-larger-hosted-runners.zh.md: 1335fc66bcbf24cdda3508210dd92786b52b9fff
|
||||
2026-07-22-evidence-based-larger-hosted-runners.md: bb10a53058a7ae1a46d493933868887bb2f55265
|
||||
2026-07-22-evidence-based-larger-hosted-runners.zh.md: ffaf958560dae6e044da58f910c0f53e21789344
|
||||
+32
-50
@@ -14,76 +14,58 @@ Larger runners make it possible to pay setup once and parallelize inside the rep
|
||||
|
||||
The organization keeps twelve x64 larger-runner pools in the repo-restricted `dsh-larger-ci` group: Ubuntu 24.04 and Windows 2025 at 4, 8, 16, 32, 64, and 96 cores. Public IPs are disabled. Each pool has an autoscaling ceiling of 256; the ceiling does not allocate idle machines or remove the need to bound workflow demand.
|
||||
|
||||
Production CI assigns each of the six Linux pool sizes exactly once, assigns one 32-core Windows pool, and keeps only the final aggregator on a standard runner. The version and language jobs are environment contracts rather than slices of one gate inventory; the primary Node work has three coarse lanes instead of a gate-level shard matrix:
|
||||
Production CI uses five larger-runner executions and one standard-runner aggregator. The primary Node inventory is not sharded:
|
||||
|
||||
- `node 24 / core` uses the 96-core Linux pool. One checkout, setup, cache restore, and install feeds 36 unsharded static, lint, documentation, hygiene, build, and artifact gates. `run-gates` starts up to 32 independent gates and ESLint uses 32 workers. Build starts eagerly; its artifact consumers still wait for emitted output.
|
||||
- `node 24 / cpu` uses the 64-core Linux pool for six CPU- or dependency-critical gates: typecheck, coverage, build followed by snapshot replay, and two Node 24 compatibility smokes. Coverage uses at most 12 workers and snapshot uses at most 16. This lane builds separately so snapshot replay consumes same-lane output. Coverage stays at 12 forks because 32 forks crashed Node 24's CJS lexer twice and a later 16-fork run reproduced the same worker loss and invalid coverage.
|
||||
- `node 24 / production site` uses the 16-core Linux pool for the longest independent primary gate. This is one coarse split, not a shard matrix: the job performs one setup and one production VitePress build.
|
||||
- Node 22.19 compatibility, Python 3.10, and Node 26 compatibility use the 4-, 8-, and 32-core Linux pools respectively. Distinct labels avoid both standard-runner setup outliers and the delayed second allocation observed when two jobs shared one pool.
|
||||
- `windows node 24 / complete` uses the 32-core Windows pool. One setup feeds the required package build, the required production site build, and the complete observational portability inventory. The outer scheduler has 32 slots. Required failures fail the job; observational failures are printed as non-blocking and preserve their former advisory status. ESLint itself stays single-threaded because 16 ESLint worker threads increased full-lint time to 174.54 seconds; outer gate concurrency uses the runner without multiplying Windows worker startup and TypeScript project loading.
|
||||
- `node 24 / complete` uses one 96-core Linux runner. One checkout, setup, cache restore, and install feeds all 42 primary gates. `run-gates` starts up to 32 independent gates; ESLint uses 32 workers, coverage uses at most 12, and snapshot replay uses at most 16. Build starts immediately beside typecheck, coverage, lint, and documentation work, while snapshot replay and publication consumers retain explicit dependencies on emitted `lib/` output.
|
||||
- Node 22.19 and Node 26 use the 4- and 32-core Linux pools for their runtime compatibility smokes. Python 3.10 uses the 8-core Linux pool for the complete keyless SDK suite. These are environment contracts, not slices of the primary Node gate inventory.
|
||||
- `windows node 24 / complete` uses one 32-core Windows runner. One setup feeds the required package build, required production site build, and complete observational portability inventory. Required failures fail the job; observational failures are reported as non-blocking. ESLint stays single-threaded because 16 ESLint workers took 174.54 seconds, while the outer scheduler retains 32 slots.
|
||||
|
||||
The Windows shape followed two cold-path observations. A first candidate used two 16-core Windows jobs, and GitHub took 93 seconds to provision the second same-label runner despite the configured autoscaling ceiling. A later [documentation-head validation](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29900502413) took 266 seconds on a separate Windows blocking job after spending 138 seconds restoring a 153 MB pnpm cache. Combining all Windows work on one 32-core box removed the duplicate setup wave.
|
||||
The former gate-level and coarse primary shard jobs are absent from the workflow. Their static, lint, coverage, snapshot, and scenario shard selectors are also absent from the repository, so an unused diagnostic path cannot preserve a second CI architecture.
|
||||
|
||||
Two later runs set the Linux boundaries. A [standard-runner validation](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902209492) took 67 seconds for Node 26 even though repository work took five seconds, because GitHub spent 36 seconds in `Set up job`. Moving the environment contracts to distinct larger pools removed that lottery. The next [all-larger-runner validation](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902541203) took 68 seconds on the 96-core primary job: repository work remained 26 seconds, but setup, cache, install, and finalization consumed 42 seconds. Moving typecheck, coverage, and the build-to-snapshot dependency chain to one coarse 64-core lane reduced the 96-core lane's repository critical path to 14.81 seconds without returning to per-gate shards.
|
||||
An [exact-head all-size benchmark](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29908491351) ran the complete unsharded primary Node aggregate on every Linux pool before the eager-build correction:
|
||||
|
||||
A [documentation-head repeat](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903735616) showed that 16 coverage forks still admitted the CJS-lexer crash. The coverage process completed its remaining tests in 23.73 seconds, but the dead worker left one file below threshold and correctly failed the lane. Twelve forks completed the same gate in 26.06 seconds in the final run, keeping the CPU lane below one minute while restoring process headroom.
|
||||
|
||||
One later exact-head run exposed a host-image tax rather than a repository bottleneck: the CPU lane completed its six gates in 28 seconds but took 66 seconds overall because registering the 50 KB Bubblewrap package scanned the runner's 202,507-file package database and consumed 18 seconds. `scripts/prepare-ci-bubblewrap.sh` instead downloads the pinned Ubuntu 24.04 package payload, verifies its archive checksum, extracts it into the ephemeral runner directory, and runs the same functional confinement probe used by the provider. Dependency installation and this sub-second preparation still overlap. This preserves the real Bubblewrap coverage without mutating the hosted image or adding another shard.
|
||||
|
||||
The workflow retains four manual diagnostics. `suite=larger-runner-benchmark` compares isolated critical lanes across every size, `suite=consolidated-runner-benchmark` compares whole aggregates, `suite=sharded-reference` preserves the former production shard topology, and `suite=serial-reference` remains the unsharded cross-platform completeness oracle. `suite=optimized-larger-runners` runs the exact production topology against a branch ref when a pull request cannot form a merge commit.
|
||||
|
||||
The first [twelve-size critical-lane benchmark](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29895295659) used a workflow-only commit on top of the standard-runner [baseline](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29850033610), so the code, lockfile, and commands were identical:
|
||||
|
||||
| Critical job | Standard | 4 cores | 8 cores | 16 cores | 32 cores | 64 cores | 96 cores |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| Linux typecheck | 56 s | 38 s | 35 s | 40 s | 35 s | 44 s | 40 s |
|
||||
| Windows production site | 160 s | 117 s | 103 s | 113 s | 75 s | 105 s | 108 s |
|
||||
|
||||
Those isolated results showed that setup dominated but did not identify the production size. A [whole-aggregate benchmark without native ESLint concurrency](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29897826082) found a 69-second single-threaded Linux lint gate. After enabling native Linux ESLint concurrency, the [second whole-aggregate benchmark](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29898331705) produced these active job times:
|
||||
|
||||
| Job | 4 cores | 8 cores | 16 cores | 32 cores | 64 cores | 96 cores |
|
||||
| Complete Linux primary | 4 cores | 8 cores | 16 cores | 32 cores | 64 cores | 96 cores |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| Linux complete primary | 147 s | 104 s | 95 s | failed at 57 s | 66 s | 60 s |
|
||||
| Windows blocking builds | 137 s | 127 s | 113 s | 107 s | 105 s | 131 s |
|
||||
| Active time | 243 s | 144 s | 103 s | 87 s | 62 s | 65 s |
|
||||
|
||||
The Linux 32-core failure was the first CJS-lexer worker crash. The 96-core aggregate was the only successful all-size result at the one-minute boundary. Although Windows repository work gained little above 16 cores, the 32-core pool can start the complete outer inventory together and, more importantly, removes an entire paid setup from production.
|
||||
The 96-core trace spent 39.14 seconds in repository gates. Typecheck occupied 25.71 seconds, then a scheduler dependency delayed the 2.13-second build and 11.29-second snapshot replay until it finished. The same run already proved build and typecheck independently, and the former CPU lane ran them concurrently. Removing that dependency makes lint at 33.30 seconds the measured critical gate while preserving dependencies only for consumers of build output. The 64-core trace exposed the same idle chain: typecheck, build, and snapshot consumed 44.85 seconds in sequence while its independent lint and documentation builds finished in 36.83 and 36.15 seconds. More cores therefore become useful only after the repository scheduler can feed them.
|
||||
|
||||
The exact [all-pool validation run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29905362252) passed every job at the tested branch head:
|
||||
The same benchmark measured the required Windows build surfaces across every provisioned size:
|
||||
|
||||
| Production job (pool) | Active time | Repository work | Result |
|
||||
|---|---:|---:|---:|
|
||||
| Node 22.19 compatibility (Linux 4) | 26 s | compatibility smokes | passed |
|
||||
| Python 3.10 (Linux 8) | 23 s | complete keyless SDK suite | passed |
|
||||
| Production site (Linux 16) | 48 s | VitePress in 25 s | passed |
|
||||
| Node 26 compatibility (Linux 32) | 28 s | compatibility smokes | passed |
|
||||
| Primary CPU (Linux 64) | 46 s | 6 gates in 25.74 s | passed |
|
||||
| Primary core (Linux 96) | 42 s | 36 gates in 15.13 s | passed |
|
||||
| Windows complete (Windows 32) | 137 s | 37 gates in 37.74 s | passed |
|
||||
| Windows blocking builds | 4 cores | 8 cores | 16 cores | 32 cores | 64 cores | 96 cores |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| Active time | 152 s | 104 s | 104 s | 92 s | 103 s | 110 s |
|
||||
|
||||
All seven paid jobs began within one second. The slowest non-Windows job finished in 48 seconds. The Windows job spent 25 seconds checking out, 22 seconds enabling Developer Mode, 19 seconds restoring its pnpm cache, and 16 seconds installing dependencies, so its 137-second active time measures hosted setup variance as well as repository work. Every non-Windows job stays below one minute and the sole Windows job stays below three minutes.
|
||||
Repository work gains little above 16 Windows cores, but the 32-core pool can start the complete outer inventory together. A [retargeted production validation](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29907581119/attempts/2) completed the full one-box Windows inventory in 173 seconds, including coverage and snapshot replay, so Windows remains consolidated.
|
||||
|
||||
Two host effects remain part of the decision. A standard Node 26 job once spent 36 of its 67 seconds in `Set up job`, which is why environment contracts use distinct larger-runner pools instead of standard capacity. A Linux candidate spent 18 seconds registering a 50 KB Bubblewrap package because the hosted image scanned 202,507 package-database files. [`scripts/prepare-ci-bubblewrap.sh`](../../../../scripts/prepare-ci-bubblewrap.sh) instead verifies and extracts the pinned payload into the ephemeral runner directory, runs a functional confinement probe, and overlaps that preparation with dependency installation.
|
||||
|
||||
Coverage remains capped at 12 forks. Thirty-two forks crashed Node 24's CJS lexer twice, and a later 16-fork run reproduced the worker loss and invalid coverage result. Twelve forks preserve process headroom without becoming the single-host critical path.
|
||||
|
||||
The workflow retains three manual measurement suites. `suite=larger-runner-benchmark` compares isolated critical lanes across every size, `suite=consolidated-runner-benchmark` compares whole aggregates, and `suite=serial-reference` remains the unsharded cross-platform completeness oracle. `suite=optimized-larger-runners` dispatches the exact production topology against a branch ref when a pull request cannot form a merge commit.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep the former shard topology in production.** The shards can be fast when provisioned together, but 49 larger-runner jobs repeat setup and create more chances for a cold outlier. The 231-second Windows control demonstrated that a short lint shard does not protect the end-to-end job target.
|
||||
**Keep the three coarse primary Linux lanes.** The core, CPU, and production-site jobs met the latency targets, but they paid three setup waves and left primary Node work sharded after larger runners were available. The all-size trace showed that one unnecessary dependency, not a lack of host capacity, kept the single-box aggregate above one minute.
|
||||
|
||||
**Select a production size from the critical-lane benchmark.** Four cores looked cost-effective for isolated typecheck and site builds, but the full aggregate found repository-wide lint and dependent artifact work that those commands did not represent.
|
||||
**Keep the former gate-level shard topology as a manual reference.** A dormant second topology kept hundreds of workflow lines, selector modules, and scenario-partition behavior alive. The all-size and serial suites provide timing and completeness controls without preserving production code that no required job exercises.
|
||||
|
||||
**Run the production site inside the Linux core aggregate.** This reached 50 seconds with warm hosted setup, then crossed the threshold at 64 seconds when the site gate took 29.05 seconds. One coarse independent site job protects the target without returning to gate-level sharding.
|
||||
**Use the 64-core pool for the complete primary aggregate.** Its sampled active time was three seconds lower than the 96-core result because hosted setup was nine seconds faster, but its repository gates were 5.72 seconds slower. Production uses 96 cores for the shorter controllable critical path; the benchmark suite retains both pools so a sustained image or pricing change can reverse that choice with evidence.
|
||||
|
||||
**Keep every primary gate on the 96-core Linux runner.** Repository work completed in 26 seconds, but a 42-second cold path still pushed the job to 68 seconds. The 64-core CPU lane owns the three longest independent or dependency-critical paths; the remaining 36-gate core inventory completes its repository work in 14.81 seconds.
|
||||
**Keep build behind typecheck.** This orders independent compiler invocations and turns snapshot replay into a three-stage critical chain. Build output has its own success dependency, so only snapshot and publication consumers wait for it.
|
||||
|
||||
**Keep required and observational Windows checks in separate jobs.** The split preserved status semantics at the workflow level but paid setup twice, and a cold cache pushed the required job to 266 seconds. `run-gates` now preserves those semantics inside one process: build and production site are required, while the remaining inventory is explicitly non-blocking.
|
||||
**Keep compatibility and Python on standard runners.** Warm standard runs can fit, but runner setup alone has crossed the non-Windows target. Distinct larger pools isolate these environment contracts from that allocation lottery.
|
||||
|
||||
**Prebuild before starting the Linux aggregate.** This moved build onto the setup path and produced a 66-second candidate. Starting build eagerly inside `run-gates` preserves artifact dependencies while overlapping it with unrelated checks.
|
||||
**Keep required and observational Windows checks in separate jobs.** The split preserves status semantics at the workflow level but pays setup twice. `run-gates` preserves the same required versus non-blocking distinction inside one process.
|
||||
|
||||
**Use native ESLint worker concurrency on Windows.** Sixteen workers made lint more than five times slower than the final single-threaded result. Outer gate parallelism uses the 32-core runner without multiplying ESLint's Windows worker startup and TypeScript project loading.
|
||||
|
||||
**Keep compatibility and Python on standard runners.** Warm runs completed in 43 seconds or less, but one Node 26 job later spent 36 seconds in GitHub setup and crossed the target despite only five seconds of repository work. Distinct larger pools stabilize those environment contracts; the three-second final aggregator remains on a standard runner because it begins only after the paid jobs release capacity.
|
||||
**Install Bubblewrap through the system package manager.** This uses the host's package database and can dominate the job even when the payload is tiny. Pinned extraction plus a confinement probe preserves the runtime contract without mutating the hosted image.
|
||||
|
||||
## Consequences
|
||||
|
||||
The all-pool validation consumed one billed minute at each Linux size and three billed 32-core Windows minutes. At the configured larger-runner rates, its larger-runner cost was $1.058. A Windows run below 120 seconds costs $0.896 instead; both shapes remain inside the three-minute target. The all-size critical benchmark cost $2.936. GitHub rounds each larger-runner job up to a whole minute, so reducing paid job count from 49 to seven matters as much as shortening repository work.
|
||||
Primary Node CI has one job, one setup wave, one complete gate inventory, and no shard selectors. Together with two Node compatibility executions, Python, and Windows, production has five paid larger-runner executions instead of seven coarse-lane executions or 49 gate-level executions.
|
||||
|
||||
The existing zero-dollar Actions budget did not block larger-runner jobs. The repo-only runner group, bounded workflow topology, manual benchmark triggers, and job timeouts are the observed cost controls; the budget is not treated as an execution guard.
|
||||
GitHub rounds each larger-runner execution up to a whole minute, so eliminating setup waves reduces billed time as well as workflow complexity. The final aggregator remains on a standard runner because it begins only after the paid jobs release capacity.
|
||||
|
||||
Production CI depends on the organization-owned runner names in this note and in `.github/workflows/ci.yml`. Missing or renamed pools leave jobs queued instead of falling back to standard capacity. Manual all-size, consolidated, former-shard, and serial suites remain available so image, dependency, scheduler, or pricing changes can be remeasured before changing production labels.
|
||||
The current targets are observed performance contracts, not cancellation deadlines. Exact-head production runs must show every non-Windows job below one minute and the consolidated Windows job below three minutes; manual all-size and serial suites remain available when image, dependency, scheduler, or pricing changes need remeasurement.
|
||||
|
||||
Production CI depends on the organization-owned runner labels in [`.github/workflows/ci.yml`](../../../../.github/workflows/ci.yml). Missing or renamed pools leave jobs queued instead of falling back to standard capacity. All twelve pools remain provisioned so the manual benchmarks can re-evaluate the production size without an administrative setup cycle.
|
||||
+32
-50
@@ -14,76 +14,58 @@ Status: implemented
|
||||
|
||||
组织在仅限本仓库使用的 `dsh-larger-ci` 运行器组中保留 12 个 x64 大型运行器池：Ubuntu 24.04 和 Windows 2025 各设 4、8、16、32、64、96 核规格。公网 IP 已禁用。每个池的自动扩缩容上限为 256；该上限既不会分配闲置机器，也不能免除限制工作流需求的必要性。
|
||||
|
||||
生产 CI 对 6 种 Linux 池规格各使用一次，另使用一个 32 核 Windows 池，只有最终聚合作业继续使用标准运行器。版本和语言作业属于环境契约，并非从同一门禁清单切出的分片；主 Node 工作采用 3 个粗粒度通道，而非门禁级分片矩阵：
|
||||
生产 CI 包含 5 次大型运行器执行和 1 个标准运行器聚合作业。主 Node 门禁清单不再分片：
|
||||
|
||||
- `node 24 / core` 使用 96 核 Linux 池。只需执行一次代码检出、设置、缓存恢复和安装，即可供 36 项未分片的静态、lint、文档、hygiene、构建和产物门禁使用。`run-gates` 最多同时启动 32 项相互独立的门禁，ESLint 使用 32 个工作线程。构建会尽早启动；其产物消费方仍会等待生成的输出。
|
||||
- `node 24 / cpu` 使用 64 核 Linux 池运行 6 项 CPU 或依赖关键门禁：类型检查、覆盖率、构建后回放快照，以及两项 Node 24 兼容性冒烟测试。覆盖率最多使用 12 个工作进程，快照最多使用 16 个。该通道单独构建，使快照回放使用同一通道生成的输出。覆盖率运行固定使用 12 个 fork，因为 32 个 fork 曾两次导致 Node 24 的 CJS 词法分析器崩溃，而后续一次使用 16 个 fork 的运行又复现了相同的工作进程丢失，并使覆盖率结果无效。
|
||||
- `node 24 / production site` 使用 16 核 Linux 池运行耗时最长的独立主门禁。这是一项粗粒度拆分，而非分片矩阵：该作业执行一次设置和一次生产版 VitePress 构建。
|
||||
- Node 22.19 兼容性、Python 3.10 和 Node 26 兼容性分别使用 4 核、8 核和 32 核 Linux 池。不同标签既避免了标准运行器设置耗时的异常值，也避免了两个作业共享一个池时曾出现的第二次分配延迟。
|
||||
- `windows node 24 / complete` 使用 32 核 Windows 池。一次设置供必需的包构建、必需的生产网站构建以及完整的观测性可移植性清单共用。外层调度器设有 32 个槽位。任何必需项失败都会使作业失败；观测项失败会明确打印为非阻塞，并保留其原有的建议性状态。ESLint 本身仍采用单线程，因为启用 16 个 ESLint 工作线程会把完整 lint 的耗时增加至 174.54 秒；外层门禁并发能够利用该运行器，同时不会成倍增加 Windows 工作线程的启动和 TypeScript 项目加载开销。
|
||||
- `node 24 / complete` 使用一台 96 核 Linux 运行器。只需执行一次代码检出、设置、缓存恢复和安装，即可供全部 42 项主门禁使用。`run-gates` 最多同时启动 32 项相互独立的门禁；ESLint 使用 32 个工作线程，覆盖率最多使用 12 个，快照回放最多使用 16 个。构建会立即与类型检查、覆盖率、lint 和文档工作同时启动，而快照回放和发布消费方仍显式依赖生成的 `lib/` 输出。
|
||||
- Node 22.19 和 Node 26 分别使用 4 核和 32 核 Linux 池运行各自的运行时兼容性冒烟测试。Python 3.10 使用 8 核 Linux 池运行完整的无密钥 SDK 套件。这些作业属于环境契约，并非主 Node 门禁清单的分片。
|
||||
- `windows node 24 / complete` 使用一台 32 核 Windows 运行器。一次设置供必需的包构建、必需的生产网站构建以及完整的观测性可移植性清单共用。任何必需项失败都会使作业失败；观测项失败则报告为非阻塞。ESLint 保持单线程，因为 16 个 ESLint 工作线程耗时 174.54 秒，外层调度器则保留 32 个槽位。
|
||||
|
||||
Windows 结构由两项冷路径观测结果确定。首个候选方案使用两个 16 核 Windows 作业；尽管已配置自动扩缩容上限，GitHub 仍花费 93 秒才预配好第二台同标签运行器。随后一次[文档分支头验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29900502413)中，一个独立的 Windows 阻塞性作业耗时 266 秒，其中恢复 153 MB pnpm 缓存就用了 138 秒。把所有 Windows 工作合并到一台 32 核机器上，消除了这轮重复设置。
|
||||
原有的门禁级和粗粒度主流程分片作业已从工作流中移除。相应的静态、lint、覆盖率、快照和场景分片选择器也已从仓库中移除，因此未使用的诊断路径无法继续维系第二套 CI 架构。
|
||||
|
||||
随后两次运行确定了 Linux 的边界。一次[标准运行器验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902209492)中，Node 26 作业耗时 67 秒，而仓库工作仅耗时 5 秒，因为 GitHub 在 `Set up job` 上花费了 36 秒。把各项环境契约迁移到不同的大型运行器池后，这种随机波动随之消失。接下来的[全大型运行器验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29902541203)中，96 核主作业耗时 68 秒：仓库工作仍为 26 秒，但设置、缓存、安装和收尾共占用 42 秒。把类型检查、覆盖率以及构建至快照的依赖链迁移到一个粗粒度 64 核通道后，96 核通道的仓库关键路径缩短至 14.81 秒，且无需恢复按门禁分片。
|
||||
一次[分支头精确的全规格基准测试](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29908491351)在修正构建尽早启动逻辑前，对每种 Linux 池都运行了完整且未分片的主 Node 聚合流程：
|
||||
|
||||
一次[文档分支头重复验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29903735616)表明，覆盖率使用 16 个 fork 时仍未能阻止 CJS 词法分析器崩溃。覆盖率进程在 23.73 秒内完成了余下测试，但已终止的工作进程让一个文件的覆盖率低于阈值，因此该通道按预期失败。最终运行使用 12 个 fork，在 26.06 秒内完成同一门禁，既让 CPU 通道保持在 1 分钟以内，也恢复了进程余量。
|
||||
|
||||
后来一次分支头精确运行暴露的是主机映像的额外开销，而非仓库瓶颈：CPU 通道在 28 秒内完成了 6 项门禁，总耗时却达到 66 秒，因为注册 50 KB 的 Bubblewrap 包时扫描了运行器中记录 202,507 个文件的包数据库，耗时 18 秒。`scripts/prepare-ci-bubblewrap.sh` 改为下载固定版本的 Ubuntu 24.04 包内容，验证归档文件的校验和，将其解压到临时运行器目录，并执行与提供方相同的功能性隔离探针。依赖安装仍与这项不足 1 秒的准备工作重叠执行。这样既保留了对真实 Bubblewrap 的覆盖率，又无需修改托管映像或增加分片。
|
||||
|
||||
工作流保留 4 项手动诊断。`suite=larger-runner-benchmark` 比较所有规格下相互独立的关键通道，`suite=consolidated-runner-benchmark` 比较完整聚合流程，`suite=sharded-reference` 保留原生产分片拓扑，`suite=serial-reference` 则继续作为未分片的跨平台完整性判定基准。当拉取请求无法生成合并提交时，`suite=optimized-larger-runners` 会直接针对分支引用运行与生产环境完全相同的拓扑。
|
||||
|
||||
首次[涵盖 12 种规格的关键通道基准测试](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29895295659)以标准运行器[基线](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29850033610)为基础，只叠加了一个仅修改工作流的提交，因此代码、锁文件和命令完全相同：
|
||||
|
||||
| 关键作业 | 标准 | 4 核 | 8 核 | 16 核 | 32 核 | 64 核 | 96 核 |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| Linux 类型检查 | 56 秒 | 38 秒 | 35 秒 | 40 秒 | 35 秒 | 44 秒 | 40 秒 |
|
||||
| Windows 生产网站 | 160 秒 | 117 秒 | 103 秒 | 113 秒 | 75 秒 | 105 秒 | 108 秒 |
|
||||
|
||||
这些单项结果表明设置开销占主导地位，却无法确定生产环境应选用的规格。一项[未启用 ESLint 原生并发的完整聚合基准测试](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29897826082)发现，Linux 单线程 lint 门禁耗时 69 秒。启用 Linux ESLint 原生并发后，[第二次完整聚合基准测试](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29898331705)得到了以下作业活动耗时：
|
||||
|
||||
| 作业 | 4 核 | 8 核 | 16 核 | 32 核 | 64 核 | 96 核 |
|
||||
| Linux 完整主流程 | 4 核 | 8 核 | 16 核 | 32 核 | 64 核 | 96 核 |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| Linux 完整主流程 | 147 秒 | 104 秒 | 95 秒 | 57 秒时失败 | 66 秒 | 60 秒 |
|
||||
| Windows 阻塞性构建 | 137 秒 | 127 秒 | 113 秒 | 107 秒 | 105 秒 | 131 秒 |
|
||||
| 活动耗时 | 243 秒 | 144 秒 | 103 秒 | 87 秒 | 62 秒 | 65 秒 |
|
||||
|
||||
Linux 32 核作业的失败是首次发生的 CJS 词法分析器工作进程崩溃。在所有规格的结果中，96 核聚合作业是唯一成功达到 1 分钟边界的结果。尽管 Windows 仓库工作在超过 16 核后收益很小，32 核池可以让完整的外层清单同时启动；更重要的是，它能从生产环境中消除整整一次付费设置。
|
||||
96 核运行轨迹中的仓库门禁耗时 39.14 秒。类型检查占用 25.71 秒，随后一项调度器依赖使耗时 2.13 秒的构建和耗时 11.29 秒的快照回放都要等到类型检查结束后才启动。同一次运行已经分别证明构建和类型检查可以独立执行，原 CPU 通道也曾让二者并发运行。移除这项依赖后，耗时 33.30 秒的 lint 成为实测关键门禁，而只有构建输出的消费方仍保留依赖关系。64 核运行轨迹暴露了相同的空闲链：类型检查、构建和快照依次执行，共耗时 44.85 秒，而相互独立的 lint 和文档构建分别在 36.83 秒和 36.15 秒内完成。因此，只有仓库调度器能够为更多核心持续提供工作时，增加核心数才有价值。
|
||||
|
||||
精确的[全运行器池验证运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29905362252)在受测分支头通过了所有作业：
|
||||
同一项基准测试还测量了每种已预配规格上的 Windows 必需构建项：
|
||||
|
||||
| 生产作业（运行器池） | 活动耗时 | 仓库工作 | 结果 |
|
||||
|---|---:|---:|---:|
|
||||
| Node 22.19 兼容性（Linux 4） | 26 秒 | 兼容性冒烟测试 | 通过 |
|
||||
| Python 3.10（Linux 8） | 23 秒 | 完整无密钥 SDK 套件 | 通过 |
|
||||
| 生产网站（Linux 16） | 48 秒 | VitePress 耗时 25 秒 | 通过 |
|
||||
| Node 26 兼容性（Linux 32） | 28 秒 | 兼容性冒烟测试 | 通过 |
|
||||
| 主流程 CPU（Linux 64） | 46 秒 | 6 项门禁耗时 25.74 秒 | 通过 |
|
||||
| 主流程核心（Linux 96） | 42 秒 | 36 项门禁耗时 15.13 秒 | 通过 |
|
||||
| Windows 完整流程（Windows 32） | 137 秒 | 37 项门禁耗时 37.74 秒 | 通过 |
|
||||
| Windows 阻塞性构建 | 4 核 | 8 核 | 16 核 | 32 核 | 64 核 | 96 核 |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| 活动耗时 | 152 秒 | 104 秒 | 104 秒 | 92 秒 | 103 秒 | 110 秒 |
|
||||
|
||||
7 个付费作业均在 1 秒内开始运行。最慢的非 Windows 作业耗时 48 秒。Windows 作业花费 25 秒检出代码、22 秒启用开发人员模式、19 秒恢复 pnpm 缓存、16 秒安装依赖，因此其 137 秒的活动耗时既反映托管环境的设置波动，也反映仓库工作耗时。每个非 Windows 作业均低于 1 分钟，唯一的 Windows 作业也低于 3 分钟。
|
||||
Windows 仓库工作在超过 16 核后收益很小，但 32 核池可以让完整的外层清单同时启动。一次[重新定向的生产验证](https://github.com/deepseek-harness/deepseek-harness/actions/runs/29907581119/attempts/2)在 173 秒内完成了单机 Windows 完整清单，其中包括覆盖率和快照回放，因此 Windows 继续采用合并执行方式。
|
||||
|
||||
两项主机效应仍构成这项决策的依据。一个标准 Node 26 作业曾在总共 67 秒的耗时中，把 36 秒用在 `Set up job` 上，因此各项环境契约使用不同的大型运行器池，而非标准容量。一个 Linux 候选作业在注册 50 KB 的 Bubblewrap 包时耗时 18 秒，因为托管映像扫描了 202,507 个包数据库文件。[`scripts/prepare-ci-bubblewrap.sh`](../../../../scripts/prepare-ci-bubblewrap.sh) 改为验证固定包内容并将其解压到临时运行器目录，执行功能性隔离探针，并让这项准备工作与依赖安装重叠执行。
|
||||
|
||||
覆盖率仍限制为 12 个 fork。32 个 fork 曾两次导致 Node 24 的 CJS 词法分析器崩溃，后来一次使用 16 个 fork 的运行又复现了工作进程丢失和无效的覆盖率结果。12 个 fork 能保留进程余量，同时不会成为单机关键路径。
|
||||
|
||||
工作流保留 3 项手动测量套件。`suite=larger-runner-benchmark` 比较所有规格下相互独立的关键通道，`suite=consolidated-runner-benchmark` 比较完整聚合流程，`suite=serial-reference` 则继续作为未分片的跨平台完整性判定基准。当拉取请求无法生成合并提交时，`suite=optimized-larger-runners` 会直接针对分支引用触发与生产环境完全相同的拓扑。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**在生产环境中保留原分片拓扑。** 各分片在一同完成预配时可以很快，但 49 个大型运行器作业会重复设置，也增加了出现冷启动异常值的机会。耗时 231 秒的 Windows 对照结果表明，短小的 lint 分片并不能保障端到端作业达到时长目标。
|
||||
**保留 3 个粗粒度 Linux 主流程通道。** 核心、CPU 和生产网站作业均达到延迟目标，但它们需要 3 轮设置，而且在大型运行器已经可用后仍对主 Node 工作进行分片。全规格运行轨迹表明，让单机聚合流程超过 1 分钟的是一项不必要的依赖，而非主机容量不足。
|
||||
|
||||
**根据关键通道基准测试选择生产规格。** 对单独的类型检查和网站构建而言，4 核看起来具备成本效益，但完整聚合流程发现了这些命令未覆盖的全仓库 lint 和存在依赖关系的产物工作。
|
||||
**将原有的门禁级分片拓扑保留为手动参考。** 一套闲置的第二拓扑会让数百行工作流、选择器模块和场景分区行为继续存活。全规格和串行套件无需保留任何必需作业都不执行的生产代码，也能提供计时与完整性对照。
|
||||
|
||||
**在 Linux 核心聚合流程内运行生产网站。** 托管环境设置处于热状态时，此方案用时 50 秒；当网站门禁耗时 29.05 秒时，总耗时增至 64 秒，超过了目标。一项独立的粗粒度网站作业能够保障目标，而无需重新采用门禁级分片。
|
||||
**使用 64 核池运行完整主聚合流程。** 由于托管设置快了 9 秒，其采样活动耗时比 96 核结果少 3 秒，但仓库门禁慢了 5.72 秒。生产环境使用 96 核来缩短可控的关键路径；基准测试套件保留两种规格，因此如果映像或定价发生持续性变化，仍可根据证据反转这项选择。
|
||||
|
||||
**将所有主门禁都保留在 96 核 Linux 运行器上。** 仓库工作在 26 秒内完成，但 42 秒的冷路径仍把作业总耗时推高至 68 秒。64 核 CPU 通道负责 3 条耗时最长的独立或依赖关键路径；其余 36 项核心门禁清单的仓库工作耗时为 14.81 秒。
|
||||
**让构建继续等待类型检查。** 此方案会给相互独立的编译器调用排定先后顺序，并把快照回放变成 3 阶段关键链。构建输出本身有独立的成功依赖关系，因此只有快照和发布消费方需要等待它。
|
||||
|
||||
**将必需的 Windows 检查和观测性 Windows 检查保留在不同作业中。** 这种拆分在工作流层保留了状态语义，却要支付两次设置开销，而且冷缓存让必需作业的耗时达到 266 秒。`run-gates` 目前在一个进程内保留这些语义：构建和生产网站属于必需项，其余清单则明确为非阻塞。
|
||||
**让兼容性和 Python 继续使用标准运行器。** 标准运行器热运行可以达到目标，但仅运行器设置一项就曾超过非 Windows 目标。不同的大型运行器池可以让这些环境契约免受这种分配波动影响。
|
||||
|
||||
**在启动 Linux 聚合流程前预先构建。** 此方案让构建成为设置路径的一部分，并产生了一个耗时 66 秒的候选结果。在 `run-gates` 内尽早启动构建，既能保留产物依赖关系，又能让构建与无关检查重叠执行。
|
||||
**将必需的 Windows 检查和观测性 Windows 检查保留在不同作业中。** 这种拆分在工作流层保留状态语义，却需要支付两次设置开销。`run-gates` 在一个进程内保留了相同的必需与非阻塞区别。
|
||||
|
||||
**在 Windows 上使用 ESLint 原生工作线程并发。** 16 个工作线程让 lint 比最终的单线程结果慢 5 倍以上。外层门禁并发能够利用 32 核运行器，同时不会成倍增加 ESLint 在 Windows 上启动工作线程和加载 TypeScript 项目的开销。
|
||||
|
||||
**让兼容性和 Python 继续使用标准运行器。** 热运行均在 43 秒以内完成，但后来一个 Node 26 作业在 GitHub 设置上花费 36 秒，即使仓库工作仅耗时 5 秒，仍超过了目标。不同的大型运行器池能够稳定这些环境契约；最终聚合作业耗时 3 秒，仍使用标准运行器，因为它只会在付费作业释放容量后启动。
|
||||
**通过系统包管理器安装 Bubblewrap。** 此方案会使用主机的包数据库，即使包内容很小，也可能主导整个作业耗时。固定版本的解压方式配合隔离探针，无需修改托管映像即可保留运行时契约。
|
||||
|
||||
## 后果
|
||||
|
||||
全运行器池验证中，每种 Linux 规格各计费 1 分钟，32 核 Windows 计费 3 分钟。按已配置的大型运行器费率计算，其大型运行器成本为 $1.058；如果 Windows 运行低于 120 秒，成本则为 $0.896。两种运行形态都保持在 3 分钟目标以内。全规格关键通道基准测试的成本为 $2.936。GitHub 会把每个大型运行器作业向上取整到整分钟计费，因此把付费作业数从 49 个减少到 7 个，与缩短仓库工作耗时同样重要。
|
||||
主 Node CI 只有 1 个作业、1 轮设置、1 份完整门禁清单，而且没有分片选择器。加上 2 次 Node 兼容性执行、Python 和 Windows，生产环境共有 5 次付费大型运行器执行，而非 7 次粗粒度通道执行或 49 次门禁级执行。
|
||||
|
||||
现有的零美元 Actions 预算并未阻止大型运行器作业。仅限本仓库的运行器组、有界的工作流拓扑、手动基准测试触发和作业超时限制才是经实测有效的成本控制机制；该预算不被视为执行防护措施。
|
||||
GitHub 会把每次大型运行器执行向上取整到整分钟计费，因此消除设置轮次既能减少计费时长，也能降低工作流复杂度。最终聚合作业仍使用标准运行器，因为它只会在付费作业释放容量后启动。
|
||||
|
||||
生产 CI 依赖本 Agent Note 和 `.github/workflows/ci.yml` 中由组织持有的运行器名称。池缺失或改名会让作业一直排队，不会回退到标准容量。手动全规格、完整聚合、原分片和串行套件均予以保留，以便在映像、依赖、调度器或定价发生变化后重新测量，再调整生产标签。
|
||||
当前目标是基于观测得到的性能契约，而非取消截止时间。分支头精确的生产运行必须表明每个非 Windows 作业都低于 1 分钟，合并后的 Windows 作业低于 3 分钟；当映像、依赖、调度器或定价发生变化而需要重新测量时，仍可使用手动全规格和串行套件。
|
||||
|
||||
生产 CI 依赖 [`.github/workflows/ci.yml`](../../../../.github/workflows/ci.yml) 中由组织持有的运行器标签。池缺失或改名会让作业一直排队，不会回退到标准容量。全部 12 个池均保持已预配状态，因此手动基准测试无需再次经过管理配置周期，就能重新评估生产规格。
|
||||
+12
-492
@@ -15,7 +15,6 @@ on:
|
||||
- serial-reference
|
||||
- larger-runner-benchmark
|
||||
- consolidated-runner-benchmark
|
||||
- sharded-reference
|
||||
- optimized-larger-runners
|
||||
|
||||
concurrency:
|
||||
@@ -29,299 +28,21 @@ env:
|
||||
PRIMARY_NODE_VERSION: '24'
|
||||
|
||||
jobs:
|
||||
node-24-sharded:
|
||||
if: github.event_name == 'workflow_dispatch' && inputs.suite == 'sharded-reference'
|
||||
runs-on: ${{ startsWith(matrix.lane, 'snapshot-') && 'dsh-ubuntu-24-04-8core' || 'dsh-ubuntu-24-04-4core' }}
|
||||
name: node 24 / ${{ matrix.lane }}
|
||||
env:
|
||||
DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
|
||||
DSH_PUBLINT_CONCURRENCY: ${{ matrix.publint_concurrency }}
|
||||
DSH_ARTIFACT_SHARD: ${{ matrix.artifact_shard }}
|
||||
DSH_COVERAGE_MAX_WORKERS: ${{ matrix.coverage_max_workers }}
|
||||
DSH_COVERAGE_SHARD: ${{ matrix.coverage_shard }}
|
||||
DSH_LINT_SHARD: ${{ matrix.lint_shard }}
|
||||
DSH_SNAPSHOT_LANE: ${{ matrix.snapshot_lane }}
|
||||
DSH_STATIC_SHARD: ${{ matrix.static_shard }}
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: ${{ matrix.snapshot_max_concurrency }}
|
||||
DSH_SNAPSHOT_PREBUILT: ${{ matrix.snapshot_prebuilt }}
|
||||
DSH_ESLINT_CACHE: ${{ matrix.eslint_cache }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- lane: static-foundation-projection
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '4'
|
||||
static_shard: 'foundation,site-projection'
|
||||
- lane: static-doc-types
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '1'
|
||||
static_shard: doc-types
|
||||
- lane: static-api-contracts
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '4'
|
||||
static_shard: api-contracts
|
||||
- lane: static-catalogs
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '4'
|
||||
static_shard: catalogs
|
||||
- lane: static-prose
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '4'
|
||||
static_shard: prose
|
||||
- lane: static-site-mpa
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '1'
|
||||
static_shard: site-build
|
||||
- lane: typecheck
|
||||
command: pnpm run typecheck
|
||||
- lane: lint-package-sources-a-c
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-sources-a-c
|
||||
- lane: lint-package-sources-d-m
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-sources-d-m
|
||||
- lane: lint-package-sources-n-s
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-sources-n-s
|
||||
- lane: lint-package-sources-t-z
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-sources-t-z
|
||||
- lane: lint-package-tests-a-c
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-tests-a-c
|
||||
- lane: lint-package-tests-d-m
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-tests-d-m
|
||||
- lane: lint-package-tests-n-s
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-tests-n-s
|
||||
- lane: lint-package-tests-t-z
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-tests-t-z
|
||||
- lane: lint-repository
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: repository
|
||||
- lane: coverage-core-loop
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: core-loop
|
||||
- lane: coverage-state-session
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: state-session
|
||||
- lane: coverage-session-title
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: session-title
|
||||
- lane: coverage-models
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: models
|
||||
- lane: coverage-integrations
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: integrations
|
||||
- lane: coverage-sdk-capabilities
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: sdk-capabilities
|
||||
- lane: coverage-interfaces
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: interfaces
|
||||
- lane: coverage-execution
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: execution
|
||||
- lane: coverage-workflow
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: workflow
|
||||
- lane: coverage-workflow-worker
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: workflow-worker
|
||||
- lane: coverage-delegation
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: delegation
|
||||
- lane: coverage-repository
|
||||
command: pnpm run check:ci:coverage
|
||||
gate_concurrency: '1'
|
||||
coverage_max_workers: '4'
|
||||
coverage_shard: repository
|
||||
- lane: snapshot-support
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: support
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-agents
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: agents
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-1
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-1
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-2
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-2
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-3
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-3
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-4
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-4
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-5
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-5
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-6
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-6
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-7
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-7
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: snapshot-acp-8
|
||||
command: pnpm run check:ci:snapshot
|
||||
gate_concurrency: '1'
|
||||
snapshot_lane: acp-8
|
||||
snapshot_max_concurrency: '5'
|
||||
snapshot_prebuilt: '1'
|
||||
- lane: artifacts-metadata
|
||||
command: pnpm run check:ci:artifacts
|
||||
gate_concurrency: '3'
|
||||
publint_concurrency: '8'
|
||||
artifact_shard: metadata
|
||||
- lane: artifacts-smoke
|
||||
command: pnpm run check:ci:artifacts
|
||||
gate_concurrency: '1'
|
||||
artifact_shard: smoke
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.PRIMARY_NODE_VERSION }}
|
||||
|
||||
- name: Enable corepack and resolve pnpm store path
|
||||
id: pnpm-store
|
||||
run: |
|
||||
corepack enable
|
||||
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.path }}
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-
|
||||
|
||||
- name: Install (immutable)
|
||||
if: ${{ ! startsWith(matrix.lane, 'snapshot-') }}
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
# The snapshot lanes REPLAY the sandbox example's recorded scenarios,
|
||||
# re-executing their bash calls under a real runner. ubuntu-latest has
|
||||
# no bubblewrap preinstalled and no built Landlock launcher, so without
|
||||
# this the confined executions fail closed (SANDBOX_UNAVAILABLE). The
|
||||
# pinned Ubuntu payload is verified and extracted into the ephemeral
|
||||
# runner instead of paying for a system package transaction. Bubblewrap
|
||||
# preparation is independent of dependency installation and the build,
|
||||
# so it runs beside both.
|
||||
- name: Prepare built snapshot runtime and bubblewrap
|
||||
if: startsWith(matrix.lane, 'snapshot-')
|
||||
run: |
|
||||
pnpm install --frozen-lockfile &
|
||||
install_pid=$!
|
||||
bash scripts/prepare-ci-bubblewrap.sh &
|
||||
sandbox_pid=$!
|
||||
install_status=0
|
||||
wait "$install_pid" || install_status=$?
|
||||
build_status=0
|
||||
if (( install_status == 0 )); then
|
||||
pnpm run build || build_status=$?
|
||||
fi
|
||||
sandbox_status=0
|
||||
wait "$sandbox_pid" || sandbox_status=$?
|
||||
if (( install_status != 0 )); then exit "$install_status"; fi
|
||||
if (( build_status != 0 )); then exit "$build_status"; fi
|
||||
exit "$sandbox_status"
|
||||
|
||||
- uses: actions/cache@v4
|
||||
if: startsWith(matrix.lane, 'lint-')
|
||||
with:
|
||||
path: .cache/eslint
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-eslint-${{ matrix.lint_shard }}-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'tsconfig.json', 'packages/*/*/tsconfig.json', 'examples/*/tsconfig.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-eslint-${{ matrix.lint_shard }}-
|
||||
|
||||
- name: Run gates
|
||||
run: ${{ matrix.command }}
|
||||
|
||||
# One large runner pays hosted setup once, then the repository scheduler
|
||||
# overlaps the unsharded core inventory. CPU and site chains use coarse lanes.
|
||||
# overlaps the complete unsharded primary Node inventory. Build starts eagerly;
|
||||
# only consumers of emitted output wait for it.
|
||||
node-24:
|
||||
if: github.event_name != 'workflow_dispatch' || inputs.suite == 'optimized-larger-runners'
|
||||
runs-on: dsh-ubuntu-24-04-96core
|
||||
name: node 24 / core
|
||||
name: node 24 / complete
|
||||
env:
|
||||
DSH_COVERAGE_MAX_WORKERS: '12'
|
||||
DSH_ESLINT_CACHE: '1'
|
||||
DSH_ESLINT_CONCURRENCY: '32'
|
||||
DSH_GATE_CONCURRENCY: '32'
|
||||
DSH_PUBLINT_CONCURRENCY: '32'
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: '16'
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
@@ -362,53 +83,19 @@ jobs:
|
||||
if (( install_status != 0 )); then exit "$install_status"; fi
|
||||
exit "$sandbox_status"
|
||||
|
||||
- name: Run unsharded primary Node core CI concurrently
|
||||
run: pnpm run check:ci:large-runner
|
||||
- name: Run complete unsharded primary Node CI concurrently
|
||||
run: pnpm run check:ci
|
||||
|
||||
# Keep the longest independent Linux gate on its own coarse-grained runner so
|
||||
# cold install variance does not push either primary aggregate over a minute.
|
||||
node-24-site:
|
||||
if: github.event_name != 'workflow_dispatch' || inputs.suite == 'optimized-larger-runners'
|
||||
runs-on: dsh-ubuntu-24-04-16core
|
||||
name: node 24 / production site
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.PRIMARY_NODE_VERSION }}
|
||||
|
||||
- name: Enable corepack and resolve pnpm store path
|
||||
id: pnpm-store
|
||||
run: |
|
||||
corepack enable
|
||||
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.path }}
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-
|
||||
|
||||
- name: Install (immutable)
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build documentation site (production SPA)
|
||||
run: pnpm run docs:build
|
||||
|
||||
node-compat:
|
||||
if: github.event_name != 'workflow_dispatch' || inputs.suite == 'optimized-larger-runners'
|
||||
# Distinct larger-runner pools avoid both standard-runner setup outliers and
|
||||
# the delayed second allocation observed when two jobs shared one pool. The
|
||||
# primary Node row also owns the CPU/dependency-critical primary gate chain.
|
||||
# delayed allocation when independent environment contracts share one pool.
|
||||
runs-on: ${{ matrix.runner }}
|
||||
name: ${{ matrix.name }}
|
||||
env:
|
||||
DSH_COVERAGE_MAX_WORKERS: ${{ matrix.coverage_workers }}
|
||||
DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
|
||||
DSH_NODE_COMPAT_SKIP_TYPECHECK: ${{ matrix.skip_typecheck }}
|
||||
DSH_SNAPSHOT_MAX_CONCURRENCY: ${{ matrix.snapshot_workers }}
|
||||
DSH_NODE_COMPAT_SKIP_TYPECHECK: '1'
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
@@ -416,24 +103,11 @@ jobs:
|
||||
- node: '22.19'
|
||||
name: node 22.19
|
||||
runner: dsh-ubuntu-24-04-4core
|
||||
command: pnpm run check:node-compat
|
||||
gate_concurrency: '2'
|
||||
skip_typecheck: '1'
|
||||
- node: 24
|
||||
name: node 24 / cpu
|
||||
runner: dsh-ubuntu-24-04-64core
|
||||
command: pnpm run check:ci:primary-cpu
|
||||
coverage_workers: '12'
|
||||
gate_concurrency: '6'
|
||||
primary_cpu: true
|
||||
snapshot_workers: '16'
|
||||
skip_typecheck: '1'
|
||||
- node: 26
|
||||
name: node 26
|
||||
runner: dsh-ubuntu-24-04-32core
|
||||
command: pnpm run check:node-compat
|
||||
gate_concurrency: '2'
|
||||
skip_typecheck: '1'
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
@@ -455,25 +129,10 @@ jobs:
|
||||
${{ runner.os }}-node-${{ matrix.node }}-pnpm-
|
||||
|
||||
- name: Install (immutable)
|
||||
if: matrix.primary_cpu != true
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Install dependencies and prepare bubblewrap
|
||||
if: matrix.primary_cpu == true
|
||||
run: |
|
||||
pnpm install --frozen-lockfile &
|
||||
install_pid=$!
|
||||
bash scripts/prepare-ci-bubblewrap.sh &
|
||||
sandbox_pid=$!
|
||||
install_status=0
|
||||
wait "$install_pid" || install_status=$?
|
||||
sandbox_status=0
|
||||
wait "$sandbox_pid" || sandbox_status=$?
|
||||
if (( install_status != 0 )); then exit "$install_status"; fi
|
||||
exit "$sandbox_status"
|
||||
|
||||
- name: Run assigned gates
|
||||
run: ${{ matrix.command }}
|
||||
- name: Run compatibility smokes
|
||||
run: pnpm run check:node-compat
|
||||
|
||||
python-sdk:
|
||||
if: github.event_name != 'workflow_dispatch' || inputs.suite == 'optimized-larger-runners'
|
||||
@@ -493,145 +152,6 @@ jobs:
|
||||
- name: Run complete keyless Python suite
|
||||
run: uv run --python 3.10 --group test --project python/sdk pytest
|
||||
|
||||
# Manual sharded Windows reference matching the former production topology.
|
||||
# Blocking build and production-site lanes keep the already-green
|
||||
# native outputs protected without putting both critical paths in one job.
|
||||
# The broader observational gate matrix below exposes the remaining
|
||||
# portability work without blocking mainline merges.
|
||||
windows-build-sharded:
|
||||
if: github.event_name == 'workflow_dispatch' && inputs.suite == 'sharded-reference'
|
||||
runs-on: dsh-windows-2025-4core
|
||||
name: windows / build
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.PRIMARY_NODE_VERSION }}
|
||||
|
||||
- name: Install (immutable)
|
||||
run: |
|
||||
corepack enable
|
||||
pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build (tsc -b + tsdown)
|
||||
run: pnpm run build
|
||||
|
||||
windows-site-sharded:
|
||||
if: github.event_name == 'workflow_dispatch' && inputs.suite == 'sharded-reference'
|
||||
runs-on: dsh-windows-2025-4core
|
||||
name: windows / production site
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.PRIMARY_NODE_VERSION }}
|
||||
|
||||
- name: Install (immutable)
|
||||
run: |
|
||||
corepack enable
|
||||
pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build documentation site (production SPA)
|
||||
run: pnpm run docs:build
|
||||
|
||||
# Observational, non-blocking Windows static, lint, and artifact lanes. Coverage
|
||||
# and snapshot stay Linux-only until their platform-specific runtime failures
|
||||
# have dedicated support. Run the gates from native PowerShell: an MSYS parent
|
||||
# would change the environment being measured. This job intentionally stays
|
||||
# out of all-checks-passed.needs.
|
||||
windows-gates-sharded:
|
||||
if: github.event_name == 'workflow_dispatch' && inputs.suite == 'sharded-reference'
|
||||
continue-on-error: true
|
||||
runs-on: dsh-windows-2025-4core
|
||||
name: windows node 24 / ${{ matrix.lane }}
|
||||
env:
|
||||
DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }}
|
||||
DSH_PUBLINT_CONCURRENCY: ${{ matrix.publint_concurrency }}
|
||||
DSH_ARTIFACT_SHARD: ${{ matrix.artifact_shard }}
|
||||
DSH_LINT_SHARD: ${{ matrix.lint_shard }}
|
||||
DSH_STATIC_SHARD: ${{ matrix.static_shard }}
|
||||
DSH_ESLINT_CACHE: ${{ matrix.eslint_cache }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- lane: static-general
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '4'
|
||||
static_shard: 'foundation,catalogs,prose'
|
||||
- lane: static-contracts
|
||||
command: pnpm run check:ci:static
|
||||
gate_concurrency: '4'
|
||||
static_shard: 'doc-types,api-contracts'
|
||||
- lane: lint-package-sources
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-sources
|
||||
- lane: lint-package-tests
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: package-tests
|
||||
- lane: lint-repository
|
||||
command: pnpm run check:ci:lint
|
||||
gate_concurrency: '1'
|
||||
eslint_cache: '1'
|
||||
lint_shard: repository
|
||||
- lane: artifacts-metadata
|
||||
command: pnpm run check:ci:artifacts
|
||||
gate_concurrency: '3'
|
||||
publint_concurrency: '8'
|
||||
artifact_shard: metadata
|
||||
- lane: artifacts-smoke
|
||||
command: pnpm run check:ci:artifacts
|
||||
gate_concurrency: '1'
|
||||
artifact_shard: smoke
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Enable Developer Mode (symlink support)
|
||||
shell: pwsh
|
||||
run: >-
|
||||
reg add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock"
|
||||
/t REG_DWORD /f /v "AllowDevelopmentWithoutDevLicense" /d "1"
|
||||
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.PRIMARY_NODE_VERSION }}
|
||||
|
||||
- name: Enable corepack and resolve pnpm store path
|
||||
id: pnpm-store
|
||||
shell: pwsh
|
||||
run: |
|
||||
corepack enable
|
||||
"path=$(pnpm store path --silent)" >> $env:GITHUB_OUTPUT
|
||||
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.path }}
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-
|
||||
|
||||
- name: Install (immutable)
|
||||
shell: pwsh
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- uses: actions/cache@v4
|
||||
if: startsWith(matrix.lane, 'lint-')
|
||||
with:
|
||||
path: .cache/eslint
|
||||
key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-eslint-${{ matrix.lint_shard }}-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'tsconfig.json', 'packages/*/*/tsconfig.json', 'examples/*/tsconfig.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-eslint-${{ matrix.lint_shard }}-
|
||||
|
||||
- name: Run gates
|
||||
shell: pwsh
|
||||
run: ${{ matrix.command }}
|
||||
|
||||
# One Windows box shares setup across the required build/site checks and the
|
||||
# complete observational portability inventory. run-gates reports failures
|
||||
# from observational gates without allowing them to fail the required job.
|
||||
@@ -1035,7 +555,7 @@ jobs:
|
||||
all-checks-passed:
|
||||
name: all checks passed
|
||||
runs-on: ubuntu-latest
|
||||
needs: [node-24, node-24-site, node-compat, python-sdk, windows]
|
||||
needs: [node-24, node-compat, python-sdk, windows]
|
||||
if: always() && (github.event_name != 'workflow_dispatch' || inputs.suite == 'optimized-larger-runners')
|
||||
steps:
|
||||
- name: Fail if any needed job did not succeed
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { defineAcpSnapshotSuite, type Scenario, type SnapshotSuiteOptions } from '@deepseek-ai/dsh-acp-snapshot'
|
||||
import { snapshotScenarioShardFromEnv } from './snapshot-scenario-shard.ts'
|
||||
|
||||
/**
|
||||
* The acp-agent example's snapshot suite: the scenario table for
|
||||
@@ -199,12 +198,9 @@ const SCENARIOS: Scenario[] = [
|
||||
{ name: 'fs-escalation-approved', hasModelTurn: true, recorded: true, headerClass: 'sandbox' },
|
||||
]
|
||||
|
||||
const scenarioShard = snapshotScenarioShardFromEnv(process.env.DSH_SNAPSHOT_SCENARIO_SHARD)
|
||||
|
||||
defineAcpSnapshotSuite({
|
||||
agent: AGENT,
|
||||
snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'),
|
||||
scenarios: SCENARIOS,
|
||||
mode: snapshotModeFromEnv(process.env.DSH_SNAPSHOT),
|
||||
...scenarioShard === undefined ? {} : { scenarioShard },
|
||||
})
|
||||
@@ -1,20 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { snapshotScenarioShardFromEnv } from './snapshot-scenario-shard.ts'
|
||||
|
||||
describe('ACP snapshot scenario shard environment', () => {
|
||||
it('keeps ordinary snapshot runs complete', () => {
|
||||
expect(snapshotScenarioShardFromEnv()).toBeUndefined()
|
||||
expect(snapshotScenarioShardFromEnv('')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('parses a valid one-based shard', () => {
|
||||
expect(snapshotScenarioShardFromEnv('2/4')).toEqual({ index: 2, total: 4 })
|
||||
})
|
||||
|
||||
it.each(['0/1', '1/0', '2/1', '1.5/2', 'missing', '999999999999999999999/999999999999999999999'])(
|
||||
'rejects %s',
|
||||
(value) => {
|
||||
expect(() => snapshotScenarioShardFromEnv(value)).toThrow('DSH_SNAPSHOT_SCENARIO_SHARD')
|
||||
},
|
||||
)
|
||||
})
|
||||
@@ -1,18 +0,0 @@
|
||||
import type { SnapshotScenarioShard } from '@deepseek-ai/dsh-acp-snapshot'
|
||||
|
||||
/**
|
||||
* Parse the optional CI scenario shard passed to this snapshot suite.
|
||||
*
|
||||
* @param value An `INDEX/TOTAL` string or an unset value.
|
||||
* @returns A validated one-based shard, or undefined for the complete suite.
|
||||
*/
|
||||
export function snapshotScenarioShardFromEnv(value?: string): SnapshotScenarioShard | undefined {
|
||||
if (value === undefined || value === '') return undefined
|
||||
const match = /^([1-9]\d*)\/([1-9]\d*)$/.exec(value)
|
||||
if (match === null) throw new Error(`DSH_SNAPSHOT_SCENARIO_SHARD must be INDEX/TOTAL, got ${JSON.stringify(value)}`)
|
||||
const shard = { index: Number(match[1]), total: Number(match[2]) }
|
||||
if (!Number.isSafeInteger(shard.index) || !Number.isSafeInteger(shard.total) || shard.index > shard.total) {
|
||||
throw new Error(`DSH_SNAPSHOT_SCENARIO_SHARD is out of range: ${JSON.stringify(value)}`)
|
||||
}
|
||||
return shard
|
||||
}
|
||||
@@ -26,8 +26,6 @@
|
||||
"test:snapshot:record": "DSH_SNAPSHOT=record vitest run --config vitest.snapshot.config.ts --update",
|
||||
"test:snapshot:refresh": "DSH_SNAPSHOT=refresh vitest run --config vitest.snapshot.config.ts",
|
||||
"check:ci": "tsx scripts/run-gates.ts ci-primary",
|
||||
"check:ci:primary-cpu": "tsx scripts/run-gates.ts ci-primary-cpu",
|
||||
"check:ci:large-runner": "tsx scripts/run-gates.ts ci-primary-large-runner",
|
||||
"check:ci:static": "tsx scripts/run-gates.ts ci-static",
|
||||
"check:ci:lint": "tsx scripts/run-gates.ts ci-lint",
|
||||
"check:ci:coverage": "tsx scripts/run-gates.ts ci-coverage",
|
||||
|
||||
@@ -7,7 +7,7 @@ Four layers, importable separately:
|
||||
- **`launchAcpTestAgent` (launcher)** — boots a source agent under tsx or a built `lib` agent under plain Node from a temp cwd, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, surfaces asynchronous spawn failures through startup, fails closed on unhandled permission requests, and owns graceful or signalled shutdown. Shutdown waits for process exit, inherited stdio closure, and ACP parser exhaustion before resolving or propagating a child error, so captures are complete and callers can remove owned paths after either outcome. When Windows accepts forced termination but publishes its exit marker asynchronously, shutdown gives that marker a bounded grace before treating fallback refusal as a second failure. Snapshot and ordinary e2e suites share this process boundary; a test supplies only agent paths, cwd, environment overrides, and any permission policy.
|
||||
- **`runScenario` (harness)** — drives ACP JSON-RPC stdio from a deterministic `input.json` script through the launcher, tees raw stdout for the expected-output and purity checks, and harvests every persisted raw JSONL session log (parent and subagent children, primary-first) after graceful stdin EOF. `AgentUnderTest` supplies absolute `binScript`, optional `libBinScript`, `configPath`, and `tsconfigPath` paths because the subprocess cwd is outside the repo. Startup failures preserve captured agent stderr in the rejected diagnostic.
|
||||
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; cwd-rooted separators selected as canonical `/` or host-native; `session_info_update.updatedAt` → `{{updatedAt}}`; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept, the same cwd-path policy), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), and `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header Agent Note](../../../.agents/notes/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
|
||||
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, the per-header-class pin (`system-prompt.expected.md` plus `tool-schemas.expected.json`) with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL prompt/schema-scrubbed, non-pinning fixtures fully header-scrubbed). Replay may partition subprocess-backed scenarios with `scenarioShard`; every lane still runs fixture guards against the complete table, while record and refresh reject sharding because they write fixtures. Refresh preserves existing volatile fields by event position and gives a newly inserted `session/title` its preceding event's time, so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session.<n>.jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time.
|
||||
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario expected-output and re-persisted-log comparisons, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, the per-header-class pin (`system-prompt.expected.md` plus `tool-schemas.expected.json`) with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL prompt/schema-scrubbed, non-pinning fixtures fully header-scrubbed). Refresh preserves existing volatile fields by event position and gives a newly inserted `session/title` its preceding event's time, so feature-driven insertions do not churn the remainder of a fixture. Each scenario directory's `session.jsonl` plus contiguous `session.<n>.jsonl` siblings are the ordered primary/child inventory; the scenario table does not duplicate their count. Must be called at vitest collection time.
|
||||
|
||||
A consuming `*.snapshot.ts` is the scenario table plus one factory call:
|
||||
|
||||
|
||||
@@ -48,4 +48,3 @@ export {
|
||||
type Scenario,
|
||||
type SnapshotSuiteOptions,
|
||||
} from './suite.ts'
|
||||
export type { SnapshotScenarioShard } from './scenario-shard.ts'
|
||||
@@ -1,36 +0,0 @@
|
||||
/** Scenario-level sharding for one ACP snapshot suite. */
|
||||
|
||||
/** A one-based, exhaustive partition of a scenario table. */
|
||||
export interface SnapshotScenarioShard {
|
||||
/** One-based lane index. */
|
||||
index: number
|
||||
/** Total number of lanes. */
|
||||
total: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Select one stable modulo partition while rejecting empty or malformed lanes.
|
||||
*
|
||||
* @param scenarios Complete ordered scenario table.
|
||||
* @param shard Optional one-based shard description.
|
||||
* @returns The complete table or the selected non-empty partition.
|
||||
*/
|
||||
export function selectSnapshotScenarios<T>(
|
||||
scenarios: readonly T[],
|
||||
shard?: SnapshotScenarioShard,
|
||||
): T[] {
|
||||
if (shard === undefined) return [...scenarios]
|
||||
if (!Number.isSafeInteger(shard.index) || shard.index < 1) {
|
||||
throw new Error(`acp-snapshot: shard index must be a positive integer, got ${shard.index}`)
|
||||
}
|
||||
if (!Number.isSafeInteger(shard.total) || shard.total < 1) {
|
||||
throw new Error(`acp-snapshot: shard total must be a positive integer, got ${shard.total}`)
|
||||
}
|
||||
if (shard.index > shard.total) {
|
||||
throw new Error(`acp-snapshot: shard index ${shard.index} exceeds total ${shard.total}`)
|
||||
}
|
||||
if (shard.total > scenarios.length) {
|
||||
throw new Error(`acp-snapshot: ${shard.total} shards exceed ${scenarios.length} scenarios`)
|
||||
}
|
||||
return scenarios.filter((_, offset) => offset % shard.total === shard.index - 1)
|
||||
}
|
||||
@@ -29,7 +29,6 @@ import {
|
||||
scrubSystemPrompts,
|
||||
scrubToolSchemas,
|
||||
} from './normalize.ts'
|
||||
import { selectSnapshotScenarios, type SnapshotScenarioShard } from './scenario-shard.ts'
|
||||
|
||||
/** The readable system-prompt snapshot beside each header-pinning fixture. */
|
||||
const SYSTEM_PROMPT_SNAPSHOT = 'system-prompt.expected.md'
|
||||
@@ -170,11 +169,6 @@ export interface SnapshotSuiteOptions {
|
||||
snapshotsDir: string
|
||||
/** The scenario table; exactly one entry per header class must set `pinsHeader`. */
|
||||
scenarios: Scenario[]
|
||||
/**
|
||||
* Optional replay-only scenario partition. Fixture guards still validate the
|
||||
* complete table in every lane; only subprocess-backed scenario tests split.
|
||||
*/
|
||||
scenarioShard?: SnapshotScenarioShard
|
||||
/**
|
||||
* `replay` (keyless, the default tier), `record` (live API; re-records the
|
||||
* `recorded` scenarios' fixtures and refreshes the Vitest expected outputs under
|
||||
@@ -517,11 +511,7 @@ export function stabilizeRefreshLog(fresh: string, existing: string, replacement
|
||||
* @param options The agent, snapshots directory, scenario table, and mode.
|
||||
*/
|
||||
export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
const { agent, snapshotsDir, scenarios, mode, scenarioShard } = options
|
||||
if (scenarioShard !== undefined && mode !== 'replay') {
|
||||
throw new Error('acp-snapshot: scenario sharding is supported only in replay mode')
|
||||
}
|
||||
const selectedScenarios = selectSnapshotScenarios(scenarios, scenarioShard)
|
||||
const { agent, snapshotsDir, scenarios, mode } = options
|
||||
const RECORDING = mode === 'record'
|
||||
const REFRESHING = mode === 'refresh'
|
||||
const childMode: 'replay' | 'record' = RECORDING ? 'record' : 'replay'
|
||||
@@ -546,7 +536,7 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
}
|
||||
|
||||
scenarioSuite('snapshot scenarios', () => {
|
||||
for (const scenario of selectedScenarios) {
|
||||
for (const scenario of scenarios) {
|
||||
// In RECORD mode, only re-run the `recorded` (live-API) scenarios; the `authored` ones
|
||||
// (sidecar-driven errors/cancel) are never re-recorded. `posixOnly` scenarios skip on
|
||||
// Windows, where their process semantics cannot be driven.
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { selectSnapshotScenarios } from '../src/scenario-shard.ts'
|
||||
|
||||
describe('ACP snapshot scenario shards', () => {
|
||||
it('keeps the ordinary suite complete', () => {
|
||||
expect(selectSnapshotScenarios(['a', 'b', 'c'])).toEqual(['a', 'b', 'c'])
|
||||
})
|
||||
|
||||
it('partitions the ordered table without gaps or overlap', () => {
|
||||
const scenarios = ['a', 'b', 'c', 'd', 'e']
|
||||
expect(selectSnapshotScenarios(scenarios, { index: 1, total: 2 })).toEqual(['a', 'c', 'e'])
|
||||
expect(selectSnapshotScenarios(scenarios, { index: 2, total: 2 })).toEqual(['b', 'd'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
[{ index: 0, total: 1 }, 'index must be a positive integer'],
|
||||
[{ index: 1.5, total: 2 }, 'index must be a positive integer'],
|
||||
[{ index: 1, total: 0 }, 'total must be a positive integer'],
|
||||
[{ index: 1, total: Number.NaN }, 'total must be a positive integer'],
|
||||
[{ index: 3, total: 2 }, 'exceeds total'],
|
||||
[{ index: 1, total: 4 }, 'exceed 3 scenarios'],
|
||||
] as const)('rejects malformed shard %#', (shard, message) => {
|
||||
expect(() => selectSnapshotScenarios(['a', 'b', 'c'], shard)).toThrow(message)
|
||||
})
|
||||
})
|
||||
@@ -107,16 +107,6 @@ describe('defineAcpSnapshotSuite: replay mode', () => {
|
||||
defineAcpSnapshotSuite({ agent: AGENT, snapshotsDir: REPLAY_DIR, scenarios: REPLAY_SCENARIOS, mode: 'replay' })
|
||||
})
|
||||
|
||||
describe('defineAcpSnapshotSuite: sharded replay mode', () => {
|
||||
defineAcpSnapshotSuite({
|
||||
agent: AGENT,
|
||||
snapshotsDir: REPLAY_DIR,
|
||||
scenarios: REPLAY_SCENARIOS,
|
||||
mode: 'replay',
|
||||
scenarioShard: { index: 2, total: 2 },
|
||||
})
|
||||
})
|
||||
|
||||
// The record suite's tests run in registration order: rec-pin re-records the
|
||||
// pinned fixture FIRST, so rec-child's uniformity guard reads the fresh pin.
|
||||
describe('defineAcpSnapshotSuite: record mode', () => {
|
||||
@@ -166,18 +156,6 @@ describe('defineAcpSnapshotSuite: record inventory write-back', () => {
|
||||
})
|
||||
|
||||
describe('defineAcpSnapshotSuite: registration contract', () => {
|
||||
it('rejects scenario sharding in a fixture-writing mode', () => {
|
||||
expect(() => {
|
||||
defineAcpSnapshotSuite({
|
||||
agent: AGENT,
|
||||
snapshotsDir: REPLAY_DIR,
|
||||
scenarios: REPLAY_SCENARIOS,
|
||||
mode: 'refresh',
|
||||
scenarioShard: { index: 1, total: 2 },
|
||||
})
|
||||
}).toThrow('supported only in replay mode')
|
||||
})
|
||||
|
||||
it("throws when a scenario's header class has no pinning scenario", () => {
|
||||
expect(() => {
|
||||
defineAcpSnapshotSuite({
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
import { readdirSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { coverageArgs, coverageShards } from './coverage-shards.ts'
|
||||
|
||||
const repositoryRoot = resolve(import.meta.dirname, '..')
|
||||
|
||||
describe('coverage shards', () => {
|
||||
it('assigns every workspace package to exactly one lane', () => {
|
||||
const packagesRoot = resolve(repositoryRoot, 'packages')
|
||||
const workspacePackages = readdirSync(packagesRoot, { withFileTypes: true })
|
||||
.filter(group => group.isDirectory())
|
||||
.flatMap(group => readdirSync(resolve(packagesRoot, group.name), { withFileTypes: true })
|
||||
.filter(entry => entry.isDirectory())
|
||||
.map(entry => `${group.name}/${entry.name}`))
|
||||
.sort()
|
||||
const assignedPackages = coverageShards.flatMap(shard => shard.packageRoots.flatMap((packageRoot) => {
|
||||
if (packageRoot.includes('/')) return [packageRoot]
|
||||
return readdirSync(resolve(packagesRoot, packageRoot), { withFileTypes: true })
|
||||
.filter(entry => entry.isDirectory())
|
||||
.map(entry => `${packageRoot}/${entry.name}`)
|
||||
}))
|
||||
|
||||
expect([...assignedPackages].sort()).toEqual(workspacePackages)
|
||||
expect(new Set(assignedPackages).size).toBe(assignedPackages.length)
|
||||
})
|
||||
|
||||
it.each(coverageShards)('selects tests and source includes for $name', (shard) => {
|
||||
const args = coverageArgs(shard.name)
|
||||
for (const packageRoot of shard.packageRoots) {
|
||||
expect(args).toContain(`packages/${packageRoot}/`)
|
||||
expect(args).toContain(packageRoot.includes('/')
|
||||
? `--coverage.include=packages/${packageRoot}/src/**/*.ts`
|
||||
: `--coverage.include=packages/${packageRoot}/*/src/**/*.ts`)
|
||||
}
|
||||
if ('extraTestRoots' in shard) {
|
||||
for (const testRoot of shard.extraTestRoots) expect(args).toContain(`${testRoot}/`)
|
||||
}
|
||||
expect(args).toContain('scripts/test-invariants.spec.ts')
|
||||
expect(new Set(args).size).toBe(args.length)
|
||||
})
|
||||
|
||||
it('rejects an unknown lane', () => {
|
||||
expect(() => coverageArgs('missing')).toThrow('unknown DSH_COVERAGE_SHARD')
|
||||
})
|
||||
})
|
||||
@@ -1,114 +0,0 @@
|
||||
/** Coverage shard definitions for the GitHub Actions source-test lanes. */
|
||||
|
||||
/** A coverage lane that owns complete package roots and optional cross-package tests. */
|
||||
export interface CoverageShard {
|
||||
/** Stable lane identifier passed through `DSH_COVERAGE_SHARD`. */
|
||||
name: string
|
||||
/** Group or package paths below `packages/` whose tests and source coverage belong to the lane. */
|
||||
packageRoots: readonly string[]
|
||||
/** Additional test roots needed for cross-package behavior or repository scripts. */
|
||||
extraTestRoots?: readonly string[]
|
||||
}
|
||||
|
||||
/** Exhaustive, non-overlapping ownership of workspace packages in coverage CI. */
|
||||
export const coverageShards = [
|
||||
{
|
||||
name: 'core-loop',
|
||||
packageRoots: ['core/agent', 'core/agent-loop', 'core/tools'],
|
||||
},
|
||||
{
|
||||
name: 'state-session',
|
||||
packageRoots: [
|
||||
'core/session',
|
||||
'core/scope',
|
||||
'core/system-prompt',
|
||||
'context',
|
||||
'session-persistence',
|
||||
'session-query',
|
||||
'support/invariants',
|
||||
],
|
||||
extraTestRoots: [
|
||||
'packages/examples/cli-demo/tests',
|
||||
'packages/llm/token-meter/tests',
|
||||
'scripts',
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'models',
|
||||
packageRoots: ['llm', 'compact'],
|
||||
},
|
||||
{
|
||||
name: 'session-title',
|
||||
packageRoots: ['session-title'],
|
||||
},
|
||||
{
|
||||
name: 'integrations',
|
||||
packageRoots: ['hooks/hook-protocol', 'lsp', 'mcp', 'hooks/hooks-claude'],
|
||||
},
|
||||
{
|
||||
name: 'sdk-capabilities',
|
||||
packageRoots: [
|
||||
'sdk',
|
||||
'hooks/hooks-codex',
|
||||
'web',
|
||||
'skill',
|
||||
'spill',
|
||||
'util',
|
||||
'guard',
|
||||
'todo',
|
||||
'timeout',
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'interfaces',
|
||||
packageRoots: ['ui', 'examples', 'goal'],
|
||||
extraTestRoots: ['examples'],
|
||||
},
|
||||
{ name: 'execution', packageRoots: ['fs', 'bash', 'sandbox', 'code-runtime'] },
|
||||
{
|
||||
name: 'workflow',
|
||||
packageRoots: ['workflow/workflow', 'workflow/tool-workflow', 'workflow/tool-ralph'],
|
||||
},
|
||||
{
|
||||
name: 'workflow-worker',
|
||||
packageRoots: ['workflow/workflow-workerthread'],
|
||||
},
|
||||
{ name: 'delegation', packageRoots: ['subagent', 'tasks'] },
|
||||
{
|
||||
name: 'repository',
|
||||
packageRoots: [
|
||||
'cordis',
|
||||
'support/acp-snapshot',
|
||||
'support/agent-loop-testkit',
|
||||
'support/llm-replay',
|
||||
'support/loader-smoke',
|
||||
],
|
||||
},
|
||||
] as const satisfies readonly CoverageShard[]
|
||||
|
||||
/**
|
||||
* Build Vitest filters and coverage include globs for one source-test lane.
|
||||
*
|
||||
* @param name Stable shard name from {@link coverageShards}.
|
||||
* @returns Positional test roots followed by per-group coverage include flags.
|
||||
*/
|
||||
export function coverageArgs(name: string): string[] {
|
||||
const shard = coverageShards.find(candidate => candidate.name === name)
|
||||
if (shard === undefined) {
|
||||
throw new Error(`run-gates: unknown DSH_COVERAGE_SHARD ${JSON.stringify(name)}.`)
|
||||
}
|
||||
|
||||
// Vitest positional filters are substrings; the trailing separator keeps
|
||||
// prefix-named sibling packages out of each lane.
|
||||
const testRoots = new Set([
|
||||
...shard.packageRoots.map(packageRoot => `packages/${packageRoot}/`),
|
||||
...('extraTestRoots' in shard ? shard.extraTestRoots.map(testRoot => `${testRoot}/`) : []),
|
||||
'scripts/test-invariants.spec.ts',
|
||||
])
|
||||
return [
|
||||
...testRoots,
|
||||
...shard.packageRoots.map(packageRoot => packageRoot.includes('/')
|
||||
? `--coverage.include=packages/${packageRoot}/src/**/*.ts`
|
||||
: `--coverage.include=packages/${packageRoot}/*/src/**/*.ts`),
|
||||
]
|
||||
}
|
||||
@@ -1,81 +0,0 @@
|
||||
import { readdirSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { selectLintShard } from './lint-shards.ts'
|
||||
|
||||
const packagesRoot = resolve(import.meta.dirname, '..', 'packages')
|
||||
|
||||
describe('lint gate shards', () => {
|
||||
it('keeps the unsharded local command complete', () => {
|
||||
expect(selectLintShard()).toEqual({ eslintTargets: ['.'], includeDuplication: true })
|
||||
expect(selectLintShard('')).toEqual({ eslintTargets: ['.'], includeDuplication: true })
|
||||
})
|
||||
|
||||
it('partitions package sources and tests into alphabetic ranges plus their repository complement', () => {
|
||||
expect(selectLintShard('package-sources-a-c')).toEqual({
|
||||
eslintTargets: ['packages/[a-c]*/*/src/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-sources-d-m')).toEqual({
|
||||
eslintTargets: ['packages/[d-m]*/*/src/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-sources-n-s')).toEqual({
|
||||
eslintTargets: ['packages/[n-s]*/*/src/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-sources-t-z')).toEqual({
|
||||
eslintTargets: ['packages/[t-z]*/*/src/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-tests-a-c')).toEqual({
|
||||
eslintTargets: ['packages/[a-c]*/*/tests/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-tests-d-m')).toEqual({
|
||||
eslintTargets: ['packages/[d-m]*/*/tests/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-tests-n-s')).toEqual({
|
||||
eslintTargets: ['packages/[n-s]*/*/tests/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-tests-t-z')).toEqual({
|
||||
eslintTargets: ['packages/[t-z]*/*/tests/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-sources')).toEqual({
|
||||
eslintTargets: ['packages/*/*/src/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('package-tests')).toEqual({
|
||||
eslintTargets: ['packages/*/*/tests/**/*.ts'],
|
||||
includeDuplication: false,
|
||||
})
|
||||
expect(selectLintShard('repository')).toEqual({
|
||||
eslintTargets: [
|
||||
'.',
|
||||
'--ignore-pattern',
|
||||
'packages/*/*/src/**',
|
||||
'--ignore-pattern',
|
||||
'packages/*/*/tests/**',
|
||||
],
|
||||
includeDuplication: true,
|
||||
})
|
||||
})
|
||||
|
||||
it('assigns every package group once in the Linux topology', () => {
|
||||
const groups = readdirSync(packagesRoot, { withFileTypes: true })
|
||||
.filter(entry => entry.isDirectory())
|
||||
.map(entry => entry.name)
|
||||
.sort()
|
||||
const ranges = [/^[a-c]/u, /^[d-m]/u, /^[n-s]/u, /^[t-z]/u]
|
||||
const assignments = ranges.flatMap(range => groups.filter(group => range.test(group))).sort()
|
||||
|
||||
expect(assignments).toEqual(groups)
|
||||
})
|
||||
|
||||
it('rejects an unknown lane', () => {
|
||||
expect(() => selectLintShard('missing')).toThrow('unknown DSH_LINT_SHARD')
|
||||
})
|
||||
})
|
||||
@@ -1,56 +0,0 @@
|
||||
/** Lint-lane selection for GitHub Actions. */
|
||||
|
||||
/** One ESLint target set and whether it owns the cross-file duplication gate. */
|
||||
export interface LintSelection {
|
||||
/** Shell-free arguments passed to ESLint before its cache options. */
|
||||
eslintTargets: readonly string[]
|
||||
/** Whether this lane also runs the repository-wide duplication check. */
|
||||
includeDuplication: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Select an exhaustive lint partition without changing the ordinary local lint command.
|
||||
*
|
||||
* @param name Optional stable shard name from `DSH_LINT_SHARD`.
|
||||
* @returns ESLint targets and ownership of the duplication gate.
|
||||
*/
|
||||
export function selectLintShard(name?: string): LintSelection {
|
||||
switch (name) {
|
||||
case undefined:
|
||||
case '':
|
||||
return { eslintTargets: ['.'], includeDuplication: true }
|
||||
case 'package-sources-a-c':
|
||||
return { eslintTargets: ['packages/[a-c]*/*/src/**/*.ts'], includeDuplication: false }
|
||||
case 'package-sources-d-m':
|
||||
return { eslintTargets: ['packages/[d-m]*/*/src/**/*.ts'], includeDuplication: false }
|
||||
case 'package-sources-n-s':
|
||||
return { eslintTargets: ['packages/[n-s]*/*/src/**/*.ts'], includeDuplication: false }
|
||||
case 'package-sources-t-z':
|
||||
return { eslintTargets: ['packages/[t-z]*/*/src/**/*.ts'], includeDuplication: false }
|
||||
case 'package-sources':
|
||||
return { eslintTargets: ['packages/*/*/src/**/*.ts'], includeDuplication: false }
|
||||
case 'package-tests-a-c':
|
||||
return { eslintTargets: ['packages/[a-c]*/*/tests/**/*.ts'], includeDuplication: false }
|
||||
case 'package-tests-d-m':
|
||||
return { eslintTargets: ['packages/[d-m]*/*/tests/**/*.ts'], includeDuplication: false }
|
||||
case 'package-tests-n-s':
|
||||
return { eslintTargets: ['packages/[n-s]*/*/tests/**/*.ts'], includeDuplication: false }
|
||||
case 'package-tests-t-z':
|
||||
return { eslintTargets: ['packages/[t-z]*/*/tests/**/*.ts'], includeDuplication: false }
|
||||
case 'package-tests':
|
||||
return { eslintTargets: ['packages/*/*/tests/**/*.ts'], includeDuplication: false }
|
||||
case 'repository':
|
||||
return {
|
||||
eslintTargets: [
|
||||
'.',
|
||||
'--ignore-pattern',
|
||||
'packages/*/*/src/**',
|
||||
'--ignore-pattern',
|
||||
'packages/*/*/tests/**',
|
||||
],
|
||||
includeDuplication: true,
|
||||
}
|
||||
default:
|
||||
throw new Error(`run-gates: unknown DSH_LINT_SHARD ${JSON.stringify(name)}.`)
|
||||
}
|
||||
}
|
||||
+17
-77
@@ -8,15 +8,9 @@ import { spawn } from 'node:child_process'
|
||||
import { availableParallelism } from 'node:os'
|
||||
import { resolve } from 'node:path'
|
||||
import { performance } from 'node:perf_hooks'
|
||||
import { coverageArgs } from './coverage-shards.ts'
|
||||
import { selectLintShard } from './lint-shards.ts'
|
||||
import { selectSnapshotLane } from './snapshot-shards.ts'
|
||||
import { selectStaticGates } from './static-shards.ts'
|
||||
|
||||
type Mode =
|
||||
| 'ci-primary'
|
||||
| 'ci-primary-cpu'
|
||||
| 'ci-primary-large-runner'
|
||||
| 'ci-static'
|
||||
| 'ci-lint'
|
||||
| 'ci-coverage'
|
||||
@@ -92,8 +86,6 @@ if (results.some(result => result.gate.allowFailure !== true && (result.status =
|
||||
function parseMode(raw: string | undefined): Mode {
|
||||
switch (raw) {
|
||||
case 'ci-primary':
|
||||
case 'ci-primary-cpu':
|
||||
case 'ci-primary-large-runner':
|
||||
case 'ci-static':
|
||||
case 'ci-lint':
|
||||
case 'ci-coverage':
|
||||
@@ -107,7 +99,7 @@ function parseMode(raw: string | undefined): Mode {
|
||||
return raw
|
||||
default:
|
||||
throw new Error(
|
||||
`run-gates: expected mode ci-primary | ci-primary-cpu | ci-primary-large-runner | ci-static | ci-lint | ci-coverage | ci-snapshot | ci-artifacts | ci-windows-blocking | ci-windows-complete | ci-windows-observational | node-compat | doc-sync, got ${JSON.stringify(raw)}.`,
|
||||
`run-gates: expected mode ci-primary | ci-static | ci-lint | ci-coverage | ci-snapshot | ci-artifacts | ci-windows-blocking | ci-windows-complete | ci-windows-observational | node-compat | doc-sync, got ${JSON.stringify(raw)}.`,
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -173,25 +165,17 @@ function gatesForMode(selected: Mode): Gate[] {
|
||||
switch (selected) {
|
||||
case 'ci-primary':
|
||||
return ciPrimaryGates()
|
||||
case 'ci-primary-cpu':
|
||||
return ciPrimaryCpuGates()
|
||||
case 'ci-primary-large-runner':
|
||||
return ciPrimaryLargeRunnerGates()
|
||||
case 'ci-static':
|
||||
return ciStaticGates()
|
||||
case 'ci-lint': {
|
||||
const selection = selectLintShard(process.env.DSH_LINT_SHARD)
|
||||
case 'ci-lint':
|
||||
return [
|
||||
lintGate(selection.eslintTargets),
|
||||
...selection.includeDuplication ? [pnpmScript('duplication', 'duplication')] : [],
|
||||
lintGate(),
|
||||
pnpmScript('duplication', 'duplication'),
|
||||
]
|
||||
}
|
||||
case 'ci-coverage':
|
||||
return [coverageGate()]
|
||||
case 'ci-snapshot':
|
||||
return flagEnabled('DSH_SNAPSHOT_PREBUILT')
|
||||
? [snapshotGate([])]
|
||||
: [pnpmScript('build', 'build'), snapshotGate()]
|
||||
return [pnpmScript('build', 'build'), snapshotGate()]
|
||||
case 'ci-artifacts':
|
||||
return ciArtifactGates()
|
||||
case 'ci-windows-blocking':
|
||||
@@ -217,11 +201,12 @@ function ciPrimaryGates(): Gate[] {
|
||||
lintGate(),
|
||||
pnpmScript('duplication', 'duplication'),
|
||||
coverageGate(),
|
||||
...nodeCompatSmokeGates(),
|
||||
snapshotGate(),
|
||||
...docSyncLeafGates(),
|
||||
pnpmScript('module-graph', 'verify-module-graph', { label: 'module graph' }),
|
||||
pnpmScript('knip', 'knip'),
|
||||
pnpmScript('build', 'build', { needs: ['typecheck'] }),
|
||||
pnpmScript('build', 'build'),
|
||||
pnpmScript('publint', 'publint', { needs: ['build'] }),
|
||||
pnpmScript('node-next-types', 'verify-node-next-types', {
|
||||
label: 'node-next types',
|
||||
@@ -232,30 +217,6 @@ function ciPrimaryGates(): Gate[] {
|
||||
]
|
||||
}
|
||||
|
||||
function ciPrimaryLargeRunnerGates(): Gate[] {
|
||||
// The CPU lane owns typecheck, coverage, and the build-to-snapshot chain.
|
||||
// This core lane starts its own build eagerly for the remaining artifact consumers.
|
||||
return ciPrimaryGates()
|
||||
.filter(gate => !['coverage', 'docs-site-build', 'snapshot', 'typecheck'].includes(gate.id))
|
||||
.map((gate) => {
|
||||
if (gate.id !== 'build') return gate
|
||||
const eagerBuild = { ...gate }
|
||||
delete eagerBuild.needs
|
||||
return eagerBuild
|
||||
})
|
||||
}
|
||||
|
||||
function ciPrimaryCpuGates(): Gate[] {
|
||||
// Build and snapshot stay together so the dependent replay consumes this lane's output.
|
||||
return [
|
||||
pnpmScript('typecheck', 'typecheck'),
|
||||
coverageGate(),
|
||||
pnpmScript('build', 'build'),
|
||||
snapshotGate(),
|
||||
...nodeCompatSmokeGates(),
|
||||
]
|
||||
}
|
||||
|
||||
function nodeCompatGates(): Gate[] {
|
||||
return [
|
||||
...flagEnabled('DSH_NODE_COMPAT_SKIP_TYPECHECK') ? [] : [pnpmScript('typecheck', 'typecheck')],
|
||||
@@ -279,7 +240,7 @@ function nodeCompatSmokeGates(): Gate[] {
|
||||
}
|
||||
|
||||
function ciStaticGates(): Gate[] {
|
||||
const gates = [
|
||||
return [
|
||||
pnpmScript('runtime-closure', 'verify-runtime-closure', { label: 'runtime closure' }),
|
||||
pnpmScript('constraints', 'constraints'),
|
||||
pnpmScript('package-invariants', 'verify-package-invariants', { label: 'package invariants' }),
|
||||
@@ -293,12 +254,10 @@ function ciStaticGates(): Gate[] {
|
||||
pnpmScript('module-graph', 'verify-module-graph', { label: 'module graph' }),
|
||||
pnpmScript('knip', 'knip'),
|
||||
]
|
||||
return selectStaticGates(gates, process.env.DSH_STATIC_SHARD)
|
||||
}
|
||||
|
||||
function ciArtifactGates(): Gate[] {
|
||||
const shard = process.env.DSH_ARTIFACT_SHARD
|
||||
const metadataGates = [
|
||||
return [
|
||||
pnpmScript('build', 'build'),
|
||||
pnpmScript('publint', 'publint', { needs: ['build'] }),
|
||||
pnpmScript('node-next-types', 'verify-node-next-types', {
|
||||
@@ -306,13 +265,8 @@ function ciArtifactGates(): Gate[] {
|
||||
needs: ['build'],
|
||||
}),
|
||||
builtPackageInvariantsGate(['build']),
|
||||
builtBinSmokeGate(),
|
||||
]
|
||||
if (shard === 'metadata') return metadataGates
|
||||
if (shard === 'smoke') return [pnpmScript('build', 'build'), builtBinSmokeGate()]
|
||||
if (shard !== undefined && shard !== '') {
|
||||
throw new Error(`run-gates: unknown DSH_ARTIFACT_SHARD ${JSON.stringify(shard)}.`)
|
||||
}
|
||||
return [...metadataGates, builtBinSmokeGate()]
|
||||
}
|
||||
|
||||
function ciWindowsBlockingGates(): Gate[] {
|
||||
@@ -396,12 +350,10 @@ function eslintConcurrencyArgs(): string[] {
|
||||
}
|
||||
|
||||
function coverageGate(): Gate {
|
||||
const shard = process.env.DSH_COVERAGE_SHARD
|
||||
return pnpmExec('coverage', [
|
||||
'vitest',
|
||||
'run',
|
||||
'--coverage',
|
||||
...(shard === undefined || shard === '' ? [] : coverageArgs(shard)),
|
||||
...positiveIntArg('DSH_COVERAGE_MAX_WORKERS', '--maxWorkers'),
|
||||
], {
|
||||
label: 'test:coverage',
|
||||
@@ -409,23 +361,12 @@ function coverageGate(): Gate {
|
||||
}
|
||||
|
||||
// The snapshot suite boots the example bins in `lib` mode (built artifact under plain Node,
|
||||
// plugins via real exports). CI normally pairs it with `build`, so it exercises what ships rather
|
||||
// than the tsx/source path dev uses; callers with prebuilt output may omit that dependency.
|
||||
function snapshotGate(needs: string[] = ['build']): Gate {
|
||||
const lane = selectSnapshotLane(process.env.DSH_SNAPSHOT_LANE)
|
||||
return pnpmExec('snapshot', [
|
||||
'vitest',
|
||||
'run',
|
||||
'--config',
|
||||
'vitest.snapshot.config.ts',
|
||||
...lane.files,
|
||||
], {
|
||||
label: 'test:snapshot',
|
||||
env: {
|
||||
DSH_EXAMPLE_MODE: 'lib',
|
||||
...lane.scenarioShard === undefined ? {} : { DSH_SNAPSHOT_SCENARIO_SHARD: lane.scenarioShard },
|
||||
},
|
||||
...needs.length === 0 ? {} : { needs },
|
||||
// plugins via real exports). CI pairs it with `build`, so it exercises what ships rather than
|
||||
// the tsx/source path dev uses and therefore waits on `build`.
|
||||
function snapshotGate(): Gate {
|
||||
return pnpmScript('snapshot', 'test:snapshot', {
|
||||
env: { DSH_EXAMPLE_MODE: 'lib' },
|
||||
needs: ['build'],
|
||||
})
|
||||
}
|
||||
|
||||
@@ -492,7 +433,7 @@ function docSyncLeafGates(options: {
|
||||
]
|
||||
}
|
||||
|
||||
function builtBinSmokeGate(shard?: string): Gate {
|
||||
function builtBinSmokeGate(): Gate {
|
||||
return pnpmExec('built-bin-smoke', [
|
||||
'vitest',
|
||||
'run',
|
||||
@@ -508,7 +449,6 @@ function builtBinSmokeGate(shard?: string): Gate {
|
||||
// (the e2e lane runs unbuilt, so these files self-skip there).
|
||||
'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts',
|
||||
'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts',
|
||||
...(shard === undefined ? [] : [`--shard=${shard}`]),
|
||||
], {
|
||||
label: 'built-bin smoke',
|
||||
needs: ['build'],
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
import { existsSync, readdirSync } from 'node:fs'
|
||||
import { join, relative, sep } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { selectSnapshotLane, snapshotLanes } from './snapshot-shards.ts'
|
||||
|
||||
const root = join(import.meta.dirname, '..')
|
||||
|
||||
function snapshotFiles(dir: string): string[] {
|
||||
if (!existsSync(dir)) return []
|
||||
return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
|
||||
const path = join(dir, entry.name)
|
||||
if (entry.isDirectory()) return snapshotFiles(path)
|
||||
return entry.name.endsWith('.snapshot.ts') ? [relative(root, path).split(sep).join('/')] : []
|
||||
})
|
||||
}
|
||||
|
||||
describe('snapshot lanes', () => {
|
||||
it('assigns every configured snapshot file and every ACP scenario shard', () => {
|
||||
const discovered = [
|
||||
...snapshotFiles(join(root, 'examples')),
|
||||
...snapshotFiles(join(root, 'packages/sdk')),
|
||||
...snapshotFiles(join(root, 'packages/ui/tui')),
|
||||
].filter(path => !path.includes('/node_modules/') && !path.includes('/lib/')).sort()
|
||||
const ordinary = snapshotLanes.filter(lane => lane.scenarioShard === undefined).flatMap(lane => lane.files)
|
||||
const acp = snapshotLanes.filter(lane => lane.scenarioShard !== undefined)
|
||||
|
||||
expect(new Set(ordinary).size).toBe(ordinary.length)
|
||||
expect(acp.map(lane => lane.files)).toEqual(Array.from(
|
||||
{ length: 8 },
|
||||
() => ['examples/acp-agent/tests/acp.snapshot.ts'],
|
||||
))
|
||||
expect(acp.map(lane => lane.scenarioShard)).toEqual([
|
||||
'1/8',
|
||||
'2/8',
|
||||
'3/8',
|
||||
'4/8',
|
||||
'5/8',
|
||||
'6/8',
|
||||
'7/8',
|
||||
'8/8',
|
||||
])
|
||||
expect([...ordinary, 'examples/acp-agent/tests/acp.snapshot.ts'].sort()).toEqual(discovered)
|
||||
})
|
||||
|
||||
it('keeps ordinary runs complete and selects known lanes', () => {
|
||||
expect(selectSnapshotLane()).toEqual({ name: 'complete', files: [] })
|
||||
expect(selectSnapshotLane('')).toEqual({ name: 'complete', files: [] })
|
||||
for (const lane of snapshotLanes) expect(selectSnapshotLane(lane.name)).toBe(lane)
|
||||
})
|
||||
|
||||
it('rejects an unknown lane', () => {
|
||||
expect(() => selectSnapshotLane('missing')).toThrow('unknown DSH_SNAPSHOT_LANE')
|
||||
})
|
||||
})
|
||||
@@ -1,49 +0,0 @@
|
||||
/** Snapshot-lane definitions for GitHub Actions. */
|
||||
|
||||
/** One explicit snapshot file lane, optionally split again by ACP scenarios. */
|
||||
export interface SnapshotLane {
|
||||
/** Stable lane name passed through `DSH_SNAPSHOT_LANE`. */
|
||||
name: string
|
||||
/** Snapshot test files owned by the lane. */
|
||||
files: readonly string[]
|
||||
/** Optional one-based ACP scenario partition. */
|
||||
scenarioShard?: string
|
||||
}
|
||||
|
||||
/** Exhaustive file ownership plus scenario partitions for the large ACP suite. */
|
||||
export const snapshotLanes: readonly SnapshotLane[] = [
|
||||
{
|
||||
name: 'support',
|
||||
files: [
|
||||
'packages/sdk/scripts/tests/config.snapshot.ts',
|
||||
'packages/sdk/create-sdk/tests/create.snapshot.ts',
|
||||
'packages/ui/tui/tests/tui.snapshot.ts',
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'agents',
|
||||
files: [
|
||||
'examples/tui-agent/tests/tui.snapshot.ts',
|
||||
'examples/acp-agent/tests/goal.snapshot.ts',
|
||||
'examples/headless-agent/tests/headless.snapshot.ts',
|
||||
],
|
||||
},
|
||||
...Array.from({ length: 8 }, (_, offset) => ({
|
||||
name: `acp-${offset + 1}`,
|
||||
files: ['examples/acp-agent/tests/acp.snapshot.ts'],
|
||||
scenarioShard: `${offset + 1}/8`,
|
||||
})),
|
||||
]
|
||||
|
||||
/**
|
||||
* Resolve one CI lane while preserving a complete ordinary snapshot run.
|
||||
*
|
||||
* @param name Optional stable lane name.
|
||||
* @returns An empty file list for the full suite, or one explicit CI lane.
|
||||
*/
|
||||
export function selectSnapshotLane(name?: string): SnapshotLane {
|
||||
if (name === undefined || name === '') return { name: 'complete', files: [] }
|
||||
const lane = snapshotLanes.find(candidate => candidate.name === name)
|
||||
if (lane === undefined) throw new Error(`run-gates: unknown DSH_SNAPSHOT_LANE ${JSON.stringify(name)}.`)
|
||||
return lane
|
||||
}
|
||||
@@ -1,26 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { selectStaticGates, staticShards } from './static-shards.ts'
|
||||
|
||||
const completeInventory = staticShards.flatMap(shard => shard.gateIds).map(id => ({ id }))
|
||||
|
||||
describe('static gate shards', () => {
|
||||
it.each(staticShards)('selects only the gates owned by $name', (shard) => {
|
||||
expect(selectStaticGates(completeInventory, shard.name).map(gate => gate.id)).toEqual(shard.gateIds)
|
||||
})
|
||||
|
||||
it('selects multiple lanes in gate inventory order', () => {
|
||||
const selectedNames = new Set(['foundation', 'catalogs', 'prose'])
|
||||
const expected = staticShards
|
||||
.filter(shard => selectedNames.has(shard.name))
|
||||
.flatMap(shard => shard.gateIds)
|
||||
expect(selectStaticGates(completeInventory, 'foundation,catalogs,prose').map(gate => gate.id)).toEqual(expected)
|
||||
})
|
||||
|
||||
it('rejects missing, duplicate, and unknown assignments', () => {
|
||||
expect(() => selectStaticGates(completeInventory.slice(1))).toThrow('assign every static gate exactly once')
|
||||
expect(() => selectStaticGates([...completeInventory, completeInventory[0]!])).toThrow('static gate IDs must be unique')
|
||||
expect(() => selectStaticGates(completeInventory, 'missing')).toThrow('unknown DSH_STATIC_SHARD')
|
||||
expect(() => selectStaticGates(completeInventory, 'foundation,foundation')).toThrow('nonempty and unique')
|
||||
expect(() => selectStaticGates(completeInventory, 'foundation,,prose')).toThrow('nonempty and unique')
|
||||
})
|
||||
})
|
||||
@@ -1,86 +0,0 @@
|
||||
/** Static-gate shard definitions for GitHub Actions. */
|
||||
|
||||
/** A static CI lane identified by the gate IDs it owns. */
|
||||
export interface StaticShard {
|
||||
/** Stable lane identifier passed through `DSH_STATIC_SHARD`. */
|
||||
name: string
|
||||
/** Gate IDs selected from the static gate inventory. */
|
||||
gateIds: readonly string[]
|
||||
}
|
||||
|
||||
/** Exhaustive, non-overlapping ownership of static CI gates. */
|
||||
export const staticShards = [
|
||||
{
|
||||
name: 'foundation',
|
||||
gateIds: [
|
||||
'runtime-closure',
|
||||
'constraints',
|
||||
'package-invariants',
|
||||
'cordis-config',
|
||||
'module-graph',
|
||||
'knip',
|
||||
],
|
||||
},
|
||||
{ name: 'doc-types', gateIds: ['build', 'doc-typecheck'] },
|
||||
{
|
||||
name: 'api-contracts',
|
||||
gateIds: ['cordis-api', 'export-jsdoc', 'scoped-events', 'type-equivalence'],
|
||||
},
|
||||
{
|
||||
name: 'catalogs',
|
||||
gateIds: ['cordis-catalog', 'tool-catalog', 'config-catalog', 'persistence-catalog', 'doc-graphs'],
|
||||
},
|
||||
{
|
||||
name: 'prose',
|
||||
gateIds: [
|
||||
'markdown-wrap',
|
||||
'markdown-links',
|
||||
'doc-refs',
|
||||
'package-paths',
|
||||
'package-readme-model-experience',
|
||||
'mermaid',
|
||||
'agent-note-classification',
|
||||
'agent-note-format',
|
||||
'translation-prompt',
|
||||
'translation-pairing',
|
||||
'doc-budgets',
|
||||
'package-readme-limitations',
|
||||
],
|
||||
},
|
||||
{ name: 'site-projection', gateIds: ['docs-site-projection'] },
|
||||
{ name: 'site-build', gateIds: ['docs-site-build'] },
|
||||
] as const satisfies readonly StaticShard[]
|
||||
|
||||
/**
|
||||
* Validate the complete gate partition and optionally select one lane.
|
||||
*
|
||||
* @param gates Complete static gate inventory.
|
||||
* @param name Optional comma-separated stable shard names.
|
||||
* @returns All gates when no shard is requested, otherwise the selected lanes in inventory order.
|
||||
*/
|
||||
export function selectStaticGates<T extends { id: string }>(gates: readonly T[], name?: string): T[] {
|
||||
const gateIds = gates.map(gate => gate.id)
|
||||
const assignedIds = staticShards.flatMap(shard => shard.gateIds)
|
||||
const uniqueGateIds = new Set<string>(gateIds)
|
||||
const uniqueAssignedIds = new Set<string>(assignedIds)
|
||||
if (uniqueGateIds.size !== gateIds.length) throw new Error('run-gates: static gate IDs must be unique.')
|
||||
if (uniqueAssignedIds.size !== assignedIds.length) throw new Error('run-gates: static shard gate IDs must be unique.')
|
||||
if (gateIds.length !== assignedIds.length
|
||||
|| gateIds.some(id => !uniqueAssignedIds.has(id))
|
||||
|| assignedIds.some(id => !uniqueGateIds.has(id))) {
|
||||
throw new Error('run-gates: static shards must assign every static gate exactly once.')
|
||||
}
|
||||
if (name === undefined || name === '') return [...gates]
|
||||
|
||||
const shardNames = name.split(',')
|
||||
if (shardNames.some(shardName => shardName === '') || new Set(shardNames).size !== shardNames.length) {
|
||||
throw new Error(`run-gates: DSH_STATIC_SHARD names must be nonempty and unique, got ${JSON.stringify(name)}.`)
|
||||
}
|
||||
const selectedShards = shardNames.map((shardName) => {
|
||||
const shard = staticShards.find(candidate => candidate.name === shardName)
|
||||
if (shard === undefined) throw new Error(`run-gates: unknown DSH_STATIC_SHARD ${JSON.stringify(shardName)}.`)
|
||||
return shard
|
||||
})
|
||||
const selectedIds = new Set<string>(selectedShards.flatMap(shard => shard.gateIds))
|
||||
return gates.filter(gate => selectedIds.has(gate.id))
|
||||
}
|
||||
Reference in New Issue
Block a user