chore(gates): route doc-sync through the run-gates scheduler
pnpm run doc-sync was a serial &&-chain of 24 pnpm run subcommands; the 24 script bodies finish in ~34s while the chain takes ~3min, all wrapper overhead. Add a doc-sync mode to scripts/run-gates.ts expanding to docSyncLeafGates() (concurrency capped at 4 like pre-push: several doc gates each build a full ts.Program) and point the package script at it, making the leaf list the single source of truth for the member set. Consolidating the two lists surfaced drift: verify-cordis-api joined the package.json chain with the runtime API catalog but was never added to docSyncLeafGates, so CI and pre-push never gated that catalog's freshness. Add the missing leaf.
This commit is contained in:
Chinesezjc
@@ -20,7 +20,7 @@ The build gate makes the hook self-contained from a clean worktree. `publint`, `
|
||||
|
||||
[scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages/<group>/<pkg>` and runs `publint` with a worker pool sized from `availableParallelism()`. `DSH_PUBLINT_CONCURRENCY` can cap or raise the worker count for local machines and CI runners with different resource profiles. Results are buffered per package and printed in deterministic package order, so parallel execution does not scramble each package's log block.
|
||||
|
||||
The aggregate package scripts remain the source of truth for ad hoc local runs. The scheduler is a parallel execution plan over their member gates, not a replacement vocabulary.
|
||||
The per-gate package scripts remain the vocabulary for ad hoc local runs. `hygiene` stays an aggregate `&&` chain the scheduler mirrors, while `doc-sync` has since moved its member list into the scheduler itself ([doc-sync through the gate scheduler](2026-07-21-doc-sync-through-gate-scheduler.md)).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-21-doc-sync-through-gate-scheduler.md: b79df2dd7d3515cb0434ac672f7f87c3271d900b
|
||||
2026-07-21-doc-sync-through-gate-scheduler.zh.md: 9395244e3c7700166ad87c49219073210c66bc7e
|
||||
@@ -0,0 +1,25 @@
|
||||
# Agent Note: doc-sync through the gate scheduler
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-21-doc-sync-through-gate-scheduler.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`pnpm run doc-sync` was a `&&` chain of 24 `pnpm run` subcommands. Each link paid a full pnpm wrapper start (workspace resolution, script lookup, tsx boot) before its script ran; measured on a development host, the 24 script bodies together finish in about 34 seconds while the chained form takes around 3 minutes, and the wrapper stall reproduces on local disk, so every developer and CI lane pays it, not just network-filesystem checkouts. The chain also ran serially even though the member gates are read-only and independent, and it silently drifted from [scripts/run-gates.ts](../../../../scripts/run-gates.ts): `verify-cordis-api` joined the chain when the runtime API catalog landed but was never added to `docSyncLeafGates`, so CI and pre-push never enforced that catalog's freshness.
|
||||
|
||||
## Decision
|
||||
|
||||
`doc-sync` in `package.json` now delegates to the existing bounded scheduler — `tsx scripts/run-gates.ts doc-sync` — the same way `check:pre-push` and the `check:ci:*` scripts already do ([parallel pre-push gates](2026-07-06-parallel-pre-push-gates.md), [parallel GitHub CI gates](2026-07-06-parallel-github-ci-gates.md)). The new `doc-sync` mode expands to exactly `docSyncLeafGates()`, making the leaf list in `run-gates.ts` the single source of truth for the member set; the chain that could drift from it is gone. Like `pre-push`, the mode caps default concurrency at four workers because several doc gates each build a full `ts.Program`; `DSH_GATE_CONCURRENCY` still overrides.
|
||||
|
||||
The drift this consolidation surfaced is fixed in the same change: `docSyncLeafGates` gains the missing `verify-cordis-api` leaf, so CI and pre-push now gate the generated runtime API catalog alongside the other generated docs.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the `&&` chain and only fix the missing leaf** — repairs today's drift but keeps two member lists that will drift again, and keeps the 24 serial pnpm wrapper starts.
|
||||
- **A dedicated `scripts/doc-sync.ts` importing each verify module in one process** — saves even the per-gate tsx boot, but requires refactoring all 24 scripts from run-at-import to callable entry points and loses the scheduler's per-gate timing, isolation, and failure grouping; the wrapper start the scheduler already avoids is the dominant cost.
|
||||
- **Shell loop over `tsx scripts/*.ts`** — avoids pnpm wrapper starts cheaply but adds a second execution vocabulary next to the scheduler CI already uses, with none of its scheduling or reporting.
|
||||
|
||||
## Consequences
|
||||
|
||||
One `pnpm run doc-sync` now costs one pnpm wrapper start plus the slowest dependency chain of member gates instead of 24 wrapper starts plus the sum of all members. Adding a doc gate is one edit in `docSyncLeafGates` (plus the package script itself for ad hoc runs); `package.json` keeps the per-gate `verify-*` scripts as the vocabulary for running one gate by hand. The scheduler prints per-gate timing, so a slow doc-sync points at the gate that dominates. `pnpm run doc-sync` output is now interleaved scheduler output rather than sequential per-command output; anything parsing that output must key on the `run-gates:` summary lines.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Agent Note: doc-sync 走门禁调度器
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-21-doc-sync-through-gate-scheduler.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`pnpm run doc-sync` 原本是把 24 个 `pnpm run` 子命令用 `&&` 串起来的链。每一环都要先付一次完整的 pnpm 包装层启动（workspace 解析、脚本查找、tsx 启动）才轮到脚本本体；在开发机上实测，24 个脚本本体合计约 34 秒即可跑完，而链式形态耗时约 3 分钟，且包装层的停顿在本地磁盘上同样复现，因此每位开发者和每条 CI 车道都在付这笔开销，并非只有网络文件系统上的检出受影响。这条链还是串行执行的，尽管各成员门禁只读且相互独立；它也在悄悄偏离 [scripts/run-gates.ts](../../../../scripts/run-gates.ts)：运行时 API 目录落地时 `verify-cordis-api` 加入了链，却从未加进 `docSyncLeafGates`，导致 CI 和 pre-push 从未把关该目录的新鲜度。
|
||||
|
||||
## 决策
|
||||
|
||||
`package.json` 中的 `doc-sync` 现在委托给既有的有界调度器——`tsx scripts/run-gates.ts doc-sync`——与 `check:pre-push` 和各 `check:ci:*` 脚本的做法一致（[并行 pre-push 门禁](2026-07-06-parallel-pre-push-gates.md)、[并行 GitHub CI 门禁](2026-07-06-parallel-github-ci-gates.md)）。新增的 `doc-sync` 模式恰好展开为 `docSyncLeafGates()`，使 `run-gates.ts` 里的叶子列表成为成员集合的唯一真源；那条可能与之漂移的链不复存在。与 `pre-push` 一样，该模式把默认并发上限设为四个 worker，因为多个文档门禁各自要构建完整的 `ts.Program`；`DSH_GATE_CONCURRENCY` 仍可覆盖。
|
||||
|
||||
这次整合暴露出的漂移在同一变更中修复：`docSyncLeafGates` 补上缺失的 `verify-cordis-api` 叶子，CI 和 pre-push 从此与其他生成文档一起把关生成的运行时 API 目录。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **保留 `&&` 链，只补缺失的叶子**——能修好今天的漂移，但保留了两份还会再漂移的成员列表，也保留了 24 次串行的 pnpm 包装层启动。
|
||||
- **专门的 `scripts/doc-sync.ts` 在单进程内 import 各校验模块**——连每个门禁的 tsx 启动也能省掉，但需要把全部 24 个脚本从 import 即执行改造成可调用入口，还会失去调度器的按门禁计时、隔离和失败分组；而调度器已经避免的包装层启动才是开销的大头。
|
||||
- **用 shell 循环跑 `tsx scripts/*.ts`**——以低成本避开 pnpm 包装层启动，却在 CI 已经使用的调度器旁边增加了第二套执行词汇，且没有它的任何调度与报告能力。
|
||||
|
||||
## 结果
|
||||
|
||||
一次 `pnpm run doc-sync` 的成本从 24 次包装层启动加全部成员之和，变为一次包装层启动加成员门禁中最慢的依赖链。新增文档门禁只需在 `docSyncLeafGates` 改一处（外加 package script 本身以便手工单独运行）；`package.json` 保留各 `verify-*` 脚本作为手工运行单个门禁的词汇。调度器输出按门禁计时，doc-sync 变慢时能直接指向占大头的门禁。`pnpm run doc-sync` 的输出从逐命令顺序输出变为调度器的交错输出；解析该输出的工具必须以 `run-gates:` 摘要行为准。
|
||||
@@ -56,7 +56,7 @@ pnpm run lint
|
||||
pnpm run duplication # cross-file TypeScript clone detection
|
||||
pnpm run build # tsc emits lib/types, tsdown bundles runtime
|
||||
pnpm run hygiene # knip + publint + workspace constraints + NodeNext consumer check
|
||||
pnpm run doc-sync # all documentation gates; see the doc-sync script in package.json
|
||||
pnpm run doc-sync # all documentation gates; see the doc-sync leaf list in scripts/run-gates.ts
|
||||
pnpm run website:build # VitePress build (doubles as the site's dead-link check)
|
||||
pnpm run demo:echo # mock-model REPL, no key needed
|
||||
pnpm run demo:repl # real REPL coding agent (needs DEEPSEEK_API_KEY)
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
development.md: 94eb4f03329b574862a1ac1de2f8c1d4db4f4a0a
|
||||
development.zh.md: b533aff43a66ff7cfc5dc61e5b9b224a01c51f12
|
||||
development.md: 1b8d94d281910277764be0e2693e54b537112e9a
|
||||
development.zh.md: 396252e5cd8350f1cf7f987a1e9a943ed3145286
|
||||
+1
-1
@@ -90,7 +90,7 @@ pnpm run verify-md-wrap # fail on hard-wrapped prose paragraphs in docs/README
|
||||
pnpm run verify-mermaid # fail if a ```mermaid diagram has invalid Mermaid syntax
|
||||
pnpm run verify-type-equiv # fail if a ```ts type-equiv doc block drifts from its source type
|
||||
pnpm run verify-doc-budgets # fail if a budgeted standing doc exceeds its word ceiling
|
||||
pnpm run doc-sync # all Markdown/doc gates; see the doc-sync script in package.json for the full list
|
||||
pnpm run doc-sync # all Markdown/doc gates, scheduled concurrently; the doc-sync leaf list in scripts/run-gates.ts is the full list
|
||||
pnpm run gen-module-graph # regenerate docs/module-graph.md from package peerDeps
|
||||
pnpm run verify-module-graph # fail if docs/module-graph.md is stale
|
||||
pnpm run build # emit lib/types intermediates, then bundle lib/index.* runtime files
|
||||
|
||||
@@ -90,7 +90,7 @@ pnpm run verify-md-wrap # fail on hard-wrapped prose paragraphs in docs/README
|
||||
pnpm run verify-mermaid # fail if a ```mermaid diagram has invalid Mermaid syntax
|
||||
pnpm run verify-type-equiv # fail if a ```ts type-equiv doc block drifts from its source type
|
||||
pnpm run verify-doc-budgets # fail if a budgeted standing doc exceeds its word ceiling
|
||||
pnpm run doc-sync # all Markdown/doc gates; see the doc-sync script in package.json for the full list
|
||||
pnpm run doc-sync # all Markdown/doc gates, scheduled concurrently; the doc-sync leaf list in scripts/run-gates.ts is the full list
|
||||
pnpm run gen-module-graph # regenerate docs/module-graph.md from package peerDeps
|
||||
pnpm run verify-module-graph # fail if docs/module-graph.md is stale
|
||||
pnpm run build # emit lib/types intermediates, then bundle lib/index.* runtime files
|
||||
|
||||
+1
-1
@@ -75,7 +75,7 @@
|
||||
"website:dev": "pnpm --filter @deepseek-ai/website run dev",
|
||||
"website:build": "pnpm --filter @deepseek-ai/website run build",
|
||||
"constraints": "tsx scripts/check-workspace-constraints.ts",
|
||||
"doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-cordis-api && pnpm run verify-export-jsdoc && pnpm run verify-tool-catalog && pnpm run verify-config-catalog && pnpm run verify-persistence-catalog && pnpm run verify-doc-graphs && pnpm run verify-scoped-events && pnpm run verify-website-api && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-package-readme-model-experience && pnpm run verify-mermaid && pnpm run verify-agent-note-classification && pnpm run verify-agent-note-format && pnpm run verify-type-equiv && pnpm run verify-translation-prompt && pnpm run verify-translation-pairing && pnpm run verify-doc-budgets && pnpm run verify-package-readme-limitations && pnpm run verify-website-yaml",
|
||||
"doc-sync": "tsx scripts/run-gates.ts doc-sync",
|
||||
"hygiene": "pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-cordis-config && pnpm run verify-node-next-types && pnpm run verify-runtime-closure",
|
||||
"demo:echo": "node --expose-internals --import tsx packages/examples/stdio-demo/src/bin.ts examples/echo-agent/cordis.yml",
|
||||
"demo:repl": "node --expose-internals --import tsx packages/examples/stdio-demo/src/bin.ts examples/repl-agent/cordis.yml",
|
||||
|
||||
+12
-4
@@ -19,6 +19,7 @@ type Mode =
|
||||
| 'ci-artifacts'
|
||||
| 'node-compat'
|
||||
| 'pre-push'
|
||||
| 'doc-sync'
|
||||
type GateStatus = 'pending' | 'running' | 'passed' | 'failed' | 'skipped'
|
||||
|
||||
interface Gate {
|
||||
@@ -88,21 +89,25 @@ function parseMode(raw: string | undefined): Mode {
|
||||
case 'ci-artifacts':
|
||||
case 'node-compat':
|
||||
case 'pre-push':
|
||||
case 'doc-sync':
|
||||
return raw
|
||||
default:
|
||||
throw new Error(
|
||||
`run-gates: expected mode ci-primary | ci-static | ci-lint | ci-coverage | ci-snapshot | ci-artifacts | node-compat | pre-push, got ${JSON.stringify(raw)}.`,
|
||||
`run-gates: expected mode ci-primary | ci-static | ci-lint | ci-coverage | ci-snapshot | ci-artifacts | node-compat | pre-push | doc-sync, got ${JSON.stringify(raw)}.`,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
function defaultConcurrency(selectedMode: Mode, total: number): ConcurrencyDefault {
|
||||
const available = availableParallelism()
|
||||
const modeLimit = selectedMode === 'pre-push' ? Math.min(4, available) : available
|
||||
// Local modes cap workers: several doc gates each build a full ts.Program,
|
||||
// so an uncapped default on a large host trades wall clock for memory blowups.
|
||||
const localCap = selectedMode === 'pre-push' || selectedMode === 'doc-sync'
|
||||
const modeLimit = localCap ? Math.min(4, available) : available
|
||||
return {
|
||||
workers: Math.min(total, modeLimit),
|
||||
source: selectedMode === 'pre-push'
|
||||
? `${available} available CPU(s), pre-push cap 4`
|
||||
source: localCap
|
||||
? `${available} available CPU(s), ${selectedMode} cap 4`
|
||||
: `${available} available CPU(s)`,
|
||||
}
|
||||
}
|
||||
@@ -197,6 +202,8 @@ function gatesForMode(selected: Mode): Gate[] {
|
||||
}),
|
||||
pnpmScript('module-graph', 'verify-module-graph', { label: 'module graph' }),
|
||||
]
|
||||
case 'doc-sync':
|
||||
return docSyncLeafGates()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -331,6 +338,7 @@ function docSyncLeafGates(options: {
|
||||
return [
|
||||
pnpmScript('doc-typecheck', 'doc-typecheck', docTypecheckOptions),
|
||||
pnpmScript('cordis-catalog', 'verify-cordis-catalog', { label: 'cordis catalog' }),
|
||||
pnpmScript('cordis-api', 'verify-cordis-api', { label: 'cordis api' }),
|
||||
pnpmScript('export-jsdoc', 'verify-export-jsdoc', { label: 'export jsdoc' }),
|
||||
pnpmScript('tool-catalog', 'verify-tool-catalog', { label: 'tool catalog' }),
|
||||
pnpmScript('config-catalog', 'verify-config-catalog', { label: 'config catalog' }),
|
||||
|
||||
Reference in New Issue
Block a user