diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 17f689c7e7..21965beb21 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -43,6 +43,14 @@ jobs: - name: Lint run: yarn lint + # Doc-sync gates (RFC 006). doc-typecheck compiles the fenced ts blocks in + # the docs and resolves vendor packages via their built declarations, which + # the typecheck step above emits — so it runs after typecheck. The event + # taxonomy check only reads source. Same `doc-sync` script the pre-push + # hook runs (ADR 0007: one source of truth). + - name: Doc-sync gates (doc code blocks + event taxonomy) + run: yarn doc-sync + - name: Tests with coverage gate (per-file 100%) run: yarn test:coverage diff --git a/.gitignore b/.gitignore index 414bf3032e..2dbe0baea5 100644 --- a/.gitignore +++ b/.gitignore @@ -7,5 +7,6 @@ lib/ yarn-error.log examples/*/*.jsonl coverage/ +.doc-typecheck-*/ .vscode/ .DS_Store diff --git a/docs/adr/0014-doc-sync-enforcement.md b/docs/adr/0014-doc-sync-enforcement.md new file mode 100644 index 0000000000..9a6596c46b --- /dev/null +++ b/docs/adr/0014-doc-sync-enforcement.md @@ -0,0 +1,23 @@ +# ADR 0014: Doc-sync enforcement + +Status: accepted (2026-06-14) + +## Context + +AGENTS.md promises that docs and code stay strictly in sync, but the promise was verified by eyeball. Review caught drift twice — a cookbook example contradicting the type policy, and a README citing the wrong `registerAdapter` call. Out-of-sync docs are worse than no docs, and this codebase is built primarily by agents that follow gates far more reliably than prose (ADR 0007). Two classes of doc drift are mechanically checkable: code blocks that no longer compile, and the event-taxonomy table that duplicates the `interface Events` declarations. + +## Decision + +Two gates, mirroring the existing `scripts/` style (tsx ESM, one job each): + +1. **`doc-typecheck`** extracts every fenced ` ```ts ` block from `README.md`, `docs/**`, and `packages/*/README.md`, writes them to a temp project, and compiles with `tsc --noEmit`. The temp tsconfig copies only resolution-relevant options and the workspace `paths` map from `tsconfig.typecheck.json` (vendor → built `lib`, harness → `src`) — resolving vendor to `lib` is essential, or tsc type-checks raw vendor source and floods the run. A block that is a deliberate sketch opts out with an explicit ` ```ts ignore-check ` info string; the script reports the opt-out ratio and fails if it exceeds half, so the escape hatch can't quietly become the norm. +2. **`verify-event-taxonomy`** extracts the event names from the `interface Events` blocks across `packages/*/src` and from the taxonomy table in `docs/architecture.md`, and asserts the two sets match exactly. Verify, don't generate: the table keeps its hand-written Mode/Purpose columns; only the set of names is checked. (Landing this surfaced three events the table had been missing — `tools/change`, `llm/adapter-change`, `system-prompt/change`.) + +Both run via a shared `doc-sync` package.json script that the lefthook pre-push hook and CI both invoke (ADR 0007: hooks and CI call the same scripts, so the gate fires locally before a push — not only after it). They run after `yarn typecheck` (which emits the vendor `lib/` that doc-typecheck resolves against). API-extractor golden reports (RFC 006 part 3) were deliberately **deferred** — low value for an internal monorepo where reviewers already see the source diff, and a heavy, finicky dependency. + +## Consequences + +- Doc drift in the two checkable classes now fails the pre-push hook and CI instead of waiting for a reviewer to notice. This is an instance of ADR 0007's "mechanical gates over prose." +- Making doc snippets compile costs a few stub imports/`declare`s; the `ignore-check` ratio must stay low or the gate is theater (the ratio guard enforces this). +- The taxonomy check is name-only — a wrong Mode or Purpose column still needs human review. Generating the table from source was considered and rejected as more machinery than the problem warrants. +- API reports remain available to revisit if the packages are ever published externally. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1842301e9e..eef1bfe957 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -25,3 +25,4 @@ Do NOT write an ADR for: a mechanical or local choice (a variable name, a one-fi | [0011](0011-runtime-arg-validation.md) | Runtime arg validation at the model boundary | accepted | | [0012](0012-dev-invariants-over-deep-readonly.md) | Dev-mode invariants over compile-time deep-readonly | accepted | | [0013](0013-property-based-testing.md) | Property-based testing for protocol-shaped code | accepted | +| [0014](0014-doc-sync-enforcement.md) | Doc-sync enforcement (doc code blocks + event taxonomy) | accepted | diff --git a/docs/architecture.md b/docs/architecture.md index e3f6594a78..1b7e2a7be1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -164,7 +164,7 @@ Error containment: a throwing `agent/turn-continuation` listener or a rejecting ### Event taxonomy -Declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package). +The `agent/*` events are declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package); each other service declares its own events (`tools/*`, `llm/*`, `system-prompt/*`, `session/*`). The table below is CI-verified against the `interface Events` declarations in source (`scripts/verify-event-taxonomy.ts`). | Event | Mode | Purpose | |---|---|---| @@ -177,8 +177,11 @@ Declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package). | `agent/turn-continuation` | **waterfall** | override the continue/stop decision | | `agent/error` | emit | step/turn errors | | `tools/execute` (dsh-tools) | **waterfall** | wrap/veto/sandbox tool execution | +| `tools/change` (dsh-tools) | emit | a tool was registered/unregistered | | `llm/stream` / `llm/generate` (dsh-llm) | **waterfall** | model-call interception | +| `llm/adapter-change` (dsh-llm) | emit | an adapter was registered/unregistered | | `system-prompt/assemble` (dsh-system-prompt) | **waterfall** | mutate the assembly | +| `system-prompt/change` (dsh-system-prompt) | emit | a section/tool-provider changed | | `session/created` / `session/event` (dsh-session) | emit | session lifecycle + log feed | | `session/flush` (dsh-session) | parallel (awaited) | durability checkpoint | diff --git a/docs/cookbook/adding-a-tool.md b/docs/cookbook/adding-a-tool.md index 431e473df0..905fde960a 100644 --- a/docs/cookbook/adding-a-tool.md +++ b/docs/cookbook/adding-a-tool.md @@ -5,6 +5,7 @@ How to give the model a new capability. Reference implementations: `examples/ech ## The minimal shape ```ts +import { readFile } from 'node:fs/promises' import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' diff --git a/docs/cookbook/adding-an-llm-adapter.md b/docs/cookbook/adding-an-llm-adapter.md index a110e751b4..a3ba821b0e 100644 --- a/docs/cookbook/adding-an-llm-adapter.md +++ b/docs/cookbook/adding-an-llm-adapter.md @@ -4,7 +4,7 @@ How to connect a new model provider. Reference implementations: `packages/llm-de ## The shape -```ts +```ts ignore-check class MyAdapter extends LlmAdapter { async * stream(options: GenerateOptions): AsyncIterable { … } } diff --git a/docs/cookbook/extension-cookbook.md b/docs/cookbook/extension-cookbook.md index fcf082f690..d8d9b3e40d 100644 --- a/docs/cookbook/extension-cookbook.md +++ b/docs/cookbook/extension-cookbook.md @@ -11,6 +11,11 @@ A tool registers on `ctx.tools`. The annotated `defineTool` example (typed `exec A hook wraps the `tools/execute` waterfall to veto or rewrite a call — the seam where sandbox, permission, and plan-mode plugins live. ```ts +import type { Context } from 'cordis' +import type { ToolExecution } from '@deepseek-ai/dsh-tools' + +declare function isAllowed(exec: ToolExecution): Promise + export const name = 'permission-gate' export function apply(ctx: Context) { @@ -32,6 +37,11 @@ export function apply(ctx: Context) { A UI plugin consumes `agent/stream-chunk` and session events for rendering, and drives input back in via `agent.send()` / `agent.steer()`. ```ts +import type { Context } from 'cordis' + +declare function render(text: string): void +declare function onUserInput(handler: (text: string) => void): void + export const name = 'my-ui' export const inject = ['agents'] diff --git a/docs/rfc/006-doc-sync-and-api-reports.md b/docs/rfc/006-doc-sync-and-api-reports.md index 3b59a1f07b..3c2d64d489 100644 --- a/docs/rfc/006-doc-sync-and-api-reports.md +++ b/docs/rfc/006-doc-sync-and-api-reports.md @@ -1,6 +1,6 @@ # RFC 006: Doc-sync enforcement and API reports -Status: proposed +Status: implemented (parts 1-2) — see [ADR 0014](../adr/0014-doc-sync-enforcement.md). Part 3 (API reports) deferred. ## Problem diff --git a/docs/rfc/README.md b/docs/rfc/README.md index fbbe442d31..c763e09024 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -9,6 +9,6 @@ Proposals for substantial future work — reviewed before implementation, unlike | [003](003-deterministic-and-stress-testing.md) | Deterministic tests + replay invariant fixture + race stress | proposed | | [004](004-architectural-conformance.md) | Architectural rules: dependency-cruiser, adapter conformance kit | proposed | | [005](005-runtime-validation-and-error-taxonomy.md) | Runtime arg validation, structured error taxonomy, dev-mode invariants | partially implemented | -| [006](006-doc-sync-and-api-reports.md) | Doc-sync enforcement and API extractor reports | proposed | +| [006](006-doc-sync-and-api-reports.md) | Doc-sync enforcement and API extractor reports | implemented (pts 1-2; pt 3 deferred) | | [007](007-supply-chain-and-vendor-drift.md) | Supply chain checks and vendor drift verification | proposed | | [008](008-immutable-public-surfaces.md) | Deep-readonly public surfaces | implemented (revised) | diff --git a/lefthook.yml b/lefthook.yml index 7d2d9985e8..6f29bb0990 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -27,3 +27,6 @@ pre-push: - name: hygiene run: yarn hygiene + + - name: doc-sync + run: yarn doc-sync diff --git a/package.json b/package.json index 4f752ac364..c56ba924e5 100644 --- a/package.json +++ b/package.json @@ -21,6 +21,9 @@ "test:e2e": "vitest run --config vitest.e2e.config.ts", "knip": "knip", "publint": "tsx scripts/publint-all.ts", + "doc-typecheck": "tsx scripts/doc-typecheck.ts", + "verify-event-taxonomy": "tsx scripts/verify-event-taxonomy.ts", + "doc-sync": "yarn doc-typecheck && yarn verify-event-taxonomy", "hygiene": "yarn knip && yarn publint && yarn constraints", "demo:echo": "node --expose-internals --import tsx examples/echo-agent/start.ts", "demo:coding": "node --expose-internals --import tsx examples/coding-agent/start.ts", diff --git a/packages/agent-loop/README.md b/packages/agent-loop/README.md index e68698612f..49176701c5 100644 --- a/packages/agent-loop/README.md +++ b/packages/agent-loop/README.md @@ -17,7 +17,7 @@ This is the only package in the harness that contains concrete loop logic. Every ### Configuration (schemastery) ```ts -Config: { +interface Config { agents: Array<{ id: string // required model?: string diff --git a/packages/invariants/README.md b/packages/invariants/README.md index 3e7337fe11..ffbfe1cd43 100644 --- a/packages/invariants/README.md +++ b/packages/invariants/README.md @@ -9,8 +9,11 @@ Dev-mode event-contract invariants and session-log freeze. A pure-listener plugi A functional plugin — register the module namespace (this is what loading by name in `cordis.yml` does): ```ts +import type { Context } from 'cordis' import * as Invariants from '@deepseek-ai/dsh-invariants' +declare const ctx: Context + await ctx.plugin(Invariants) // freeze on (default) await ctx.plugin(Invariants, { freeze: false }) // assert contract, don't freeze ``` diff --git a/packages/tools/README.md b/packages/tools/README.md index 0f3360ebcb..5d1678512e 100644 --- a/packages/tools/README.md +++ b/packages/tools/README.md @@ -39,8 +39,12 @@ Tool registry and execution waterfall. Tool plugins register their schemas and e First-party plugin authors can use the `defineTool()` helper (exported from this package) for typed tool parameter schemas: ```ts +import { readFile } from 'node:fs/promises' +import type { Context } from 'cordis' import { defineTool } from '@deepseek-ai/dsh-tools' +declare const ctx: Context + ctx.tools.register(defineTool({ name: 'read_file', description: 'Read a file from disk.', diff --git a/scripts/doc-typecheck.ts b/scripts/doc-typecheck.ts new file mode 100644 index 0000000000..cd873402c3 --- /dev/null +++ b/scripts/doc-typecheck.ts @@ -0,0 +1,151 @@ +/** + * Doc-sync gate (RFC 006 part 1): typecheck the fenced `ts` code blocks in our + * Markdown so documentation can't drift from the API it documents. + * + * Every ```ts block in README.md, docs/** and packages/* /README.md is + * extracted to a temp file and compiled with `tsc --noEmit` against the + * workspace sources (resolved through the same `paths` map vitest uses, so no + * build is required first). A block that is a deliberate sketch rather than + * compilable code opts out with an explicit ` ```ts ignore-check ` info string + * — the opt-out is visible in the source, and this script reports the ratio so + * the escape hatch can't quietly become the norm. + * + * Run: `tsx scripts/doc-typecheck.ts`. + */ + +import { execFileSync } from 'node:child_process' +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { join, relative, resolve } from 'node:path' +import { glob } from 'node:fs/promises' + +const root = resolve(import.meta.dirname, '..') + +/** One extracted code block. */ +interface Block { + file: string + /** 1-based line of the opening fence. */ + line: number + /** `true` when the fence is ` ```ts ignore-check ` (skip compilation). */ + ignored: boolean + code: string +} + +/** Extract every ```ts / ```ts ignore-check block from one Markdown file. */ +function extractBlocks(absPath: string): Block[] { + const text = readFileSync(absPath, 'utf8') + const lines = text.split('\n') + const file = relative(root, absPath) + const blocks: Block[] = [] + let open: { line: number; ignored: boolean; body: string[] } | null = null + + lines.forEach((raw, i) => { + const fence = /^```(\s*)(\S.*)?$/.exec(raw) + if (!fence) { + if (open) open.body.push(raw) + return + } + if (open) { + // closing fence + blocks.push({ file, line: open.line, ignored: open.ignored, code: open.body.join('\n') }) + open = null + return + } + // opening fence — only care about ts blocks + const info = (fence[2] ?? '').trim() + if (info === 'ts' || info === 'ts ignore-check') { + open = { line: i + 1, ignored: info === 'ts ignore-check', body: [] } + } + }) + return blocks +} + +/** + * Read the workspace `paths` map from tsconfig.typecheck.json (JSONC). This map + * resolves vendored packages to their BUILT declarations (`lib`) and harness + * packages to source (`src`) — the same resolution `yarn lint`/`typecheck` use. + * Resolving vendor to `lib` (not `src`) is essential: otherwise tsc type-checks + * raw vendor source and floods the run with unrelated errors. Requires the + * vendor `lib/` to exist (a fresh clone runs `yarn build` first; CI does too). + */ +function workspacePaths(): Record { + const raw = readFileSync(join(root, 'tsconfig.typecheck.json'), 'utf8') + // Strip // line comments and /* */ block comments so JSON.parse accepts it. + const stripped = raw + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/(^|[^:])\/\/.*$/gm, '$1') + return (JSON.parse(stripped) as { compilerOptions: { paths: Record } }) + .compilerOptions.paths +} + +/** The standalone tsconfig for the temp project (copies base resolution, no + * composite/declaration settings that would fight `--noEmit`). */ +function tempTsconfig(): string { + return JSON.stringify({ + compilerOptions: { + target: 'es2024', + module: 'esnext', + moduleResolution: 'bundler', + allowImportingTsExtensions: true, + strict: true, + noEmit: true, + skipLibCheck: true, + types: ['node'], + baseUrl: root, + ignoreDeprecations: '6.0', + paths: workspacePaths(), + }, + }) +} + +const markdownGlobs = ['README.md', 'docs/**/*.md', 'packages/*/README.md'] + +const files: string[] = [] +for (const pattern of markdownGlobs) { + for await (const match of glob(pattern, { cwd: root })) files.push(resolve(root, match)) +} +files.sort() + +const all = files.flatMap(extractBlocks) +const checked = all.filter(b => !b.ignored) +const ignored = all.filter(b => b.ignored) + +if (checked.length === 0) { + console.log('doc-typecheck: no ts code blocks to check.') + process.exit(0) +} + +const tmp = mkdtempSync(join(root, '.doc-typecheck-')) +try { + writeFileSync(join(tmp, 'tsconfig.json'), tempTsconfig()) + const fileForBlock = new Map() + checked.forEach((block, i) => { + const name = `block-${i}.ts` + writeFileSync(join(tmp, name), block.code.endsWith('\n') ? block.code : `${block.code}\n`) + fileForBlock.set(name, block) + }) + + try { + execFileSync('node_modules/.bin/tsc', ['-p', join(tmp, 'tsconfig.json')], { cwd: root, stdio: 'pipe' }) + } catch (error: unknown) { + const out = (error as { stdout?: Buffer }).stdout?.toString() ?? '' + // Rewrite "block-N.ts(line,col)" to the real "file:fenceLine" for triage. + const remapped = out.replace(/block-(\d+)\.ts\((\d+),(\d+)\)/g, (_m, idx: string, ln: string, col: string) => { + const block = fileForBlock.get(`block-${idx}.ts`) + if (!block) return `block-${idx}.ts(${ln},${col})` + return `${block.file} (block at line ${block.line}, +${ln}:${col})` + }) + console.error('doc-typecheck: documentation code blocks failed to compile.\n') + console.error(remapped) + process.exit(1) + } + + const ratio = ignored.length / all.length + console.log(`doc-typecheck: ${checked.length} block(s) compiled, ${ignored.length} ignored (${(ratio * 100).toFixed(0)}% opt-out).`) + // Guard against the escape hatch becoming the norm. + if (all.length >= 4 && ratio > 0.5) { + console.error(`doc-typecheck: too many blocks opt out of checking (${ignored.length}/${all.length}). Make them compile or delete them.`) + process.exit(1) + } +} finally { + rmSync(tmp, { recursive: true, force: true }) +} diff --git a/scripts/verify-event-taxonomy.ts b/scripts/verify-event-taxonomy.ts new file mode 100644 index 0000000000..d474efa7d5 --- /dev/null +++ b/scripts/verify-event-taxonomy.ts @@ -0,0 +1,114 @@ +/** + * Doc-sync gate (RFC 006 part 2): verify the event-taxonomy table in + * docs/architecture.md against the events actually declared in source. + * + * The table duplicates the `declare module 'cordis' { interface Events }` + * blocks across packages/* /src. This script extracts both sets of event names + * and asserts they match exactly — every declared event appears in the table, + * and the table names no event that isn't declared. Verify, don't generate + * (per the RFC): the table keeps its hand-written Mode/Purpose columns; only + * the set of names is checked. + * + * Run: `tsx scripts/verify-event-taxonomy.ts`. + */ + +import { readFileSync } from 'node:fs' +import { join, relative, resolve } from 'node:path' +import { glob } from 'node:fs/promises' + +const root = resolve(import.meta.dirname, '..') + +/** + * Remove `/* *​/` block comments and `//` line comments from TS source. Used to + * de-risk the brace walk in {@link declaredEvents} — a JSDoc `{@link}` tag would + * otherwise throw off the `{`/`}` depth counter. Good enough for our own source + * (no string literals contain `//` or comment-like brace sequences in an Events + * block); it is not a general tokenizer. + */ +function stripComments(text: string): string { + return text + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/(^|[^:])\/\/.*$/gm, '$1') +} + +/** + * Event names declared in source: the keys inside every `interface Events` + * block under packages/* /src. A declared event is a quoted `'scope/name'(` + * method signature at the start of a line within such a block. + */ +async function declaredEvents(): Promise> { + const found = new Map() + for await (const match of glob('packages/*/src/**/*.ts', { cwd: root })) { + const abs = resolve(root, match) + // Strip comments first so a JSDoc `{@link …}` tag (or a `// {` line) inside + // an Events block can't unbalance the brace walk below. Event names live in + // code, never in comments, so this loses nothing. + const text = stripComments(readFileSync(abs, 'utf8')) + // Walk `interface Events {` blocks brace-balanced and pull quoted keys. + const re = /interface\s+Events\s*\{/g + let m: RegExpExecArray | null + while ((m = re.exec(text)) !== null) { + let depth = 1 + let i = m.index + m[0].length + const start = i + while (i < text.length && depth > 0) { + const ch = text[i] + if (ch === '{') depth++ + else if (ch === '}') depth-- + i++ + } + const body = text.slice(start, i - 1) + // A declaration is a quoted event name followed by `(` (method form). + for (const k of body.matchAll(/['"]([a-z][a-z-]*\/[a-z-]+)['"]\s*\(/g)) { + const name = k[1] + if (name) found.set(name, relative(root, abs)) + } + } + } + return found +} + +/** Event names referenced in the architecture-doc taxonomy table (in `code`). */ +function tableEvents(): Set { + const text = readFileSync(join(root, 'docs/architecture.md'), 'utf8') + const lines = text.split('\n') + const heading = lines.findIndex(l => /^###\s+Event taxonomy/.test(l)) + if (heading === -1) throw new Error('verify-event-taxonomy: "### Event taxonomy" heading not found') + const names = new Set() + for (let i = heading + 1; i < lines.length; i++) { + const line = lines[i] ?? '' + if (/^###\s/.test(line)) break // next section ends the table + if (!line.includes('|')) continue + for (const code of line.matchAll(/`([^`]+)`/g)) { + // A cell may read "`a/b` / `c/d` (pkg)" — pull each scoped name. + for (const name of (code[1] ?? '').matchAll(/[a-z][a-z-]*\/[a-z-]+/g)) names.add(name[0]) + } + } + return names +} + +const declared = await declaredEvents() +const table = tableEvents() + +const declaredNames = new Set(declared.keys()) +const missingFromTable = [...declaredNames].filter(n => !table.has(n)).sort() +const missingFromSource = [...table].filter(n => !declaredNames.has(n)).sort() + +if (missingFromTable.length === 0 && missingFromSource.length === 0) { + console.log(`verify-event-taxonomy: ${declaredNames.size} events match the architecture-doc table.`) + process.exit(0) +} + +if (missingFromTable.length > 0) { + console.error('verify-event-taxonomy: declared in source but MISSING from the docs/architecture.md table:') + for (const n of missingFromTable) { + console.error(` ${n} (declared in ${declared.get(n) ?? '?'})`) + } +} +if (missingFromSource.length > 0) { + console.error('verify-event-taxonomy: named in the table but NOT declared in source (stale doc):') + for (const n of missingFromSource) { + console.error(` ${n}`) + } +} +process.exit(1)