Merge remote-tracking branch 'origin/master' into codex/cli-one-shot-demo
# Conflicts: # AGENTS.md # README.i18n.yaml # README.md # README.zh.md # docs/architecture.md # docs/cookbook/extension-cookbook.i18n.yaml # docs/cookbook/extension-cookbook.md # docs/cookbook/extension-cookbook.zh.md # docs/event-producer-consumer.md # docs/graph-atlas.md # docs/module-graph.md # docs/testing.md # examples/README.md # examples/coding-agent/README.md # package.json # packages/examples/README.md # scripts/gen-doc-graphs.ts
This commit is contained in:
+2
-2
@@ -7,7 +7,7 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning
|
||||
- **Unit** (`pnpm run test`): vitest over `packages|examples/*/tests/**/*.spec.ts`, colocated with what they test. Every registry gets an HMR-safety test (dispose the contributing fiber, assert cleanup). Prefer edge cases, error paths, event ordering, concurrency races, and permanent contract regressions (see `packages/core/agent-loop/tests/contract-regressions.spec.ts`).
|
||||
- **Coverage gate** (`pnpm run test:coverage`): the gating run, per-file 100% on `packages/*/*/src`. An uncovered line is often dead code the gate is correctly flagging for deletion, not a missing test to bolt on. Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped.
|
||||
- **Real-API e2e** (`pnpm run test:e2e`): with-key tests against live provider APIs — the DeepSeek model plus provider-specific smokes that gate on their own keys (`EXA_API_KEY`, `PERPLEXITY_API_KEY`, …); each suite self-skips without its key so keyless CI stays green ([real-API e2e RFC](rfc/implemented/testing/2026-06-19-real-api-e2e-ci.md)).
|
||||
- **Snapshot** (`pnpm run test:snapshot`): real example subprocesses replay recorded model sessions keylessly and compare normalized stdout plus re-persisted logs ([snapshot RFC](rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md)). `examples/acp-agent` pins ACP JSON-RPC, while `examples/headless-agent` independently pins its `stream-json` event surface. Use `pnpm run test:snapshot:record` when an ACP model transcript changes and `pnpm run test:snapshot:refresh` when replay outputs change; review the golden diff. One ACP scenario per header class pins system-prompt/tool-schema content; other ACP fixtures tokenize it ([pinned-header RFC](rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
|
||||
- **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless goldens cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized stdout plus the re-persisted log ([ACP snapshot RFC](rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md)); `examples/headless-agent` independently pins its `stream-json` event surface through the real one-shot subprocess. TUI completed journeys replay recorded primary/child JSONL through the real agent loop and tools before projecting ANSI into semantic terminal-state goldens; package-local snapshots retain transient renderer states, and a real PTY conversation covers the process boundary ([TUI snapshot RFC](rfc/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when committed replay input remains correct; review every JSONL and golden diff. System-prompt/tool-schema content is pinned by ONE ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header RFC](rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
|
||||
|
||||
## The with-key policy: inference is cheap here
|
||||
|
||||
@@ -35,4 +35,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
|
||||
|
||||
## When a snapshot test is required
|
||||
|
||||
A change affecting an editor transcript, headless event stream, or agent UX adds or updates the owning `examples/<name>/tests/snapshots/` scenario, or explains its omission in the PR. `examples/acp-agent` owns the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) table; `examples/headless-agent` owns the `stream-json` snapshot and its replay fixtures. Plans for new capability seams, lifecycle shapes, or transcript surfaces identify every test tier and any required harness work before implementation.
|
||||
Any change affecting an editor-facing transcript, headless event stream, or end-to-end agent UX adds or updates a scenario in the owning snapshot suite, or states in the PR why none applies. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise.
|
||||
Reference in New Issue
Block a user