From a93e9b86e7d51cac88ea10fa15f9378e49ecf7dc Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Mon, 20 Jul 2026 18:42:06 +0800 Subject: [PATCH] docs(testing): require real-example snapshots --- AGENTS.md | 4 ++-- docs/testing.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 87f46c88a0..803ac5888f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -119,9 +119,9 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`, - **Prefer symmetry for parallel values**; unexplained asymmetry usually signals a missed extraction. - **Tests describe behavior, not correctness.** Change obsolete behavior with its tests; explain why in the PR. - **Every non-trivial change MUST include at least one Agent Note in the same PR.** Update the owning note or add one, validate its premises against code, and exempt only mechanical/local edits ([scope](.agents/notes/README.md#when-to-write-one)). -- **Testing policy** — [docs/testing.md](docs/testing.md). Transcript changes need snapshots or a PR note. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. +- **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or human-visible change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and echo-agent fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers. - **A tool's ACP render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)). -- **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces, and schedule any missing harness support before implementation. +- **Plan unit, e2e, and snapshot coverage** for new seams, lifecycle shapes, and transcript surfaces; missing snapshot-harness support is part of the implementation, not deferred follow-up. - **Keep PRs coherent and merge with merge commits.** Split an independently meaningful feature or design decision into a separate or stacked PR when combining it obscures ownership, intent, or verification. Never squash/rebase or rewrite pushed branches; put a review fix on its introducing PR, then merge down the stack ([guide](docs/cookbook/responding-to-pr-review-on-a-stack.md)). - TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)). - Files end with exactly one trailing newline; `git diff --check` (pre-push) gates it. diff --git a/docs/testing.md b/docs/testing.md index 84629aae45..6672bfc6f3 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -35,4 +35,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword ## When a snapshot test is required -Any change affecting an editor-facing transcript, headless event stream, or end-to-end agent UX adds or updates a scenario in the owning snapshot suite, or states in the PR why none applies. ACP surfaces use `examples//tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it — a harness gap is scheduled work, not a mid-build surprise. +Every non-trivial model- or human-visible change adds or updates a keyless scenario through a real runnable example in the owning snapshot suite in the same PR. Package tests, e2e-only assertions, test-only or echo-agent compositions, and a PR rationale do not substitute for the assembled application transcript; when the current harness cannot express the behavior, extending that harness is part of the change. ACP surfaces use `examples//tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. New capability seams, lifecycle shapes, or transcript surfaces name their coverage at every tier at plan time and verify the harness can express it before implementation.