Tianyi Cui
7c400e9c02
Collapse docs/adr/ and docs/rfc/ into a single docs/rfc/ with proposed/, implemented/, and rejected/ subfolders. Every file is renamed to yyyy-mm-dd-topic-title.md, where the date is when the topic was first proposed (from git history). ADRs and RFCs that covered exactly the same topic are merged (property-based testing, session persistence); the umbrella RFC 005 stays split across its three implemented decisions, and RFC 006's deferred part-3 (API extractor reports) splits into its own proposed RFC. All cross-references become machine-checkable relative links instead of bare "ADR NNNN" / "RFC NNN" prose. Add a verify-md-links doc-sync gate (scripts/verify-md-links.ts) that checks every relative Markdown cross-link resolves, wired into doc-sync alongside verify-md-wrap. This makes the reorganization self-verifying: the same change that rewrote ~forty inter-doc links adds the check that proves none dangle. Document the cross-link convention in a new docs/AGENTS.md and record the gate as an implemented RFC. doc-sync, typecheck, lint, and the full test suite (667) all pass.
1.6 KiB
1.6 KiB
RFC: Mutation testing as the coverage counterweight
Status: proposed
Problem
The per-file 100% coverage gate (the quality-gates decision) proves every line executes under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite kills deliberately injected bugs.
Proposal
Stryker (@stryker-mutator/vitest-runner) over packages/*/src:
- PR-scoped incremental runs (changed files only) as a CI job — fast enough to gate merges once tuned.
- Nightly full runs with a tracked mutation score; start by recording, then set the threshold at the observed baseline and ratchet upward (same policy as coverage: thresholds only ever tighten).
- Surviving mutants are work items: an agent picks a survivor, writes the killing test, repeats — a well-shaped autonomous loop.
- Equivalent mutants (provably behavior-preserving) get annotated exclusions with reasons, mirroring the
/* v8 ignore */policy.
Plan
- Add Stryker config scoped to one package (llm — smallest, most algorithmic) and measure runtime.
- Expand to all packages; record baseline scores in the config.
- Wire the nightly job; add the incremental PR job once runtime is acceptable.
Risks
Runtime: mutation testing is expensive; per-file 100% coverage helps (every mutant is at least reached). If PR-scoped runs stay too slow, keep them nightly-only and rely on the score ratchet.