chore(web-e2e): gate fixes — catalog, budgets, knip

Regenerate config-catalog for the llm-replay paceMs row; condense the
testing.md web-lane entry to pointer form and raise its ceiling 1020->1060
(the two-sentence tier entry for a genuinely new surface does not fit the
old ceiling after relocation-first trims); internalize two harness helpers
knip flagged (rawSessionLog/normalizeAria are module-internal).
This commit is contained in:
Tianyi Cui
2026-07-24 20:30:33 +08:00
parent 2cd37e2765
commit d08050ab69
5 changed files with 32 additions and 12 deletions
+2 -7
View File
@@ -230,10 +230,8 @@ export async function launchWebHarness(options: LaunchOptions = {}): Promise<Web
* Serialize a live session back to raw session-JSONL (header + events) — the
* in-memory record-mode harvest, so the on-disk zstd default never matters.
* Mirrors the TUI suite's rawSessionLog.
* @param session - the live session to serialize.
* @returns raw JSONL text ending in one newline.
*/
export function rawSessionLog(session: Session): string {
function rawSessionLog(session: Session): string {
return [
JSON.stringify({ type: 'session', ...session.header }),
...session.events.map(event => JSON.stringify(event)),
@@ -333,11 +331,8 @@ export async function seedSession(harness: WebHarness, fixtureText: string, id:
/**
* Normalize an aria snapshot: uuid, cwd, workspace-basename, and duration
* volatility collapse to stable tokens.
* @param snapshot - raw ariaSnapshot text.
* @param workspaceCwd - the harness workspace (basename doubles as the header breadcrumb).
* @returns tokenized snapshot text.
*/
export function normalizeAria(snapshot: string, workspaceCwd: string): string {
function normalizeAria(snapshot: string, workspaceCwd: string): string {
// The header breadcrumb renders the workspace's basename, not the full
// path, so both spellings must collapse to the token.
const base = workspaceCwd.split('/').pop()!
+1 -1
View File
@@ -643,7 +643,7 @@ export interface ReplayModelConfig {
}
```
Source: [`packages/support/llm-replay/src/index.ts:453`](../packages/support/llm-replay/src/index.ts)
Source: [`packages/support/llm-replay/src/index.ts:454`](../packages/support/llm-replay/src/index.ts)
## `@deepseek-ai/dsh-llm-retry`
+2 -2
View File
@@ -8,7 +8,7 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning
- **Coverage gate** (`pnpm run test:coverage`): the gating run, per-file 100% on `packages/*/*/src`. An uncovered line is often dead code the gate is correctly flagging for deletion, not a missing test to bolt on. Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped.
- **Real-API e2e** (`pnpm run test:e2e`): with-key tests against live provider APIs — the DeepSeek model plus provider-specific smokes that gate on their own keys (`EXA_API_KEY`, `PERPLEXITY_API_KEY`, …); each suite self-skips without its key so keyless CI stays green ([real-API e2e Agent Note](../.agents/notes/implemented/testing/2026-06-19-real-api-e2e-ci.md)).
- **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless expected outputs cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized JSON-RPC plus the re-persisted log ([ACP snapshot Agent Note](../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md)); the headless suite independently pins `stream-json` through its real one-shot subprocess. TUI completed journeys replay recorded primary/child JSONL through the real agent loop and tools before projecting ANSI into semantic terminal-state expected outputs; package-local snapshots retain transient renderer states, and a real PTY conversation covers the process boundary ([TUI snapshot Agent Note](../.agents/notes/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when committed replay input remains correct; review every JSONL and expected-output diff. System-prompt/tool-schema content is pinned by one ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header Agent Note](../.agents/notes/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **Web browser snapshot** (inside `pnpm run test:web`, gate-exempt like the rest of that lane): the web GUI's keyless assembled-transcript tier — a real chromium over the real in-process web assembly (`llm: false` + `dsh-llm-replay`), scenario fixtures and normalized conversation aria goldens under `apps/web/tests/snapshots/`. Its record/refresh commands diverge from `test:snapshot` (`DSH_SNAPSHOT=record pnpm run test:web` re-records against the live model; `DSH_SNAPSHOT=refresh` rewrites goldens keylessly); the [web e2e lane Agent Note](../.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md) owns the design, and CI browser provisioning is deferred there.
- **Web browser snapshot** (gate-exempt `pnpm run test:web`): real chromium over the in-process web assembly replays recorded fixtures against conversation aria goldens (`apps/web/tests/snapshots/`); `DSH_SNAPSHOT=record`/`refresh` semantics and the deferred CI browser decision live in the [web e2e lane Agent Note](../.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md).
## The with-key policy: inference is cheap here
@@ -42,4 +42,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
## When a snapshot test is required
Every non-trivial model- or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. Browser-rendered web GUI journeys use `apps/web/tests/snapshots/` through the web e2e lane's harness. New capability seams, lifecycle shapes, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.
Every non-trivial model- or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. Browser-rendered web GUI journeys use `apps/web/tests/snapshots/`. New capability seams, lifecycle shapes, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.
@@ -476,6 +476,31 @@ describe('installLlmReplay (through the real LlmService)', () => {
expect(() => { handle.assertConsumed() }).not.toThrow()
})
it('paces a throw-entry prefix too (the recorded partial streams at the same cadence)', async () => {
writeFileSync(file, sessionJsonl([]), 'utf8')
const overrideFile = join(dir, 'replay.override.json')
const partial: StreamChunk[] = [{ type: 'block-start', index: 0, blockType: 'text' }]
writeFileSync(overrideFile, JSON.stringify([
{ kind: 'throw', chunks: partial, message: 'boom', code: 'STREAM_CLOSED' },
]), 'utf8')
const ctx = new Context()
await ctx.plugin(LlmService)
installLlmReplay(ctx, { file, overrideFile, paceMs: 10 })
const started = performance.now()
await expect(drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [] }))).rejects.toThrow('boom')
expect(performance.now() - started).toBeGreaterThanOrEqual(5)
})
it('assertConsumed names an underrunning identified session by its id', async () => {
writeLog(TEXT_CHUNKS, TEXT_CHUNKS)
const ctx = new Context()
await ctx.plugin(LlmService)
const handle = installLlmReplay(ctx, { file })
const sessionId = 'live-underrun' as NonNullable<GenerateOptions['sessionId']>
await drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [], sessionId }))
expect(() => { handle.assertConsumed() }).toThrow(/session live-underrun consumed 1\/2/)
})
it('assertConsumed reports recorded scripts no live session ever bound', async () => {
writeLog(TEXT_CHUNKS)
const childFile = join(dir, 'session.1.jsonl')
@@ -690,7 +715,7 @@ describe('apply (the plugin entry)', () => {
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8')
const ctx = new Context()
await ctx.plugin(LlmService)
apply(ctx, { file, providers: [{ id: 'm', models: [{ id: 'm' }] }] })
apply(ctx, { file, providers: [{ id: 'm', models: [{ id: 'm' }] }], paceMs: 1 })
expect(ctx.llm.listProviders()).toEqual([{ id: 'm', name: 'm' }])
expect(await drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
})
+1 -1
View File
@@ -4,7 +4,7 @@
"docs/architecture.md": 1800,
"docs/cordis-primer.md": 600,
"docs/defensive-patterns.md": 550,
"docs/testing.md": 1020,
"docs/testing.md": 1060,
"examples/AGENTS.md": 310,
"packages/AGENTS.md": 660,
"packages/README.md": 760