refactor(web-e2e): rename harness -> scaffold; add interaction coverage
The shared test module was named harness.ts inside a repo whose product IS a harness — hopelessly ambiguous. Renamed to scaffold.ts with launchWebScaffold/WebScaffold; tsconfig plane-split entries, the seam JSDoc/README mentions, and both Agent Note languages updated. Both scenarios gain a Playwright interaction step over the settled transcript (after the golden capture, so committed aria surfaces stay untouched): replay-round-trip clicks the reasoning fold open/closed over wire-delivered state; seeded-history expands a read tool row rebuilt from the cold log and asserts the recorded result text appears (read rows are expand-in-place — rowExpands routes the click to the inline fold, not the details column). test:web 30 passed | 1 skipped.
This commit is contained in:
Tianyi Cui
@@ -12,11 +12,11 @@ The web GUI ships as a real assembled chain — chromium page → client plugin
|
||||
|
||||
`pnpm run test:web` carries a keyless, deterministic browser e2e lane under `apps/web/tests/`: recorded session-log fixtures replayed through `@deepseek-ai/dsh-llm-replay` against the real in-process web assembly, asserting a normalized conversation aria golden plus in-process world state. No new package; the product deltas are the `BootHostOptions.llm` seam and two additive `dsh-llm-replay` surfaces.
|
||||
|
||||
### Harness: `apps/web/tests/harness.ts`
|
||||
### Scaffold: `apps/web/tests/scaffold.ts`
|
||||
|
||||
A plain shared-fixture module (the [testing-policy sanctioned shape](../../../../docs/testing.md)), not a package: the gate-worthy logic — replay derivation, session parsing, log scrubbing, persistence — lives in the gated packages `dsh-llm-replay`, `dsh-acp-snapshot`, and `dsh-session-persistence-jsonl`; what remains is boot wiring and browser glue, and chromium-driving code cannot hold per-file 100% coverage on the browserless coverage runners.
|
||||
|
||||
`launchWebHarness()` boots the real web assembly in-process from the exported production functions — `startHost({ boot: { …, llm: false } })`, `installLlmReplay(host.ctx, { file, providers, paceMs })`, `mountWebPlugins(host.ctx, roster, anchor)`, `createHostWebPluginRegistry`, `startWebServer({ port: 0, … })`. This is the web analog of the TUI suite mounting the production bundle in-process ([TUI snapshots](2026-07-18-tui-terminal-state-snapshots.md)): the real entry boundary (`dsh web` bin arg-parsing, dist resolution) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`, and the web surface has no `cordis.yml` to bypass — assembly is written in the app per the [GUI layering decision](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md), a ruling this lane's design review explicitly reaffirmed (Loader-izing `dsh web` was declined; it would be its own proposal). Two deliberate assembly divergences from the `dsh web` shell, noted in the harness header: `workspaceContext: false` (recorded fixtures must not embed this repo's AGENTS.md) and `sessionTitleLlm` left at bootHost's disabled default (its fire-and-forget title call would share the session's replay cursor nondeterministically).
|
||||
`launchWebScaffold()` boots the real web assembly in-process from the exported production functions — `startHost({ boot: { …, llm: false } })`, `installLlmReplay(host.ctx, { file, providers, paceMs })`, `mountWebPlugins(host.ctx, roster, anchor)`, `createHostWebPluginRegistry`, `startWebServer({ port: 0, … })`. This is the web analog of the TUI suite mounting the production bundle in-process ([TUI snapshots](2026-07-18-tui-terminal-state-snapshots.md)): the real entry boundary (`dsh web` bin arg-parsing, dist resolution) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`, and the web surface has no `cordis.yml` to bypass — assembly is written in the app per the [GUI layering decision](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md), a ruling this lane's design review explicitly reaffirmed (Loader-izing `dsh web` was declined; it would be its own proposal). Two deliberate assembly divergences from the `dsh web` shell, noted in the scaffold header: `workspaceContext: false` (recorded fixtures must not embed this repo's AGENTS.md) and `sessionTitleLlm` left at bootHost's disabled default (its fire-and-forget title call would share the session's replay cursor nondeterministically).
|
||||
|
||||
The `llm: false` seam is the reviewed resolution of the keyless-boot question: `'deepseek' | false` on `BootHostOptions`, matching the `workspaceContext: Config | false` shape, with `RunningHost.ctx` JSDoc naming "filling a deliberately-open capability seam" as its third sanctioned use. Replay runs in providers-catalog mode with a published `contextWindow` (the TUI `PROVIDERS` shape), never catch-all: with no adapter registered, catch-all would leave `resolveModelContext` unroutable and `compact-basic`'s post-step pressure check would warn every step instead of being provably inert.
|
||||
|
||||
@@ -28,13 +28,13 @@ The barrier stack for a prompted turn, in order: (1) host-side `await agent.when
|
||||
|
||||
No single-shot transient-DOM assertions: every hop from replay yield to React commit can coalesce chunks, so sampling `[data-streaming]` is a race by construction. Streaming incrementality is asserted from the persisted `assistant/chunk` events (model-visible ⟺ logged makes the log the authoritative proof). `dsh-llm-replay`'s opt-in `paceMs` (default absent = burst) is a realism knob so the browser observes genuinely incremental SSE; correctness never leans on it, and abort during a pace wait cancels promptly.
|
||||
|
||||
Every scenario fails on any pageerror and on the client's connection-loss/gap-repair console warnings: the reconnect machine plus history resync would otherwise self-heal a dead SSE path and the suite would certify a broken wire. Harness `close()` calls the `ReplayHandle.assertConsumed()` teardown check (every recorded script bound, every cursor drained), converting silent underruns and shifted bindings into crisp diagnostics. No vitest retry on the lane; one chromium per file, fresh context per scenario, one host per scenario; viewport pinned; selectors anchor on roles, `data-*` attributes, and visible text.
|
||||
Every scenario fails on any pageerror and on the client's connection-loss/gap-repair console warnings: the reconnect machine plus history resync would otherwise self-heal a dead SSE path and the suite would certify a broken wire. Scaffold `close()` calls the `ReplayHandle.assertConsumed()` teardown check (every recorded script bound, every cursor drained), converting silent underruns and shifted bindings into crisp diagnostics. No vitest retry on the lane; one chromium per file, fresh context per scenario, one host per scenario; viewport pinned; selectors anchor on roles, `data-*` attributes, and visible text.
|
||||
|
||||
### Expected outputs
|
||||
|
||||
One committed golden per scenario: a normalized `ariaSnapshot()` of the conversation region (`ui.expected.md`) — uuid/cwd/workspace-basename/duration tokens normalized, captured poll-until-equal at the settled milestone — plus a few role/text anchor assertions that stay green under a semantics-preserving component rewrite while the golden churns reviewably. The aria tree is the mechanization of the client rule "assert what the user would see, never class names". World-state assertions ride `host.ctx` session events inline (which tools ran, `turn/end` completed) instead of a second committed log golden: the persisted-log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence, and re-pinning it here would double refresh cost against the tier discipline. `refresh` is the sole golden writer — a missing golden in replay mode fails with the healing command rather than self-bootstrapping.
|
||||
|
||||
The typecheck plane split is structural: `apps/web/tests/{harness,support,replay-round-trip.e2e,seeded-history.e2e}.ts` are host-plane programs (they boot the host spine), so they are excluded from the client-registered `apps/web` project and included file-by-file in `tsconfig.host.json` — one program cannot hold both sides of the cordis `Context` merges.
|
||||
The typecheck plane split is structural: `apps/web/tests/{scaffold,support,replay-round-trip.e2e,seeded-history.e2e}.ts` are host-plane programs (they boot the host spine), so they are excluded from the client-registered `apps/web` project and included file-by-file in `tsconfig.host.json` — one program cannot hold both sides of the cordis `Context` merges.
|
||||
|
||||
### Modes and fixtures
|
||||
|
||||
@@ -81,7 +81,7 @@ The lane itself: `pnpm run test:web` runs both scenarios keylessly alongside the
|
||||
|
||||
## Deferred
|
||||
|
||||
- **Web header-class pin**: web fixtures tokenize `{{system}}`/`{{tools}}` everywhere and no scenario pins bootHost's composed prompt/tool schemas (`TODO(web-header-pin)` — the harness `recordFixture` JSDoc marks it). Following the TUI scrub-everywhere precedent; revisit when the web assembly's header diverges from the repl composition it mirrors.
|
||||
- **Web header-class pin**: web fixtures tokenize `{{system}}`/`{{tools}}` everywhere and no scenario pins bootHost's composed prompt/tool schemas (`TODO(web-header-pin)` — the scaffold `recordFixture` JSDoc marks it). Following the TUI scrub-everywhere precedent; revisit when the web assembly's header diverges from the repl composition it mirrors.
|
||||
- **CI browser provisioning**: reversal of the no-browser-in-CI ruling, staged criteria above (`TODO(ci-browser)`).
|
||||
- **Follow-up-prompt-after-resume scenario**: the history/live stitch path over the real wire; add as its own scenario when that code changes or regresses.
|
||||
|
||||
|
||||
@@ -12,11 +12,11 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
`pnpm run test:web` 携带 `apps/web/tests/` 下的无密钥、确定性浏览器 e2e 车道：录制的会话日志 fixture 经 `@deepseek-ai/dsh-llm-replay` 对真实进程内 web 组装回放，断言规范化后的会话区 aria 预期输出加进程内世界状态。不新增包（package）；产品侧增量只有 `BootHostOptions.llm` seam 和 `dsh-llm-replay` 的两处增量接口。
|
||||
|
||||
### Harness：`apps/web/tests/harness.ts`
|
||||
### Scaffold：`apps/web/tests/scaffold.ts`
|
||||
|
||||
一个普通的共享 fixture 模块（[测试政策认可的形态](../../../../docs/testing.md)），不是包：值得门禁把守的逻辑——回放推导、会话解析、日志脱敏、持久化——都在已受门禁的包 `dsh-llm-replay`、`dsh-acp-snapshot`、`dsh-session-persistence-jsonl` 中；剩下的只是启动接线和浏览器胶水，而驱动 chromium 的源码在无浏览器的覆盖率 runner 上无法诚实保持逐文件 100% 覆盖率。
|
||||
|
||||
`launchWebHarness()` 用导出的生产函数在进程内启动真实 web 组装——`startHost({ boot: { …, llm: false } })`、`installLlmReplay(host.ctx, { file, providers, paceMs })`、`mountWebPlugins(host.ctx, roster, anchor)`、`createHostWebPluginRegistry`、`startWebServer({ port: 0, … })`。这是 TUI 套件进程内挂载生产 bundle 的 web 对应物（[TUI 快照](2026-07-18-tui-terminal-state-snapshots.md)）：真实入口边界（`dsh web` bin 的参数解析、dist 解析）仍由 `smoke-real.e2e.ts` 中的无密钥 CLI 冒烟把守，且 web 表面没有可绕过的 `cordis.yml`——按[GUI 分层决策](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md)，组装写在应用里；本车道的设计评审明确重申了这一裁定（Loader 化 `dsh web` 被否决；那需要自己的提案）。与 `dsh web` shell 的两处刻意组装差异已注明在 harness 头部：`workspaceContext: false`（录制的 fixture 不得嵌入本仓库的 AGENTS.md），以及 `sessionTitleLlm` 保持 bootHost 的禁用默认值（其发后不管的标题调用会与循环自身的调用不确定地共享会话的回放游标）。
|
||||
`launchWebScaffold()` 用导出的生产函数在进程内启动真实 web 组装——`startHost({ boot: { …, llm: false } })`、`installLlmReplay(host.ctx, { file, providers, paceMs })`、`mountWebPlugins(host.ctx, roster, anchor)`、`createHostWebPluginRegistry`、`startWebServer({ port: 0, … })`。这是 TUI 套件进程内挂载生产 bundle 的 web 对应物（[TUI 快照](2026-07-18-tui-terminal-state-snapshots.md)）：真实入口边界（`dsh web` bin 的参数解析、dist 解析）仍由 `smoke-real.e2e.ts` 中的无密钥 CLI 冒烟把守，且 web 表面没有可绕过的 `cordis.yml`——按[GUI 分层决策](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md)，组装写在应用里；本车道的设计评审明确重申了这一裁定（Loader 化 `dsh web` 被否决；那需要自己的提案）。与 `dsh web` shell 的两处刻意组装差异已注明在 scaffold 头部：`workspaceContext: false`（录制的 fixture 不得嵌入本仓库的 AGENTS.md），以及 `sessionTitleLlm` 保持 bootHost 的禁用默认值（其发后不管的标题调用会与循环自身的调用不确定地共享会话的回放游标）。
|
||||
|
||||
`llm: false` seam 是无密钥启动问题经评审后的定论：`BootHostOptions` 上的 `'deepseek' | false`，与 `workspaceContext: Config | false` 形态一致，且 `RunningHost.ctx` 的 JSDoc 把「填充刻意开放的能力 seam」列为其第三种认可用法。回放必须以提供方目录（providers-catalog）模式运行并发布 `contextWindow`（TUI 的 `PROVIDERS` 形态），绝不用 catch-all：没有注册适配器时，catch-all 会让 `resolveModelContext` 无路由可走，`compact-basic` 的步后压力检查将步步告警，而不是被可证明地闲置。
|
||||
|
||||
@@ -28,13 +28,13 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
不做单次瞬态 DOM 断言：从回放产出到 React 提交的每一跳都可能合并分片，采样 `[data-streaming]` 天然就是竞态。流式输出的增量性由持久化的 `assistant/chunk` 事件断言（模型可见 ⟺ 已记录，使日志成为权威证据）。`dsh-llm-replay` 的可选 `paceMs`（默认缺省 = 突发）只是让浏览器观察到真正增量 SSE 的真实感旋钮；正确性绝不依赖它，且节奏等待期间中止会即时取消。
|
||||
|
||||
每个场景都会因任何 pageerror 或客户端的连接丢失/间隙修复控制台警告而失败：否则重连机制加历史重同步会把一条死掉的 SSE 通路自愈掉，套件反而认证了坏 wire。Harness 的 `close()` 调用 `ReplayHandle.assertConsumed()` 收尾检查（每个已录脚本都被绑定、每个游标都耗尽），把静默的少放与错绑变成清晰诊断。车道不设 vitest 重试；每文件一个 chromium、每场景一个新 context、每场景一个 host；视口固定；选择器只锚定 role、`data-*` 属性和可见文本。
|
||||
每个场景都会因任何 pageerror 或客户端的连接丢失/间隙修复控制台警告而失败：否则重连机制加历史重同步会把一条死掉的 SSE 通路自愈掉，套件反而认证了坏 wire。Scaffold 的 `close()` 调用 `ReplayHandle.assertConsumed()` 收尾检查（每个已录脚本都被绑定、每个游标都耗尽），把静默的少放与错绑变成清晰诊断。车道不设 vitest 重试；每文件一个 chromium、每场景一个新 context、每场景一个 host；视口固定；选择器只锚定 role、`data-*` 属性和可见文本。
|
||||
|
||||
### 预期输出
|
||||
|
||||
每场景一份提交的预期输出：会话区规范化 `ariaSnapshot()`（`ui.expected.md`）——uuid/cwd/工作区目录名/时长归一为稳定 token，在安定里程碑处轮询至两次相等再采集——外加几条 role/文本锚断言，让保语义的组件重写在预期输出可评审地变动时仍保持绿色锚点。aria 树是 client 规则「断言用户所见，绝不断言类名」的机械化。世界状态断言内联在 `host.ctx` 会话事件上（哪些工具运行了、`turn/end` 完成）而不是第二份提交的日志预期输出：持久化日志表面已由 ACP/headless/TUI 套件经同一循环和持久化钉住，在此重复钉住会违背分层纪律、翻倍刷新成本。`refresh` 是预期输出的唯一写入者——回放模式下预期输出缺失会连同修复命令一起报错，而不是静默自举。
|
||||
|
||||
类型检查平面切分是结构性的：`apps/web/tests/{harness,support,replay-round-trip.e2e,seeded-history.e2e}.ts` 是 host 平面程序（它们启动 host 主干），因此被排除出注册在 client 侧的 `apps/web` 工程，逐文件纳入 `tsconfig.host.json`——一个程序不能同时持有 cordis `Context` 合并的两侧。
|
||||
类型检查平面切分是结构性的：`apps/web/tests/{scaffold,support,replay-round-trip.e2e,seeded-history.e2e}.ts` 是 host 平面程序（它们启动 host 主干），因此被排除出注册在 client 侧的 `apps/web` 工程，逐文件纳入 `tsconfig.host.json`——一个程序不能同时持有 cordis `Context` 合并的两侧。
|
||||
|
||||
### 模式与 fixture
|
||||
|
||||
@@ -81,7 +81,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
## 暂缓
|
||||
|
||||
- **Web 头类别钉住**：web fixture 处处 token 化 `{{system}}`/`{{tools}}`，没有场景钉住 bootHost 组装的提示词/工具 schema（`TODO(web-header-pin)`——harness 的 `recordFixture` JSDoc 有标记）。沿用 TUI 处处脱敏先例；当 web 组装的请求头与其镜像的 repl 组合进一步分叉时重审。
|
||||
- **Web 头类别钉住**：web fixture 处处 token 化 `{{system}}`/`{{tools}}`，没有场景钉住 bootHost 组装的提示词/工具 schema（`TODO(web-header-pin)`——scaffold 的 `recordFixture` JSDoc 有标记）。沿用 TUI 处处脱敏先例；当 web 组装的请求头与其镜像的 repl 组合进一步分叉时重审。
|
||||
- **CI 浏览器供给**：推翻 CI 无浏览器裁定，分阶段标准见上（`TODO(ci-browser)`）。
|
||||
- **恢复后追问场景**：真实 wire 上的历史/实时缝合路径；当该代码变更或回归时作为独立场景补充。
|
||||
|
||||
|
||||
@@ -16,8 +16,8 @@ import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
assertFixtureInventory, captureStableAria, compareOrRefreshGolden, fixtureUserPrompts,
|
||||
launchWebHarness, recordFixture, watchConsole, webSnapshotMode, type WebHarness,
|
||||
} from './harness.ts'
|
||||
launchWebScaffold, recordFixture, watchConsole, webSnapshotMode, type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { saveFailureShot } from './support.ts'
|
||||
|
||||
const SNAPSHOT_DIR = fileURLToPath(new URL('./snapshots/fresh-round-trip', import.meta.url))
|
||||
@@ -31,27 +31,27 @@ const MODE = webSnapshotMode()
|
||||
const PROMPT = 'Use the bash tool to run exactly: echo WEB_E2E_OK. Then reply with the single word DONE and stop.'
|
||||
|
||||
describe('web e2e: fresh round trip through the real assembly', () => {
|
||||
let harness: WebHarness
|
||||
let scaffold: WebScaffold
|
||||
let browser: Browser
|
||||
let page: Page
|
||||
let tripwire: ReturnType<typeof watchConsole>
|
||||
const sessionEvents: SessionEvent[] = []
|
||||
|
||||
beforeAll(async () => {
|
||||
harness = await launchWebHarness({
|
||||
scaffold = await launchWebScaffold({
|
||||
...(MODE === 'record' ? {} : { replayFixture: FIXTURE, paceMs: 15 }),
|
||||
})
|
||||
harness.host.ctx.on('session/event', (_session, event: SessionEvent) => { sessionEvents.push(event) })
|
||||
scaffold.host.ctx.on('session/event', (_session, event: SessionEvent) => { sessionEvents.push(event) })
|
||||
browser = await chromium.launch()
|
||||
page = await browser.newPage({ viewport: { width: 1680, height: 1000 } })
|
||||
tripwire = watchConsole(page)
|
||||
await page.goto(harness.baseUrl, { waitUntil: 'load' })
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
}, 120_000)
|
||||
|
||||
afterAll(async () => {
|
||||
await browser?.close()
|
||||
await harness?.close()
|
||||
await scaffold?.close()
|
||||
})
|
||||
|
||||
it('drives the recorded prompt to a settled turn (all modes)', async () => {
|
||||
@@ -63,12 +63,12 @@ describe('web e2e: fresh round trip through the real assembly', () => {
|
||||
const input = page.locator('textarea').first()
|
||||
await input.waitFor({ timeout: 10_000 })
|
||||
// Arm the host-side settled barrier BEFORE the send click.
|
||||
const settled = harness.whenTurnSettled()
|
||||
const settled = scaffold.whenTurnSettled()
|
||||
await input.fill(PROMPT)
|
||||
await input.press('Enter')
|
||||
const sessionId = await settled
|
||||
if (MODE === 'record') {
|
||||
await recordFixture(harness, sessionId, FIXTURE)
|
||||
await recordFixture(scaffold, sessionId, FIXTURE)
|
||||
}
|
||||
}, 200_000)
|
||||
|
||||
@@ -96,14 +96,28 @@ describe('web e2e: fresh round trip through the real assembly', () => {
|
||||
// while the whole-region golden churns.
|
||||
await expect(page.getByRole('textbox').first().isVisible()).resolves.toBe(true)
|
||||
expect(await page.getByText('WEB_E2E_OK', { exact: false }).count()).toBeGreaterThanOrEqual(1)
|
||||
const snapshot = await captureStableAria(page, '[class*="centerCol"]', harness.workspaceCwd)
|
||||
const snapshot = await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd)
|
||||
await compareOrRefreshGolden(UI_EXPECTED, snapshot, MODE)
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('expands and collapses the reasoning fold from its click target', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-round-trip-think'))
|
||||
// Interaction over the REAL wire-delivered transcript (the fixture-client
|
||||
// tier pins the same gesture against FixtureApiClient; this one runs on
|
||||
// mux-frame-fed state). Runs after the golden capture so the committed
|
||||
// aria surface stays the untouched settled state.
|
||||
const think = page.getByRole('button', { name: /^Think/ }).first()
|
||||
expect(await think.getAttribute('aria-expanded')).toBe('false')
|
||||
await think.click()
|
||||
await expect.poll(() => think.getAttribute('aria-expanded'), { timeout: 5_000 }).toBe('true')
|
||||
await think.click()
|
||||
await expect.poll(() => think.getAttribute('aria-expanded'), { timeout: 5_000 }).toBe('false')
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('stayed clean: no pageerrors, no reconnect self-healing, no server errors', async () => {
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
expect(harness.serverErrors).toEqual([])
|
||||
expect(scaffold.serverErrors).toEqual([])
|
||||
await assertFixtureInventory(SNAPSHOT_DIR, ['session.jsonl', 'ui.expected.md'])
|
||||
})
|
||||
})
|
||||
@@ -1,4 +1,4 @@
|
||||
// Shared harness for the keyless browser e2e lane (Agent Note:
|
||||
// Shared scaffold for the keyless browser e2e lane (Agent Note:
|
||||
// .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md).
|
||||
// Boots the REAL web assembly in-process from the exported production
|
||||
// functions — startHost (bootHost spine) + mountWebPlugins + registry +
|
||||
@@ -78,9 +78,9 @@ function loadRootEnv(): void {
|
||||
}
|
||||
}
|
||||
|
||||
/** A booted web harness: real assembly, mode-selected model backend, temp world. */
|
||||
export interface WebHarness {
|
||||
/** The active snapshot mode this harness booted under. */
|
||||
/** A booted web scaffold: real assembly, mode-selected model backend, temp world. */
|
||||
export interface WebScaffold {
|
||||
/** The active snapshot mode this scaffold booted under. */
|
||||
mode: WebSnapshotMode
|
||||
/** Browser-facing origin (http://127.0.0.1:<bound port>). */
|
||||
baseUrl: string
|
||||
@@ -98,7 +98,7 @@ export interface WebHarness {
|
||||
close(): Promise<void>
|
||||
}
|
||||
|
||||
/** Options for {@link launchWebHarness}. */
|
||||
/** Options for {@link launchWebScaffold}. */
|
||||
export interface LaunchOptions {
|
||||
/**
|
||||
* Replay fixture (session.jsonl) served by dsh-llm-replay in replay/refresh
|
||||
@@ -114,9 +114,9 @@ export interface LaunchOptions {
|
||||
/**
|
||||
* Boot the real web assembly under the current snapshot mode.
|
||||
* @param options - replay fixture selection and pacing.
|
||||
* @returns the running harness.
|
||||
* @returns the running scaffold.
|
||||
*/
|
||||
export async function launchWebHarness(options: LaunchOptions = {}): Promise<WebHarness> {
|
||||
export async function launchWebScaffold(options: LaunchOptions = {}): Promise<WebScaffold> {
|
||||
requireDist()
|
||||
const mode = webSnapshotMode()
|
||||
if (mode === 'record') {
|
||||
@@ -221,7 +221,7 @@ export async function launchWebHarness(options: LaunchOptions = {}): Promise<Web
|
||||
await runningHost.dispose().catch((e: unknown) => failures.push(e))
|
||||
await rm(workspaceCwd, { recursive: true, force: true }).catch((e: unknown) => failures.push(e))
|
||||
await rm(persistenceRoot, { recursive: true, force: true }).catch((e: unknown) => failures.push(e))
|
||||
if (failures.length > 0) throw new AggregateError(failures, 'web harness teardown failed')
|
||||
if (failures.length > 0) throw new AggregateError(failures, 'web scaffold teardown failed')
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -246,16 +246,16 @@ function rawSessionLog(session: Session): string {
|
||||
* work), tokenize the run-local session id and cwd ({{sessionId}}/{{cwd}},
|
||||
* the committed ACP fixture convention — re-records then diff only on real
|
||||
* content), and write the committed fixture.
|
||||
* @param harness - the record-mode harness.
|
||||
* @param scaffold - the record-mode scaffold.
|
||||
* @param sessionId - the driven session.
|
||||
* @param fixturePath - the committed session.jsonl / seed.jsonl target.
|
||||
*/
|
||||
export async function recordFixture(harness: WebHarness, sessionId: SessionId, fixturePath: string): Promise<void> {
|
||||
const agent = harness.host.ctx.agents.get(sessionId)
|
||||
export async function recordFixture(scaffold: WebScaffold, sessionId: SessionId, fixturePath: string): Promise<void> {
|
||||
const agent = scaffold.host.ctx.agents.get(sessionId)
|
||||
if (agent === undefined) throw new Error(`record harvest: no live agent for ${sessionId}`)
|
||||
const tokenized = scrubRequestHeaders(rawSessionLog(agent.session))
|
||||
.split(sessionId).join('{{sessionId}}')
|
||||
.split(harness.workspaceCwd).join('{{cwd}}')
|
||||
.split(scaffold.workspaceCwd).join('{{cwd}}')
|
||||
await writeFile(fixturePath, tokenized)
|
||||
}
|
||||
|
||||
@@ -274,27 +274,27 @@ export function fixtureUserPrompts(fixtureText: string): string[] {
|
||||
}
|
||||
|
||||
/**
|
||||
* Seed a recorded session fixture into the harness's persistence root through
|
||||
* Seed a recorded session fixture into the scaffold's persistence root through
|
||||
* the REAL backend API (throwaway Context + SessionStore + JSONL plugin — the
|
||||
* semantic-checkpoint precedent), never raw file writes: no knowledge of
|
||||
* bucket hashing, filename encoding, or compression, and malformed shapes
|
||||
* fail loud at seed time. The fixture's recorded cwd is rewritten to the
|
||||
* harness workspace so header/path identity and event payload paths agree.
|
||||
* @param harness - the target harness.
|
||||
* scaffold workspace so header/path identity and event payload paths agree.
|
||||
* @param scaffold - the target scaffold.
|
||||
* @param fixtureText - raw recorded session.jsonl contents.
|
||||
* @param id - the seeded session id (stable for deterministic goldens).
|
||||
* @returns the seeded id.
|
||||
*/
|
||||
export async function seedSession(harness: WebHarness, fixtureText: string, id: string): Promise<SessionId> {
|
||||
export async function seedSession(scaffold: WebScaffold, fixtureText: string, id: string): Promise<SessionId> {
|
||||
// Committed fixtures tokenize run-local identity ({{sessionId}}/{{cwd}},
|
||||
// written by recordFixture); realize both for this world before parsing.
|
||||
const realized = fixtureText
|
||||
.split('{{sessionId}}').join(id)
|
||||
.split('{{cwd}}').join(harness.workspaceCwd)
|
||||
.split('{{cwd}}').join(scaffold.workspaceCwd)
|
||||
const fixtureCwd = (JSON.parse(realized.split('\n', 1)[0]!) as { cwd?: string }).cwd
|
||||
const rewritten = fixtureCwd === undefined
|
||||
? realized
|
||||
: realized.split(fixtureCwd).join(harness.workspaceCwd)
|
||||
: realized.split(fixtureCwd).join(scaffold.workspaceCwd)
|
||||
const events = parseSessionLog(rewritten)
|
||||
if (events.length === 0) throw new Error('seed fixture has no events')
|
||||
const last = events[events.length - 1]!
|
||||
@@ -305,7 +305,7 @@ export async function seedSession(harness: WebHarness, fixtureText: string, id:
|
||||
version: SESSION_FORMAT_VERSION,
|
||||
id: SessionId(id),
|
||||
createdAt: Date.now() - 60_000,
|
||||
cwd: harness.workspaceCwd,
|
||||
cwd: scaffold.workspaceCwd,
|
||||
delegationDepth: 0,
|
||||
}
|
||||
const ctx = new Context()
|
||||
@@ -313,7 +313,7 @@ export async function seedSession(harness: WebHarness, fixtureText: string, id:
|
||||
await ctx.plugin(SessionStore)
|
||||
// Same root as the host with the plugin's own default compression, so the
|
||||
// host's directory-scan list() sees one consistent encoding.
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root: harness.persistenceRoot })
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root: scaffold.persistenceRoot })
|
||||
await ctx.sessionPersistence.create(meta)
|
||||
await ctx.sessionPersistence.append(meta.id, events)
|
||||
// Deterministic sidebar order: cold summaries take updatedAt from mtime.
|
||||
@@ -15,8 +15,8 @@ import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import { join } from 'node:path'
|
||||
import {
|
||||
assertFixtureInventory, captureStableAria, compareOrRefreshGolden, fixtureUserPrompts,
|
||||
launchWebHarness, recordFixture, seedSession, watchConsole, webSnapshotMode, type WebHarness,
|
||||
} from './harness.ts'
|
||||
launchWebScaffold, recordFixture, seedSession, watchConsole, webSnapshotMode, type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { saveFailureShot } from './support.ts'
|
||||
|
||||
const SNAPSHOT_DIR = fileURLToPath(new URL('./snapshots/seeded-history', import.meta.url))
|
||||
@@ -28,44 +28,44 @@ const SEED_ID = 'seeded-history-web-e2e'
|
||||
const PROMPT = 'Use the read tool twice in one assistant message: read a.txt and b.txt. Then reply with the single word DONE and stop.'
|
||||
|
||||
describe('web e2e: seeded history renders through cold resume', () => {
|
||||
let harness: WebHarness
|
||||
let scaffold: WebScaffold
|
||||
let browser: Browser
|
||||
let page: Page
|
||||
let tripwire: ReturnType<typeof watchConsole>
|
||||
|
||||
beforeAll(async () => {
|
||||
harness = await launchWebHarness({})
|
||||
scaffold = await launchWebScaffold({})
|
||||
// The read-tool targets exist in both modes: record needs them for the
|
||||
// live turn; replay's seeded log carries their recorded contents but the
|
||||
// workspace stays consistent for any user poking the harness.
|
||||
await writeFile(join(harness.workspaceCwd, 'a.txt'), 'alpha\n')
|
||||
await writeFile(join(harness.workspaceCwd, 'b.txt'), 'beta\n')
|
||||
// workspace stays consistent for any user poking the scaffold.
|
||||
await writeFile(join(scaffold.workspaceCwd, 'a.txt'), 'alpha\n')
|
||||
await writeFile(join(scaffold.workspaceCwd, 'b.txt'), 'beta\n')
|
||||
if (MODE !== 'record') {
|
||||
const raw = await readFile(SEED, 'utf8')
|
||||
expect(fixtureUserPrompts(raw), 'seed fixture must carry exactly the drive prompt').toEqual([PROMPT])
|
||||
await seedSession(harness, raw, SEED_ID)
|
||||
await seedSession(scaffold, raw, SEED_ID)
|
||||
}
|
||||
browser = await chromium.launch()
|
||||
page = await browser.newPage({ viewport: { width: 1680, height: 1000 } })
|
||||
tripwire = watchConsole(page)
|
||||
await page.goto(harness.baseUrl, { waitUntil: 'load' })
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
}, 120_000)
|
||||
|
||||
afterAll(async () => {
|
||||
await browser?.close()
|
||||
await harness?.close()
|
||||
await scaffold?.close()
|
||||
})
|
||||
|
||||
it.skipIf(MODE !== 'record')('records the seed turn live through the composer', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-seeded-record'))
|
||||
const input = page.locator('textarea').first()
|
||||
await input.waitFor({ timeout: 10_000 })
|
||||
const settled = harness.whenTurnSettled()
|
||||
const settled = scaffold.whenTurnSettled()
|
||||
await input.fill(PROMPT)
|
||||
await input.press('Enter')
|
||||
const sessionId = await settled
|
||||
await recordFixture(harness, sessionId, SEED)
|
||||
await recordFixture(scaffold, sessionId, SEED)
|
||||
}, 200_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('lists the seeded session cold and renders its history from the log', async () => {
|
||||
@@ -89,17 +89,34 @@ describe('web e2e: seeded history renders through cold resume', () => {
|
||||
|
||||
it.skipIf(MODE === 'record')('matches the historical conversation aria golden', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-seeded-aria'))
|
||||
const snapshot = (await captureStableAria(page, '[class*="centerCol"]', harness.workspaceCwd))
|
||||
const snapshot = (await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd))
|
||||
.split(SEED_ID).join('{{seededId}}')
|
||||
await compareOrRefreshGolden(UI_EXPECTED, snapshot, MODE)
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('expands and collapses a tool row rebuilt from the cold log', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-seeded-toolrow'))
|
||||
// Interaction over cold-resumed history: read rows are expand-in-place
|
||||
// rows (rowExpands routes the click to toggleExpand, not openDetails), so
|
||||
// the gesture under test is the inline fold over log-rebuilt content.
|
||||
// Runs after the golden capture; still zero model calls.
|
||||
const row = page.locator('[data-variant] [data-clickable][role="button"]').first()
|
||||
await row.waitFor({ timeout: 10_000 })
|
||||
expect(await row.getAttribute('aria-expanded')).toBe('false')
|
||||
await row.click()
|
||||
await expect.poll(() => row.getAttribute('aria-expanded'), { timeout: 5_000 }).toBe('true')
|
||||
// The expanded body renders the recorded tool result (a.txt's contents).
|
||||
await expect.poll(() => page.getByText('alpha', { exact: false }).count(), { timeout: 5_000 }).toBeGreaterThan(0)
|
||||
await row.click()
|
||||
await expect.poll(() => row.getAttribute('aria-expanded'), { timeout: 5_000 }).toBe('false')
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('issued zero model calls and stayed clean', async () => {
|
||||
// No replay fixture was installed and the llm seam is open — any stray
|
||||
// stream would have failed the turn loudly. Cleanliness pins the wire.
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
expect(harness.serverErrors).toEqual([])
|
||||
expect(scaffold.serverErrors).toEqual([])
|
||||
await assertFixtureInventory(SNAPSHOT_DIR, ['seed.jsonl', 'ui.expected.md'])
|
||||
})
|
||||
})
|
||||
@@ -17,12 +17,12 @@
|
||||
"src",
|
||||
"tests"
|
||||
],
|
||||
// The web e2e lane (harness + replay specs) boots the host spine and reads
|
||||
// The web e2e lane (scaffold + replay specs) boots the host spine and reads
|
||||
// its Context merges — host-plane programs, checked in tsconfig.host.json;
|
||||
// this client-registered project must not also hold them (one program
|
||||
// cannot see both sides of the cordis Context merges).
|
||||
"exclude": [
|
||||
"tests/harness.ts",
|
||||
"tests/scaffold.ts",
|
||||
"tests/replay-round-trip.e2e.ts",
|
||||
"tests/seeded-history.e2e.ts"
|
||||
],
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Host runtime assembly for `dsh`: `bootHost` composes the core plugin spine (LLM service + DeepSeek adapter, sessions with JSONL persistence and immediate fallback titles, optional first-message model summaries, system prompt, tools, agents, agent loop, workspace instructions, local bash, and the provider-neutral user-interaction service), `createApiProxy` implements the [`dsh-host-apiproxy`](../apiproxy/README.md) contract over that composition, and `startHost` is the one-step shell seam returning `{ api, handler, defaults, ctx, dispose }`.
|
||||
|
||||
Which plugins mount and with what defaults is decided only here — shells must not `ctx.plugin` to alter the assembly. `RunningHost.ctx` is a formal seam with exactly three sanctioned uses: mounting protocol front-door plugins (e.g. a future `dsh acp`), headless session-event subscription, and filling a capability seam the boot options deliberately left open (`llm: false` → the embedder installs its own LLM backend, e.g. the keyless web e2e harness's replay); consuming clients must not bypass `api` through it.
|
||||
Which plugins mount and with what defaults is decided only here — shells must not `ctx.plugin` to alter the assembly. `RunningHost.ctx` is a formal seam with exactly three sanctioned uses: mounting protocol front-door plugins (e.g. a future `dsh acp`), headless session-event subscription, and filling a capability seam the boot options deliberately left open (`llm: false` → the embedder installs its own LLM backend, e.g. the keyless web e2e scaffold's replay); consuming clients must not bypass `api` through it.
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ export interface BootHostOptions {
|
||||
* LLM adapter selection: `'deepseek'` (default) mounts the DeepSeek adapter
|
||||
* (requires an API key at load), `false` mounts no adapter and leaves the
|
||||
* `llm` capability seam open for the embedder to fill on the returned ctx
|
||||
* (e.g. the keyless web e2e harness installing a replay backend). With
|
||||
* (e.g. the keyless web e2e scaffold installing a replay backend). With
|
||||
* `false` and nothing filled, the first stream fails loud with NO_ADAPTER —
|
||||
* the earliest resolvable point for an open capability seam.
|
||||
*/
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@
|
||||
"rewriteRelativeImportExtensions": false
|
||||
},
|
||||
"include": [
|
||||
"apps/web/tests/harness.ts",
|
||||
"apps/web/tests/scaffold.ts",
|
||||
"apps/web/tests/support.ts",
|
||||
"apps/web/tests/replay-round-trip.e2e.ts",
|
||||
"apps/web/tests/seeded-history.e2e.ts",
|
||||
|
||||
Reference in New Issue
Block a user