test(web): expand long-chat regression and performance coverage
This commit is contained in:
kingwl
imccyu
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: e9559aa1f6fba89a99d91e79026552511b337793
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: 5f2047125e38f6e1eecf4b8b81d6f5da16686afa
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: f8519a9622d2f7216226a695db95dbebdbf24ea1
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: 294f3e840e0242d9a0d9c53ac510d44d3b0d100f
|
||||
@@ -24,7 +24,7 @@ Keyless model displacement is the disabled adapter row plus `installLlmReplay` f
|
||||
|
||||
### Determinism rules
|
||||
|
||||
The barrier stack for replay-mode browser assertions is, in order: (1) host-side `await agent.whenIdle()` under a timeout, keyed off the in-process `turn/end` — the idle flip follows the persistence flush, so one await covers turn completion and durability; (2) browser settled poll (streaming detached, final text visible). Record-mode log harvest runs after `whenIdle()` and before scaffold disposal while the live session remains available. An in-process `turn/end` listener alone is a wrong barrier (it fires before the SSE frame reaches the browser and before the fsync); file polling is banned (slow on NFS, superseded by `whenIdle`); `networkidle` is banned outright (never resolves while an SSE stream is open).
|
||||
The barrier stack for replay-mode browser assertions is, in order: (1) host-side `await agent.whenIdle()` under a timeout, keyed off the in-process `turn/end` — the idle flip follows the persistence flush, so one await covers turn completion and durability; (2) browser settled poll (streaming detached, final text visible). Record-mode log harvest runs after `whenIdle()` and before scaffold disposal while the live session remains available. An in-process `turn/end` listener alone is a wrong barrier (it fires before the SSE frame reaches the browser and before the fsync); polling persistence files as a turn-completion or durability barrier is banned (slow on NFS, superseded by `whenIdle`), while a tool-controlled temp readiness marker may be polled only as an interaction gate before that completion barrier; `networkidle` is banned outright (never resolves while an SSE stream is open). Navigation assertions arm both initial `session.list` and `workspace.list` responses before page load, then wait for the seeded DOM projection; the mounted shell alone is not readiness because late bootstrap can replace controlled state.
|
||||
|
||||
No single-shot transient-DOM assertions: every hop from replay yield to React commit can coalesce chunks, so sampling `[data-streaming]` is a race by construction. Streaming incrementality is asserted from the persisted `assistant/chunk` events (model-visible ⟺ logged makes the log the authoritative proof). `dsh-llm-replay`'s opt-in `paceMs` (default absent = burst) is a realism knob so the browser observes genuinely incremental SSE; correctness never leans on it, and abort during a pace wait cancels promptly.
|
||||
|
||||
@@ -42,13 +42,13 @@ The typecheck plane split is structural: the host scaffold, its support module,
|
||||
|
||||
### Coverage contract
|
||||
|
||||
The lane covers three behavior families. Live-turn scenarios pin ordinary tool execution, cancellation, non-retryable failure, transient retry, resident questions, and mid-turn steering; synchronization uses durable events, `whenIdle()`, or an explicit replay marker rather than delays. Cold-history scenarios seed through the real persistence API and cover history rendering, sidebar search, trajectory and waterfall views, and tool details without model calls. Browser-lifecycle scenarios cover first-send workspace materialization, reload recovery, layout reset, theme and locale preferences, and workspace create/rename/view operations. Each family asserts the browser surface and the authoritative host state; a stray model call or under-consumed fixture fails teardown.
|
||||
The lane covers three behavior families. Live-turn scenarios pin ordinary tool execution, cancellation, non-retryable failure, transient retry, resident questions, and mid-turn steering; synchronization uses durable events, `whenIdle()`, or an explicit replay marker rather than delays. Cold-history scenarios seed through the real persistence API and cover history rendering, sidebar search, trajectory and waterfall views, and tool details without model calls. Browser-lifecycle scenarios cover first-send workspace materialization, reload recovery, layout reset, theme and locale preferences, and workspace create/rename/view operations. Each family asserts the browser surface and the authoritative host state; a stray model call or under-consumed fixture fails teardown. The required lane additionally carries an 88-turn synthetic Chat scroll contract mixing wrapped Markdown, fenced code, and paired bash calls/results. Real wheel, composer, tool, tab, session, and viewport interactions assert a named settled row's top relative to the transcript scrollport and distance from the true bottom across concurrent history prepend plus paced streaming, pinned/away streaming, tool-disclosure offscreen cycles, expanded-history view/session remount, width reflow, immediate pinned remount, composer resize, and textarea wheel chaining; it deliberately pins neither DOM cardinality nor absolute `scrollTop`, so the same contract can qualify a virtualized implementation. A separate interaction contract over the same fixture pins heterogeneous-row order, independent adjacent tool disclosure, exact user-message clipboard content, a turn-bounded message fork, source/child isolation, and a real follow-up turn in the child; wheel input only navigates to semantic targets and carries no geometry expectation. A short live-history contract starts from a blank workspace and drives consecutive composer turns, including real bash call/result rounds and a paced long final response, pinning one session identity, exact per-turn event ownership, browser echo uniqueness, and composer recovery without timing thresholds.
|
||||
|
||||
### CI stance
|
||||
|
||||
The lane is a required compare-only gate for Linux pull requests under the [browser snapshot CI decision](2026-07-30-web-browser-snapshot-ci-gate.md). The `node 24 / snapshots and artifacts` consumer job owns the [single Linux build](../process/2026-07-30-independent-ci-consumer-build.md), installs the lockfile-selected Chromium, restores its OS-and-lockfile-keyed cache, and runs the lane with `DSH_SNAPSHOT=replay`. This is an intentional plane split: the host and specs use the [tsx source-launch contract](../architecture/2026-07-29-dsh-source-launch-tsx-esm.md), while the browser consumes `apps/web/dist` and package `lib/client.js` artifacts, so the gate depends on `built-package-invariants` for those client artifacts. The hosted and self-hosted default-branch Linux serial jobs run the same gate; the hosted job produces the browser cache consumed by pull requests, while the persistent self-hosted pool needs no hosted cache. CI never records or refreshes goldens. Scenarios remain POSIX-oriented and stay outside the Windows and macOS matrices.
|
||||
|
||||
High-cardinality performance diagnostics use the separate opt-in `apps/web/tests/**/*.perf.ts` inventory selected only by `vitest.web.perf.config.ts`. The isolated `complex-history.perf.ts` cases reuse the real scaffold: the workspace case seeds 1,000 compact sessions plus one 500-turn history containing 500 tool calls, exhausts and remounts that history in Chat, and reports Chromium main-thread, DOM, listener, heap, paging, search, and Trajectory measurements. Two continuation cases seed the same long history but compare the default 24-turn Chat window with all 500 turns expanded before each continues eight identical turns through the real composer, agent loop, SSE wire, tools, and persistence; two turns execute a real `bash` call and assert its durable result, while the final turn fills an 8,232-character mixed-language prompt and replays 120 paced text deltas. A separate soak case starts from a blank session, drives 100 consecutive real composer turns with a `bash` call and result every tenth turn, forces GC every ten turns, and reports ten-turn latency windows plus retained browser state. Per-turn diagnostics cover composer fill, click-to-user-echo, click-to-first-chunk, completion, browser mutations, persisted chunks, and tool events; the synthetic replay model has enough context capacity to keep fixture cardinality stable instead of consuming scripted calls through compaction. Structural assertions pin the intended load, stream, and tool shapes, but timing remains threshold-free because machine speed is not a correctness contract. The required `vitest.web.config.ts` inventory remains limited to `*.e2e.ts` and `*.snapshot.ts`, so neither `test:web:built` nor its CI gate collects performance cases.
|
||||
High-cardinality performance diagnostics use the separate opt-in `apps/web/tests/**/*.perf.ts` inventory selected only by `vitest.web.perf.config.ts`. The isolated `complex-history.perf.ts` cases reuse the real scaffold: the workspace case seeds 1,000 compact sessions plus one 500-turn history containing 500 tool calls, exhausts and remounts that history in Chat, and reports Chromium main-thread, DOM, listener, heap, paging, search, and Trajectory measurements. Two continuation cases seed the same long history but compare the default 24-turn Chat window with all 500 turns expanded before each continues eight identical turns through the real composer, agent loop, SSE wire, tools, and persistence; two turns execute a real `bash` call and assert its durable result, while the final turn fills an 8,232-character mixed-language prompt and replays 120 paced text deltas. A separate soak case starts from a blank session, drives 100 consecutive real composer turns with a `bash` call and result every tenth turn, forces GC every ten turns, and reports ten-turn latency windows plus retained browser state. It then submits a 101st text-only turn with a trusted browser click and measures browser-clock send-to-transcript-DOM and send-to-post-paint latency, excluding the composer's draft mirrors, separately from full-turn completion. Per-turn diagnostics cover composer fill, click-to-user-echo, click-to-first-chunk, completion, browser mutations, persisted chunks, and tool events; the synthetic replay model has enough context capacity to keep fixture cardinality stable instead of consuming scripted calls through compaction. Structural assertions pin the intended load, stream, and tool shapes, but timing remains threshold-free because machine speed is not a correctness contract. The required `vitest.web.config.ts` inventory remains limited to `*.e2e.ts` and `*.snapshot.ts`, so neither `test:web:built` nor its CI gate collects performance cases.
|
||||
|
||||
## Prior art
|
||||
|
||||
@@ -88,6 +88,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
- **Follow-up-prompt-after-resume scenario**: the history/live stitch path over the real wire; add as its own scenario when that code changes or regresses.
|
||||
- **Composer steering gesture**: the input locks while running (stop-or-wait), so the steering scenario steers over the wire from the page; `TODO(web-steer-composer)` upgrades the drive step to a real composer gesture when the product grows one.
|
||||
- **Drag session reorder**: `workspace.insertSessionBefore` has no browser scenario; it needs two sessions materialized in one workspace plus synthesized HTML5 drag events. Add it when that surface changes or regresses. The inert session Rename/Fork/Delete and workspace Delete menu rows get scenarios when they gain behavior.
|
||||
- **Long-history Chat-to-Trajectory Inspect**: the independent inspection source exhausts history after the view opens, while the selected record is addressed by a derived table index that can move as older pages prepend. Short-history Inspect remains covered; the long-history interaction contract excludes this handoff until selection has a stable semantic identity.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
### 确定性规则
|
||||
|
||||
回放模式下浏览器断言的屏障栈，按序：（1）host 侧 `await agent.whenIdle()` 加超时，以进程内 `turn/end` 为锚——空闲翻转发生在持久化落盘之后，一次等待同时覆盖轮次完成与持久性；（2）浏览器安定轮询（流式输出节点已卸载、最终文本可见）。录制模式下，日志采收在 `whenIdle()` 之后、scaffold 释放之前进行，此时运行中的会话仍然可用。单独监听进程内 `turn/end` 是错误屏障（它先于 SSE 帧到达浏览器、先于 fsync 触发）；文件轮询被禁止（NFS 上慢，且被 `whenIdle` 取代）；`networkidle` 被彻底禁止（SSE 流保持打开时它永不解析）。
|
||||
回放模式下浏览器断言的屏障栈，按序：（1）host 侧 `await agent.whenIdle()` 加超时，以进程内 `turn/end` 为锚——空闲翻转发生在持久化落盘之后，一次等待同时覆盖轮次完成与持久性；（2）浏览器安定轮询（流式输出节点已卸载、最终文本可见）。录制模式下，日志采收在 `whenIdle()` 之后、scaffold 释放之前进行，此时运行中的会话仍然可用。单独监听进程内 `turn/end` 是错误屏障（它先于 SSE 帧到达浏览器、先于 fsync 触发）；禁止轮询持久化文件来充当轮次完成或持久性屏障（NFS 上慢，且被 `whenIdle` 取代），但工具控制的临时就绪标记可以仅作为该完成屏障之前的交互门控进行轮询；`networkidle` 被彻底禁止（SSE 流保持打开时它永不解析）。导航断言会在页面加载前同时监听 `session.list` 和 `workspace.list` 的初始响应，随后等待播种数据投影到 DOM；仅凭 shell 已挂载不能判定就绪，因为较晚完成的 bootstrap 可能替换受控状态。
|
||||
|
||||
不做单次瞬态 DOM 断言：从回放产出到 React 提交的每一跳都可能合并分片，采样 `[data-streaming]` 天然就是竞态。流式输出的增量性由持久化的 `assistant/chunk` 事件断言（模型可见 ⟺ 已记录，使日志成为权威证据）。`dsh-llm-replay` 的可选 `paceMs`（默认缺省 = 突发）只是让浏览器观察到真正增量 SSE 的真实感旋钮；正确性绝不依赖它，且节奏等待期间中止会即时取消。
|
||||
|
||||
@@ -42,13 +42,13 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
### 覆盖契约
|
||||
|
||||
该车道覆盖三类行为。实时轮次场景钉住普通工具执行、取消、不可重试失败、瞬态重试、常驻提问与轮次中途 steering；同步依赖持久事件、`whenIdle()` 或显式回放标记，而不使用延时。冷历史场景通过真实持久化 API 播种，在不调用模型的情况下覆盖历史渲染、侧栏搜索、Trajectory 与 Waterfall 视图及工具详情。浏览器生命周期场景覆盖首次发送时物化工作区、重新加载恢复、布局重置、主题与语言偏好，以及工作区的创建、重命名和视图操作。每类场景都断言浏览器表面和权威的 host 状态；离群的模型调用或未耗尽的 fixture 会使拆卸失败。
|
||||
该车道覆盖三类行为。实时轮次场景钉住普通工具执行、取消、不可重试失败、瞬态重试、常驻提问与轮次中途 steering；同步依赖持久事件、`whenIdle()` 或显式回放标记，而不使用延时。冷历史场景通过真实持久化 API 播种，在不调用模型的情况下覆盖历史渲染、侧栏搜索、Trajectory 与 Waterfall 视图及工具详情。浏览器生命周期场景覆盖首次发送时物化工作区、重新加载恢复、布局重置、主题与语言偏好，以及工作区的创建、重命名和视图操作。每类场景都断言浏览器表面和权威的 host 状态；离群的模型调用或未耗尽的 fixture 会使拆卸失败。必需车道还包含一份合成的 88 轮 Chat 滚动契约，其中混合了换行 Markdown、围栏代码以及成对的 bash 调用/结果。真实 wheel、输入框、工具、tab、会话与 viewport 交互会在并发历史前插加带节奏流式输出、贴底/离底流式输出、工具 disclosure 离屏循环、扩展历史后的视图/会话重新挂载、宽度重排、贴底后立即重新挂载、输入框尺寸变化以及 textarea wheel 链场景中，断言一个具名已结算行相对 transcript scrollport 的顶部位置和到真实底部的距离；它刻意不钉 DOM 基数或绝对 `scrollTop`，因此同一契约可以验收虚拟化实现。另一份基于同一 fixture 的交互契约钉住异构行顺序、相邻工具 disclosure 的独立状态、用户消息剪贴板内容的精确值、以轮次为边界的消息 fork、源会话/子会话隔离，以及子会话中的一次真实追问轮次；wheel 输入只用于导航到语义目标，不承载几何预期。一份简短的实时历史契约从空白工作区开始，连续驱动输入框轮次，其中包括真实的 bash 调用/结果轮次和一段带节奏的长篇最终响应；它钉住单一会话身份、每轮事件的精确归属、浏览器回显唯一性与输入框恢复，不设置时间阈值。
|
||||
|
||||
### CI 立场
|
||||
|
||||
根据[浏览器快照 CI 决策](2026-07-30-web-browser-snapshot-ci-gate.md)，该车道是 Linux 拉取请求必需的只比较门禁。`node 24 / snapshots and artifacts` 消费方任务在[消费方独立构建](../process/2026-07-30-independent-ci-consumer-build.md)中负责唯一一次 Linux 构建，安装锁文件选定的 Chromium，恢复以操作系统和锁文件为键的缓存，并用 `DSH_SNAPSHOT=replay` 运行该车道。这是有意的平面切分：host 与 spec 使用 [tsx 源码启动契约](../architecture/2026-07-29-dsh-source-launch-tsx-esm.md)，浏览器则消费 `apps/web/dist` 和包的 `lib/client.js` 产物，因此门禁依赖 `built-package-invariants` 提供这些客户端产物。托管和自托管的默认分支 Linux 串行任务运行同一门禁；托管任务生成供 PR 消费的浏览器缓存，持久化自托管池则不需要托管侧缓存。CI 从不录制或刷新预期输出。场景仍面向 POSIX，并继续置于 Windows 和 macOS 矩阵之外。
|
||||
|
||||
高基数性能诊断使用单独按需启用的 `apps/web/tests/**/*.perf.ts` 清单，并且只由 `vitest.web.perf.config.ts` 选中。`complex-history.perf.ts` 的隔离用例复用真实 scaffold：工作区用例播种 1,000 个紧凑会话以及一份包含 500 次工具调用的 500 轮次历史，在 Chat 中穷尽并重新挂载该历史，并报告 Chromium 主线程、DOM、监听器、堆内存、分页、搜索和 Trajectory 测量结果。两个续聊用例播种同一份长历史，但比较默认的 24 轮次 Chat 窗口与展开全部 500 轮次的状态，然后各自通过真实输入框、agent loop、SSE wire、工具和持久化继续进行 8 个相同轮次；其中两轮执行真实 `bash` 调用并断言其持久化结果，最后一轮则填入一条包含 8,232 个字符的混合语言提示词，并回放 120 个带节奏的文本增量。一个单独的 soak 用例从空白会话开始，通过真实输入框连续驱动 100 轮，每第 10 轮执行一次 `bash` 调用并产生结果，每 10 轮强制执行一次 GC，并报告每 10 轮的延迟窗口及保留的浏览器状态。逐轮诊断涵盖输入框填入、点击到用户消息回显、点击到首个分片、完成、浏览器变更、持久化分片和工具事件；合成回放模型拥有足够的上下文容量，可使 fixture 基数保持稳定，而不会因压缩（compaction）消耗脚本化调用。结构性断言钉住预期的负载、流和工具形状，但时间仍不设阈值，因为机器速度不属于正确性契约。必需的 `vitest.web.config.ts` 清单仍仅限 `*.e2e.ts` 和 `*.snapshot.ts`，因此 `test:web:built` 及其 CI 门禁都不会收集性能用例。
|
||||
高基数性能诊断使用单独按需启用的 `apps/web/tests/**/*.perf.ts` 清单，并且只由 `vitest.web.perf.config.ts` 选中。`complex-history.perf.ts` 的隔离用例复用真实 scaffold：工作区用例播种 1,000 个紧凑会话以及一份包含 500 次工具调用的 500 轮次历史，在 Chat 中穷尽并重新挂载该历史，并报告 Chromium 主线程、DOM、监听器、堆内存、分页、搜索和 Trajectory 测量结果。两个续聊用例播种同一份长历史，但比较默认的 24 轮次 Chat 窗口与展开全部 500 轮次的状态，然后各自通过真实输入框、agent loop、SSE wire、工具和持久化继续进行 8 个相同轮次；其中两轮执行真实 `bash` 调用并断言其持久化结果，最后一轮则填入一条包含 8,232 个字符的混合语言提示词，并回放 120 个带节奏的文本增量。一个单独的 soak 用例从空白会话开始，通过真实输入框连续驱动 100 轮，每第 10 轮执行一次 `bash` 调用并产生结果，每 10 轮强制执行一次 GC，并报告每 10 轮的延迟窗口及保留的浏览器状态。随后它通过受信任的浏览器点击提交第 101 个纯文本轮次，并使用浏览器时钟分别测量发送到 transcript DOM 和发送到绘制后的延迟，排除输入框的草稿镜像，并与完整轮次完成时间分开。逐轮诊断涵盖输入框填入、点击到用户消息回显、点击到首个分片、完成、浏览器变更、持久化分片和工具事件；合成回放模型拥有足够的上下文容量，可使 fixture 基数保持稳定，而不会因压缩（compaction）消耗脚本化调用。结构性断言钉住预期的负载、流和工具形状，但时间仍不设阈值，因为机器速度不属于正确性契约。必需的 `vitest.web.config.ts` 清单仍仅限 `*.e2e.ts` 和 `*.snapshot.ts`，因此 `test:web:built` 及其 CI 门禁都不会收集性能用例。
|
||||
|
||||
## 业界先例
|
||||
|
||||
@@ -88,6 +88,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
- **恢复后追问场景**：真实 wire 上的历史/实时缝合路径；当该代码变更或回归时作为独立场景补充。
|
||||
- **输入框 steering 手势**：输入在运行期间锁定（只能停止或等待），因此 steering 场景从页面走 wire 做 steer；`TODO(web-steer-composer)` 待产品长出真实的输入框手势后，把驱动步骤升级为该手势。
|
||||
- **拖拽会话重排**：`workspace.insertSessionBefore` 尚无浏览器场景；它需要在同一个工作区里物化两个会话，并合成 HTML5 拖拽事件。当该表面变更或回归时再补充。无行为的会话 Rename/Fork/Delete 和工作区 Delete 菜单行待获得行为后再补充场景。
|
||||
- **长历史 Chat 到 Trajectory 的 Inspect**：独立的检查数据源会在视图打开后穷尽历史，而所选记录由一个派生的表格索引定位；随着较早页面前插，该索引可能移动。短历史 Inspect 仍有覆盖；在选中项具有稳定的语义身份之前，长历史交互契约不包含这项交接。
|
||||
|
||||
## 后果
|
||||
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
// Web e2e contract for a conversation grown through the real composer rather
|
||||
// than pre-seeded history. Twelve deterministic replay turns exercise repeated
|
||||
// send/settle/render cycles, including two real bash executions and one long,
|
||||
// multi-chunk final turn. Assertions stay semantic: no host timing, heap, or
|
||||
// mounted-row cardinality is treated as a correctness contract.
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import type { Browser, Page } from 'playwright'
|
||||
import { chromium } from 'playwright'
|
||||
import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import { CallId, type StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { ReplayEntry, ReplayOverrideDoc } from '@deepseek-ai/dsh-llm-replay'
|
||||
import type { SessionEvent, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
launchWebScaffold,
|
||||
watchConsole,
|
||||
webSnapshotMode,
|
||||
type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { connectFreshWorkspace, newEnglishPage, saveFailureShot } from './support.ts'
|
||||
|
||||
const MODE = webSnapshotMode()
|
||||
const TURN_COUNT = 12
|
||||
const TOOL_TURNS = [4, 9] as const
|
||||
const STREAM_PACE_MS = 10
|
||||
|
||||
interface TurnSpec {
|
||||
readonly index: number
|
||||
readonly prompt: string
|
||||
readonly userMarker: string
|
||||
readonly firstMarker: string
|
||||
readonly doneMarker: string
|
||||
readonly deltas: readonly string[]
|
||||
readonly callId?: ReturnType<typeof CallId>
|
||||
readonly toolResultMarker?: string
|
||||
}
|
||||
|
||||
function suffix(index: number): string {
|
||||
return String(index).padStart(3, '0')
|
||||
}
|
||||
|
||||
function longFinalPrompt(userMarker: string): string {
|
||||
return [
|
||||
`${userMarker} Reconcile this accumulated conversation without losing earlier turn ownership.`,
|
||||
...Array.from(
|
||||
{ length: 36 },
|
||||
(_, index) => `Context ${String(index + 1).padStart(2, '0')}: preserve token-${String(index)} and verify ${'payload '.repeat(12).trimEnd()}.`,
|
||||
),
|
||||
'Return one continuous response and finish with the requested completion marker.',
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
function turnSpec(index: number): TurnSpec {
|
||||
const id = suffix(index)
|
||||
const userMarker = `CONTINUOUS_CHAT_USER_${id}`
|
||||
const firstMarker = `CONTINUOUS_CHAT_FIRST_${id}`
|
||||
const doneMarker = `CONTINUOUS_CHAT_DONE_${id}`
|
||||
const deltaCount = index === TURN_COUNT ? 36 : 8
|
||||
const deltas = Array.from({ length: deltaCount }, (_, chunkIndex) => {
|
||||
if (chunkIndex === 0) return `${firstMarker} `
|
||||
if (chunkIndex === deltaCount - 1) return `${doneMarker}.`
|
||||
return `turn-${id}-chunk-${String(chunkIndex).padStart(2, '0')} keeps semantic ownership stable. `
|
||||
})
|
||||
if (!TOOL_TURNS.includes(index as (typeof TOOL_TURNS)[number])) {
|
||||
return {
|
||||
index,
|
||||
prompt: index === TURN_COUNT
|
||||
? longFinalPrompt(userMarker)
|
||||
: `${userMarker} Continue this same conversation through turn ${String(index)}.`,
|
||||
userMarker,
|
||||
firstMarker,
|
||||
doneMarker,
|
||||
deltas,
|
||||
}
|
||||
}
|
||||
return {
|
||||
index,
|
||||
prompt: `${userMarker} Run the requested deterministic tool for turn ${String(index)}, then continue.`,
|
||||
userMarker,
|
||||
firstMarker,
|
||||
doneMarker,
|
||||
deltas,
|
||||
callId: CallId(`continuous-chat-tool-${id}`),
|
||||
toolResultMarker: `CONTINUOUS_CHAT_TOOL_RESULT_${id}`,
|
||||
}
|
||||
}
|
||||
|
||||
function textStream(spec: TurnSpec): StreamChunk[] {
|
||||
const response = spec.deltas.join('')
|
||||
return [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
...spec.deltas.map(text => ({ type: 'text-delta' as const, index: 0, text })),
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: response } },
|
||||
{
|
||||
type: 'usage',
|
||||
usage: {
|
||||
inputTokens: Math.ceil(spec.prompt.length / 4),
|
||||
outputTokens: Math.ceil(response.length / 4),
|
||||
},
|
||||
},
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
}
|
||||
|
||||
function toolStream(spec: TurnSpec): StreamChunk[] {
|
||||
if (spec.callId === undefined || spec.toolResultMarker === undefined) {
|
||||
throw new Error(`turn ${String(spec.index)} has no tool identity`)
|
||||
}
|
||||
const args = JSON.stringify({
|
||||
command: `printf '${spec.toolResultMarker}\\n'`,
|
||||
description: spec.toolResultMarker,
|
||||
})
|
||||
return [
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{
|
||||
type: 'tool-call-delta',
|
||||
index: 0,
|
||||
id: spec.callId,
|
||||
name: 'bash',
|
||||
argumentsDelta: args,
|
||||
},
|
||||
{
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: spec.callId, name: 'bash', arguments: args },
|
||||
},
|
||||
{ type: 'usage', usage: { inputTokens: 256, outputTokens: 24 } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
]
|
||||
}
|
||||
|
||||
function replayScript(specs: readonly TurnSpec[]): ReplayOverrideDoc {
|
||||
return specs.flatMap((spec): ReplayEntry[] => {
|
||||
const final: ReplayEntry = { kind: 'chunks', chunks: textStream(spec) }
|
||||
return spec.callId === undefined
|
||||
? [final]
|
||||
: [{ kind: 'chunks', chunks: toolStream(spec) }, final]
|
||||
})
|
||||
}
|
||||
|
||||
function userText(event: Extract<SessionEvent, { type: 'user/message' }>): string {
|
||||
return event.data.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
function assistantText(event: Extract<SessionEvent, { type: 'assistant/message' }>): string {
|
||||
return event.data.message.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
function toolResultText(event: Extract<SessionEvent, { type: 'tool/result' }>): string {
|
||||
return event.data.message.content[0].content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
describe('web e2e: continuous conversation grown through the composer', () => {
|
||||
let browser: Browser
|
||||
let page: Page
|
||||
let replayDir: string
|
||||
let scaffold: WebScaffold
|
||||
let tripwire: ReturnType<typeof watchConsole>
|
||||
const consoleWarnings: string[] = []
|
||||
const sessionEvents: SessionEvent[] = []
|
||||
const specs = Array.from({ length: TURN_COUNT }, (_, offset) => turnSpec(offset + 1))
|
||||
|
||||
beforeAll(async () => {
|
||||
replayDir = await mkdtemp(join(tmpdir(), 'dsh-continuous-chat-replay-'))
|
||||
const replayOverride = join(replayDir, 'replay.override.json')
|
||||
await writeFile(replayOverride, JSON.stringify(replayScript(specs)))
|
||||
scaffold = await launchWebScaffold({
|
||||
replayFixture: join(replayDir, 'override-only.jsonl'),
|
||||
replayOverride,
|
||||
replayContextWindow: 10_000_000,
|
||||
paceMs: STREAM_PACE_MS,
|
||||
})
|
||||
scaffold.ctx.on('session/event', (_session, event: SessionEvent) => {
|
||||
sessionEvents.push(event)
|
||||
})
|
||||
browser = await chromium.launch()
|
||||
page = await newEnglishPage(browser, 900)
|
||||
tripwire = watchConsole(page)
|
||||
page.on('console', (message) => {
|
||||
if (message.type() === 'warning') consoleWarnings.push(message.text())
|
||||
})
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
await connectFreshWorkspace(page, 'continuous-chat-e2e')
|
||||
}, 120_000)
|
||||
|
||||
afterAll(async () => {
|
||||
const failures: unknown[] = []
|
||||
await browser?.close().catch((error: unknown) => failures.push(error))
|
||||
await scaffold?.close().catch((error: unknown) => failures.push(error))
|
||||
if (replayDir !== undefined) {
|
||||
await rm(replayDir, { recursive: true, force: true })
|
||||
.catch((error: unknown) => failures.push(error))
|
||||
}
|
||||
if (failures.length === 1) throw failures[0]
|
||||
if (failures.length > 1) throw new AggregateError(failures, 'continuous Chat e2e cleanup failed')
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('keeps twelve generated turns and tool rows bound to one live session', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-chat-continuous-conversation'))
|
||||
const composer = page.locator('textarea:enabled').last()
|
||||
await composer.waitFor({ timeout: 15_000 })
|
||||
let sessionId: SessionId | undefined
|
||||
|
||||
for (const spec of specs) {
|
||||
const eventStart = sessionEvents.length
|
||||
expect(await composer.inputValue()).toBe('')
|
||||
expect(await composer.isEnabled()).toBe(true)
|
||||
await composer.fill(spec.prompt)
|
||||
expect(await composer.inputValue()).toBe(spec.prompt)
|
||||
|
||||
const settled = scaffold.whenTurnSettled(60_000)
|
||||
await page.getByRole('button', { name: 'Send message', exact: true }).click()
|
||||
await page.getByText(spec.userMarker, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
const echoedUser = sessionEvents.slice(eventStart).find(
|
||||
(event): event is SessionEvent<'user/message'> => (
|
||||
event.type === 'user/message'
|
||||
&& event.data.source.kind === 'user'
|
||||
&& userText(event).includes(spec.userMarker)
|
||||
),
|
||||
)
|
||||
if (echoedUser === undefined) throw new Error(`turn ${String(spec.index)} has no user echo event`)
|
||||
const userRow = page.locator(`[data-chat-anchor-key="node:${String(echoedUser.seq)}"]`)
|
||||
await expect.poll(() => userRow.count(), { timeout: 10_000 }).toBe(1)
|
||||
expect(await userRow.getAttribute('data-chat-flow-kind')).toBe('user')
|
||||
expect(await userRow.textContent()).toContain(spec.userMarker)
|
||||
await page.getByText(spec.firstMarker, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
const settledSessionId = await settled
|
||||
if (sessionId === undefined) {
|
||||
sessionId = settledSessionId
|
||||
} else {
|
||||
expect(settledSessionId).toBe(sessionId)
|
||||
}
|
||||
|
||||
await expect.poll(() => page.locator('[data-streaming="true"]').count(), { timeout: 15_000 }).toBe(0)
|
||||
await page.getByText(spec.doneMarker, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
await expect.poll(() => composer.inputValue(), { timeout: 10_000 }).toBe('')
|
||||
await expect.poll(() => composer.isEnabled(), { timeout: 10_000 }).toBe(true)
|
||||
|
||||
const turnEvents = sessionEvents.slice(eventStart)
|
||||
const turnStarts = turnEvents.filter((event): event is SessionEvent<'turn/start'> => (
|
||||
event.type === 'turn/start'
|
||||
))
|
||||
const users = turnEvents.filter((event): event is SessionEvent<'user/message'> => (
|
||||
event.type === 'user/message' && event.data.source.kind === 'user'
|
||||
))
|
||||
const assistants = turnEvents.filter((event): event is SessionEvent<'assistant/message'> => (
|
||||
event.type === 'assistant/message'
|
||||
))
|
||||
const finalAssistants = assistants.filter(event => assistantText(event).includes(spec.doneMarker))
|
||||
const turnEnds = turnEvents.filter((event): event is SessionEvent<'turn/end'> => (
|
||||
event.type === 'turn/end'
|
||||
))
|
||||
const chunks = turnEvents.filter(event => event.type === 'assistant/chunk')
|
||||
|
||||
expect(turnStarts).toHaveLength(1)
|
||||
expect(turnStarts[0]?.data.turn).toBe(spec.index)
|
||||
expect(users).toHaveLength(1)
|
||||
expect(users[0]?.seq).toBe(echoedUser.seq)
|
||||
expect(userText(users[0]!)).toBe(spec.prompt)
|
||||
expect(finalAssistants).toHaveLength(1)
|
||||
expect(assistants).toHaveLength(spec.callId === undefined ? 1 : 2)
|
||||
expect(turnEnds).toHaveLength(1)
|
||||
expect(turnEnds[0]?.data).toEqual({ turn: spec.index, reason: { kind: 'completed' } })
|
||||
expect(chunks).toHaveLength(spec.deltas.length + (spec.callId === undefined ? 4 : 9))
|
||||
|
||||
const assistantRow = page.locator(`[data-chat-anchor-key="node:${String(finalAssistants[0]!.seq)}"]`)
|
||||
await expect.poll(() => assistantRow.count(), { timeout: 10_000 }).toBe(1)
|
||||
expect(await assistantRow.getAttribute('data-chat-flow-kind')).toBe('assistant')
|
||||
expect(await assistantRow.textContent()).toContain(spec.doneMarker)
|
||||
|
||||
const calls = turnEvents.filter((event): event is SessionEvent<'tool/call'> => event.type === 'tool/call')
|
||||
const results = turnEvents.filter((event): event is SessionEvent<'tool/result'> => event.type === 'tool/result')
|
||||
if (spec.callId === undefined || spec.toolResultMarker === undefined) {
|
||||
expect(calls).toHaveLength(0)
|
||||
expect(results).toHaveLength(0)
|
||||
continue
|
||||
}
|
||||
|
||||
expect(calls).toHaveLength(1)
|
||||
expect(results).toHaveLength(1)
|
||||
expect(calls[0]?.data).toMatchObject({
|
||||
turn: spec.index,
|
||||
callId: spec.callId,
|
||||
name: 'bash',
|
||||
})
|
||||
expect(results[0]?.data.turn).toBe(spec.index)
|
||||
expect(results[0]?.data.message.source.callId).toBe(spec.callId)
|
||||
expect(results[0]?.data.message.content[0].isError).toBe(false)
|
||||
expect(toolResultText(results[0]!)).toBe(`${spec.toolResultMarker}\n`)
|
||||
|
||||
const toolRow = page.locator(`[data-chat-call-id="${spec.callId}"]`)
|
||||
await expect.poll(() => toolRow.count(), { timeout: 10_000 }).toBe(1)
|
||||
expect(await toolRow.textContent()).toContain(spec.toolResultMarker)
|
||||
const disclosure = toolRow.locator('[data-sample="bash-global"]')
|
||||
expect(await disclosure.getAttribute('aria-expanded')).toBe('false')
|
||||
await disclosure.click()
|
||||
await expect.poll(() => disclosure.getAttribute('aria-expanded'), { timeout: 10_000 }).toBe('true')
|
||||
// The collapsed summary deliberately repeats the result marker; the
|
||||
// last exact match is the expanded terminal output owned by this call.
|
||||
await toolRow.getByText(spec.toolResultMarker, { exact: true }).last().waitFor({ timeout: 10_000 })
|
||||
await disclosure.click()
|
||||
await expect.poll(() => disclosure.getAttribute('aria-expanded'), { timeout: 10_000 }).toBe('false')
|
||||
}
|
||||
|
||||
if (sessionId === undefined) throw new Error('continuous conversation completed no turn')
|
||||
expect(scaffold.ctx.agents.get(sessionId)?.session.events.filter(event => (
|
||||
event.type === 'turn/end' && event.data.reason.kind === 'completed'
|
||||
))).toHaveLength(TURN_COUNT)
|
||||
expect(specs.at(-1)?.prompt.length).toBeGreaterThan(4_000)
|
||||
expect(sessionEvents.filter(event => (
|
||||
event.type === 'assistant/chunk' && event.data.turn === TURN_COUNT
|
||||
)).length).toBeGreaterThan(30)
|
||||
expect(consoleWarnings).toEqual([])
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
}, 180_000)
|
||||
})
|
||||
@@ -0,0 +1,268 @@
|
||||
// Long-history Chat behavior contract for a future virtualized renderer. Wheel
|
||||
// input only navigates to the semantic target; assertions pin content identity
|
||||
// and interaction routing rather than scroll geometry or mounted row counts.
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import type { Browser, Page } from 'playwright'
|
||||
import { chromium } from 'playwright'
|
||||
import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { ReplayEntry, ReplayOverrideDoc } from '@deepseek-ai/dsh-llm-replay'
|
||||
import { SessionId, type SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { createChatScrollFixture } from './chat-scroll-fixture.ts'
|
||||
import {
|
||||
launchWebScaffold,
|
||||
seedSession,
|
||||
watchConsole,
|
||||
webSnapshotMode,
|
||||
type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { newEnglishPage, saveFailureShot } from './support.ts'
|
||||
|
||||
const MODE = webSnapshotMode()
|
||||
const SESSION_ID = 'chat-long-interactions-e2e'
|
||||
const FIXTURE_TURNS = 88
|
||||
const TOOL_TURN = FIXTURE_TURNS
|
||||
const BRANCH_TURN = 80
|
||||
const TARGET_CALL_1 = 'chat-scroll-088-1'
|
||||
const TARGET_CALL_2 = 'chat-scroll-088-2'
|
||||
const CONTINUE_PROMPT = 'CHAT_INTERACTION_CONTINUE Continue from this exact branch point.'
|
||||
const CONTINUE_FIRST = 'CHAT_INTERACTION_CONTINUE_FIRST'
|
||||
const CONTINUE_DONE = 'CHAT_INTERACTION_CONTINUE_DONE'
|
||||
const FIXTURE = createChatScrollFixture({
|
||||
markerPrefix: 'INTERACTION',
|
||||
title: 'CHAT_INTERACTION long semantic identity session',
|
||||
turns: FIXTURE_TURNS,
|
||||
})
|
||||
|
||||
function continuationChunks(): StreamChunk[] {
|
||||
const response = `${CONTINUE_FIRST} The fork retained the intended prefix. ${CONTINUE_DONE}.`
|
||||
return [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
{ type: 'text-delta', index: 0, text: `${CONTINUE_FIRST} ` },
|
||||
{ type: 'text-delta', index: 0, text: `The fork retained the intended prefix. ${CONTINUE_DONE}.` },
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: response } },
|
||||
{ type: 'usage', usage: { inputTokens: 512, outputTokens: 32 } },
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
}
|
||||
|
||||
function replayEntry(chunks: StreamChunk[]): ReplayEntry {
|
||||
return { kind: 'chunks', chunks }
|
||||
}
|
||||
|
||||
function carries(event: SessionEvent, marker: string): boolean {
|
||||
return JSON.stringify(event).includes(marker)
|
||||
}
|
||||
|
||||
function textContent(content: readonly unknown[]): string {
|
||||
return content.flatMap((block) => {
|
||||
if (typeof block !== 'object' || block === null) return []
|
||||
const candidate = block as { type?: unknown; text?: unknown }
|
||||
return candidate.type === 'text' && typeof candidate.text === 'string'
|
||||
? [candidate.text]
|
||||
: []
|
||||
}).join('')
|
||||
}
|
||||
|
||||
async function nextPaint(page: Page): Promise<void> {
|
||||
await page.evaluate(async () => {
|
||||
await document.fonts.ready
|
||||
await new Promise<void>(resolve => requestAnimationFrame(() => {
|
||||
requestAnimationFrame(() => { resolve() })
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
async function openSeed(page: Page): Promise<void> {
|
||||
await page.getByText(/^\d+ sessions?$/, { exact: true }).waitFor({ timeout: 30_000 })
|
||||
const search = page.getByRole('textbox', { name: 'Search name, keywords...', exact: true })
|
||||
await search.fill(FIXTURE.markers.user(1))
|
||||
const results = page.getByRole('tree', { name: 'Search results' }).getByRole('treeitem')
|
||||
await results.first().waitFor({ timeout: 60_000 })
|
||||
const resultCount = await results.count()
|
||||
if (resultCount !== 1) throw new Error(`expected one seeded search result, received ${String(resultCount)}`)
|
||||
await results.click()
|
||||
await results.click()
|
||||
await page.getByText(FIXTURE.markers.assistant(FIXTURE.turns), { exact: false })
|
||||
.last().waitFor({ timeout: 30_000 })
|
||||
await nextPaint(page)
|
||||
}
|
||||
|
||||
async function wheelUntilMounted(page: Page, selector: string, deltaY: number): Promise<void> {
|
||||
const scrollport = page.locator('[data-conversation-scroll]')
|
||||
const box = await scrollport.boundingBox()
|
||||
if (box === null) throw new Error('conversation scrollport has no layout box')
|
||||
await page.mouse.move(box.x + box.width / 2, box.y + Math.min(140, box.height / 3))
|
||||
for (let attempt = 0; attempt < 20; attempt += 1) {
|
||||
if (await page.locator(selector).count() > 0) return
|
||||
await page.mouse.wheel(0, deltaY)
|
||||
await nextPaint(page)
|
||||
}
|
||||
throw new Error(`semantic Chat target did not mount: ${selector}`)
|
||||
}
|
||||
|
||||
function requiredEvent<T extends SessionEvent['type']>(
|
||||
events: readonly SessionEvent[],
|
||||
type: T,
|
||||
marker: string,
|
||||
): Extract<SessionEvent, { type: T }> {
|
||||
const event = events.find((candidate): candidate is Extract<SessionEvent, { type: T }> => (
|
||||
candidate.type === type && carries(candidate, marker)
|
||||
))
|
||||
if (event === undefined) throw new Error(`${type} carrying ${marker} is absent`)
|
||||
return event
|
||||
}
|
||||
|
||||
describe('web e2e: long Chat interaction contract', () => {
|
||||
let browser: Browser
|
||||
let page: Page
|
||||
let replayDir: string
|
||||
let scaffold: WebScaffold
|
||||
let tripwire: ReturnType<typeof watchConsole>
|
||||
|
||||
beforeAll(async () => {
|
||||
replayDir = await mkdtemp(join(tmpdir(), 'dsh-chat-interaction-replay-'))
|
||||
const replayOverride = join(replayDir, 'replay.override.json')
|
||||
const replay: ReplayOverrideDoc = [replayEntry(continuationChunks())]
|
||||
await writeFile(replayOverride, JSON.stringify(replay))
|
||||
scaffold = await launchWebScaffold({
|
||||
replayFixture: join(replayDir, 'override-only.jsonl'),
|
||||
replayOverride,
|
||||
replayContextWindow: 10_000_000,
|
||||
paceMs: 18,
|
||||
})
|
||||
await seedSession(scaffold, FIXTURE.log, SESSION_ID)
|
||||
browser = await chromium.launch()
|
||||
page = await newEnglishPage(browser, 900)
|
||||
tripwire = watchConsole(page)
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
await openSeed(page)
|
||||
}, 120_000)
|
||||
|
||||
afterAll(async () => {
|
||||
const failures: unknown[] = []
|
||||
await browser?.close().catch((error: unknown) => failures.push(error))
|
||||
await scaffold?.close().catch((error: unknown) => failures.push(error))
|
||||
if (replayDir !== undefined) {
|
||||
await rm(replayDir, { recursive: true, force: true })
|
||||
.catch((error: unknown) => failures.push(error))
|
||||
}
|
||||
if (failures.length === 1) throw failures[0]
|
||||
if (failures.length > 1) throw new AggregateError(failures, 'long Chat interaction cleanup failed')
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('keeps heterogeneous rows and their actions bound to exact semantic identities', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-chat-long-interactions'))
|
||||
const source = scaffold.ctx.agents.get(SessionId(SESSION_ID))
|
||||
if (source === undefined) throw new Error('seeded long-history agent is not attached')
|
||||
|
||||
const toolUserMarker = FIXTURE.markers.user(TOOL_TURN)
|
||||
const toolAssistantMarker = FIXTURE.markers.assistant(TOOL_TURN)
|
||||
const toolMarker1 = FIXTURE.markers.tool(TOOL_TURN, 1)
|
||||
const toolMarker2 = FIXTURE.markers.tool(TOOL_TURN, 2)
|
||||
const toolUserEvent = requiredEvent(source.session.events, 'user/message', toolUserMarker)
|
||||
const toolAssistantEvent = requiredEvent(source.session.events, 'assistant/message', toolAssistantMarker)
|
||||
const branchUserMarker = FIXTURE.markers.user(BRANCH_TURN)
|
||||
const branchAssistantMarker = FIXTURE.markers.assistant(BRANCH_TURN)
|
||||
const branchUserEvent = requiredEvent(source.session.events, 'user/message', branchUserMarker)
|
||||
const boundary = source.session.events.find((event): event is SessionEvent<'turn/end'> => (
|
||||
event.type === 'turn/end' && event.data.turn === BRANCH_TURN
|
||||
))
|
||||
if (boundary === undefined) throw new Error(`turn ${String(BRANCH_TURN)} has no completed boundary`)
|
||||
const expectedUserText = textContent(branchUserEvent.data.content)
|
||||
|
||||
await wheelUntilMounted(page, `[data-chat-call-id="${TARGET_CALL_2}"]`, -1_100)
|
||||
const toolUserRow = page.locator(`[data-chat-anchor-key="node:${String(toolUserEvent.seq)}"]`)
|
||||
const toolAssistantRow = page.locator(`[data-chat-anchor-key="node:${String(toolAssistantEvent.seq)}"]`)
|
||||
const call1 = page.locator(`[data-chat-call-id="${TARGET_CALL_1}"]`)
|
||||
const call2 = page.locator(`[data-chat-call-id="${TARGET_CALL_2}"]`)
|
||||
|
||||
await expect.poll(() => toolUserRow.count(), { timeout: 10_000 }).toBe(1)
|
||||
await expect.poll(() => toolAssistantRow.count(), { timeout: 10_000 }).toBe(1)
|
||||
expect(await call1.count()).toBe(1)
|
||||
expect(await call2.count()).toBe(1)
|
||||
expect(await toolUserRow.getAttribute('data-chat-flow-kind')).toBe('user')
|
||||
expect(await toolAssistantRow.getAttribute('data-chat-flow-kind')).toBe('assistant')
|
||||
expect(await toolUserRow.textContent()).toContain(toolUserMarker)
|
||||
expect(await toolAssistantRow.textContent()).toContain(toolAssistantMarker)
|
||||
expect(await call1.textContent()).toContain(toolMarker1)
|
||||
expect(await call2.textContent()).toContain(toolMarker2)
|
||||
|
||||
const expectedOrder = [
|
||||
`node:${String(toolUserEvent.seq)}`,
|
||||
`call:${TARGET_CALL_1}`,
|
||||
`call:${TARGET_CALL_2}`,
|
||||
`node:${String(toolAssistantEvent.seq)}`,
|
||||
]
|
||||
const actualOrder = await page.locator('[data-chat-anchor-key]').evaluateAll((rows, keys) => (
|
||||
rows.map(row => (row as HTMLElement).dataset.chatAnchorKey)
|
||||
.filter((key): key is string => key !== undefined && keys.includes(key))
|
||||
), expectedOrder)
|
||||
expect(actualOrder).toEqual(expectedOrder)
|
||||
const groupKeys = await Promise.all([call1, call2].map(row => row.evaluate(element => (
|
||||
element.closest<HTMLElement>('[data-chat-flow-kind="tool-group"]')?.dataset.chatFlowKey ?? null
|
||||
))))
|
||||
expect(groupKeys[0]).not.toBeNull()
|
||||
expect(groupKeys[1]).toBe(groupKeys[0])
|
||||
|
||||
const summary1 = call1.locator('[data-sample="bash-global"]')
|
||||
const summary2 = call2.locator('[data-sample="bash-global"]')
|
||||
expect(await summary1.getAttribute('aria-expanded')).toBe('false')
|
||||
expect(await summary2.getAttribute('aria-expanded')).toBe('false')
|
||||
await summary2.focus()
|
||||
await summary2.press('Enter')
|
||||
await expect.poll(() => summary2.getAttribute('aria-expanded'), { timeout: 10_000 }).toBe('true')
|
||||
expect(await summary1.getAttribute('aria-expanded')).toBe('false')
|
||||
await call2.getByText(`${toolMarker2} output line 12`, { exact: true }).waitFor({ timeout: 10_000 })
|
||||
|
||||
await wheelUntilMounted(page, `[data-chat-anchor-key="node:${String(branchUserEvent.seq)}"]`, -1_100)
|
||||
const userRow = page.locator(`[data-chat-anchor-key="node:${String(branchUserEvent.seq)}"]`)
|
||||
expect(await userRow.textContent()).toContain(branchUserMarker)
|
||||
await page.context().grantPermissions(['clipboard-read', 'clipboard-write'])
|
||||
await userRow.hover()
|
||||
await userRow.getByRole('button', { name: 'Copy', exact: true }).click()
|
||||
await expect.poll(() => page.evaluate(() => navigator.clipboard.readText()), { timeout: 5_000 })
|
||||
.toBe(expectedUserText)
|
||||
|
||||
await userRow.hover()
|
||||
await userRow.getByRole('button', { name: 'Branch into a new conversation', exact: true }).click()
|
||||
await expect.poll(
|
||||
() => scaffold.ctx.agents.list().find(agent => agent.session.header.parentSession === SessionId(SESSION_ID)),
|
||||
{ timeout: 15_000 },
|
||||
).toBeDefined()
|
||||
const child = scaffold.ctx.agents.list()
|
||||
.find(agent => agent.session.header.parentSession === SessionId(SESSION_ID))
|
||||
if (child === undefined) throw new Error('message branch did not create a child session')
|
||||
expect(child.session.header.seedLength).toBe(boundary.seq + 1)
|
||||
expect(child.session.events.some(event => carries(event, branchAssistantMarker))).toBe(true)
|
||||
expect(child.session.events.some(event => carries(event, FIXTURE.markers.user(BRANCH_TURN + 1)))).toBe(false)
|
||||
expect(child.session.events.some(event => carries(event, FIXTURE.markers.user(FIXTURE.turns)))).toBe(false)
|
||||
|
||||
const currentCrumb = page.getByRole('navigation', { name: 'Session hierarchy' })
|
||||
.getByRole('button').last()
|
||||
await expect.poll(() => currentCrumb.textContent(), { timeout: 15_000 })
|
||||
.toBe(`${FIXTURE.title} (1)`)
|
||||
await page.getByText(branchAssistantMarker, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
const settled = scaffold.whenTurnSettled(60_000)
|
||||
const composer = page.locator('textarea:enabled').last()
|
||||
await composer.fill(CONTINUE_PROMPT)
|
||||
await page.getByRole('button', { name: 'Send message', exact: true }).click()
|
||||
await expect.poll(() => page.getByText(CONTINUE_PROMPT, { exact: true }).count(), { timeout: 15_000 }).toBe(1)
|
||||
expect(await settled).toBe(child.session.id)
|
||||
await page.getByText(CONTINUE_DONE, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
await expect.poll(() => page.locator('[data-streaming="true"]').count(), { timeout: 15_000 }).toBe(0)
|
||||
expect(await composer.inputValue()).toBe('')
|
||||
expect(await composer.isEnabled()).toBe(true)
|
||||
expect(source.session.events.some(event => carries(event, CONTINUE_PROMPT))).toBe(false)
|
||||
expect(child.session.events.filter(event => carries(event, CONTINUE_PROMPT))).toHaveLength(1)
|
||||
const lastTurnEnd = child.session.events.findLast((event): event is SessionEvent<'turn/end'> => (
|
||||
event.type === 'turn/end'
|
||||
))
|
||||
expect(lastTurnEnd?.data.reason).toEqual({ kind: 'completed' })
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
}, 180_000)
|
||||
})
|
||||
@@ -0,0 +1,667 @@
|
||||
// Browser geometry contracts for a long Chat transcript. These scenarios are
|
||||
// deliberately virtualizer-neutral: they assert semantic-row position,
|
||||
// bottom ownership, interaction state, and the real outer scroll host rather
|
||||
// than DOM cardinality or implementation-specific spacer markup.
|
||||
import { access, mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import type { Browser, Page } from 'playwright'
|
||||
import { chromium } from 'playwright'
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { ReplayEntry, ReplayOverrideDoc } from '@deepseek-ai/dsh-llm-replay'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { createChatScrollFixture, type ChatScrollFixture } from './chat-scroll-fixture.ts'
|
||||
import {
|
||||
launchWebScaffold,
|
||||
seedSession,
|
||||
watchConsole,
|
||||
webSnapshotMode,
|
||||
type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { newEnglishPage, saveFailureShot } from './support.ts'
|
||||
|
||||
const MODE = webSnapshotMode()
|
||||
const HISTORY_SESSION_ID = 'chat-scroll-history-e2e'
|
||||
const TOOL_SESSION_ID = 'chat-scroll-tool-e2e'
|
||||
const RESTORE_SESSION_A_ID = 'chat-scroll-restore-a-e2e'
|
||||
const RESTORE_SESSION_B_ID = 'chat-scroll-restore-b-e2e'
|
||||
const REPLAY_CONTEXT_WINDOW = 10_000_000
|
||||
const STREAM_PACE_MS = 24
|
||||
const GEOMETRY_TOLERANCE = 2
|
||||
const LIVE_TEXT_PROMPT = 'CHAT_SCROLL_LIVE_USER Continue this long conversation while I inspect older history.'
|
||||
const LIVE_TEXT_FIRST = 'CHAT_SCROLL_LIVE_FIRST'
|
||||
const LIVE_TEXT_DONE = 'CHAT_SCROLL_LIVE_DONE'
|
||||
const LIVE_TOOL_PROMPT = 'CHAT_SCROLL_TOOL_USER Run the requested diagnostic and then summarize it.'
|
||||
const LIVE_TOOL_CALL_ID = CallId('chat-scroll-live-tool-call')
|
||||
const LIVE_TOOL_RESULT = 'CHAT_SCROLL_LIVE_TOOL_RESULT'
|
||||
const LIVE_TOOL_FIRST = 'CHAT_SCROLL_TOOL_STREAM_FIRST'
|
||||
const LIVE_TOOL_DONE = 'CHAT_SCROLL_TOOL_STREAM_DONE'
|
||||
const TOOL_READY_FILE = '.chat-scroll-tool-ready'
|
||||
const TOOL_RELEASE_FILE = '.chat-scroll-tool-release'
|
||||
|
||||
const HISTORY_FIXTURE = createChatScrollFixture({
|
||||
markerPrefix: 'HISTORY',
|
||||
title: 'CHAT_SCROLL_HISTORY long paging session',
|
||||
})
|
||||
const TOOL_FIXTURE = createChatScrollFixture({
|
||||
markerPrefix: 'TOOL',
|
||||
title: 'CHAT_SCROLL_TOOL live tool session',
|
||||
})
|
||||
const RESTORE_FIXTURE_A = createChatScrollFixture({
|
||||
markerPrefix: 'RESTORE_A',
|
||||
title: 'CHAT_SCROLL_RESTORE_A long session',
|
||||
})
|
||||
const RESTORE_FIXTURE_B = createChatScrollFixture({
|
||||
markerPrefix: 'RESTORE_B',
|
||||
title: 'CHAT_SCROLL_RESTORE_B comparison session',
|
||||
turns: 32,
|
||||
})
|
||||
|
||||
interface ScrollGeometry {
|
||||
readonly distanceFromBottom: number
|
||||
readonly scrollTop: number
|
||||
}
|
||||
|
||||
interface FlowAnchor {
|
||||
readonly key: string
|
||||
readonly top: number
|
||||
}
|
||||
|
||||
interface ScrollWorld {
|
||||
readonly events: SessionEvent[]
|
||||
readonly page: Page
|
||||
readonly replayDir?: string
|
||||
readonly scaffold: WebScaffold
|
||||
readonly tripwire: ReturnType<typeof watchConsole>
|
||||
}
|
||||
|
||||
interface ScrollWorldOptions {
|
||||
readonly failureShot: string
|
||||
readonly replay?: ReplayOverrideDoc
|
||||
readonly seeds: readonly { fixture: ChatScrollFixture; id: string }[]
|
||||
}
|
||||
|
||||
function textStream(first: string, done: string, deltaCount: number): StreamChunk[] {
|
||||
const deltas = Array.from({ length: deltaCount }, (_, index) => {
|
||||
if (index === 0) return `${first} `
|
||||
if (index === deltaCount - 1) return `${done}.`
|
||||
return `stream-chunk-${String(index).padStart(3, '0')} ${'incremental response '.repeat(3)}`
|
||||
})
|
||||
const response = deltas.join('')
|
||||
return [
|
||||
{ type: 'block-start', index: 0, blockType: 'text' },
|
||||
...deltas.map(text => ({ type: 'text-delta' as const, index: 0, text })),
|
||||
{ type: 'block-end', index: 0, block: { type: 'text', text: response } },
|
||||
{
|
||||
type: 'usage',
|
||||
usage: { inputTokens: 512, outputTokens: Math.ceil(response.length / 4) },
|
||||
},
|
||||
{ type: 'finish', reason: { kind: 'stop' } },
|
||||
]
|
||||
}
|
||||
|
||||
function toolStream(): StreamChunk[] {
|
||||
const command = [
|
||||
`: > ${TOOL_READY_FILE}`,
|
||||
`while [ ! -f ${TOOL_RELEASE_FILE} ]; do sleep 0.02; done`,
|
||||
'line=1',
|
||||
`while [ "$line" -le 64 ]; do printf '${LIVE_TOOL_RESULT} line %02d\\n' "$line"; line=$((line + 1)); done`,
|
||||
].join('; ')
|
||||
const args = JSON.stringify({ command, description: LIVE_TOOL_RESULT })
|
||||
return [
|
||||
{ type: 'block-start', index: 0, blockType: 'tool-call' },
|
||||
{
|
||||
type: 'tool-call-delta',
|
||||
index: 0,
|
||||
id: LIVE_TOOL_CALL_ID,
|
||||
name: 'bash',
|
||||
argumentsDelta: args,
|
||||
},
|
||||
{
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: LIVE_TOOL_CALL_ID, name: 'bash', arguments: args },
|
||||
},
|
||||
{ type: 'usage', usage: { inputTokens: 256, outputTokens: 48 } },
|
||||
{ type: 'finish', reason: { kind: 'tool-calls' } },
|
||||
]
|
||||
}
|
||||
|
||||
function replayEntry(chunks: StreamChunk[]): ReplayEntry {
|
||||
return { kind: 'chunks', chunks }
|
||||
}
|
||||
|
||||
async function launchScrollWorld(options: ScrollWorldOptions): Promise<ScrollWorld> {
|
||||
let replayDir: string | undefined
|
||||
let scaffold: WebScaffold | undefined
|
||||
let page: Page | undefined
|
||||
try {
|
||||
if (options.replay !== undefined) {
|
||||
replayDir = await mkdtemp(join(tmpdir(), 'dsh-chat-scroll-replay-'))
|
||||
const replayOverride = join(replayDir, 'replay.override.json')
|
||||
await writeFile(replayOverride, JSON.stringify(options.replay))
|
||||
scaffold = await launchWebScaffold({
|
||||
replayFixture: join(replayDir, 'override-only.jsonl'),
|
||||
replayOverride,
|
||||
paceMs: STREAM_PACE_MS,
|
||||
replayContextWindow: REPLAY_CONTEXT_WINDOW,
|
||||
})
|
||||
} else {
|
||||
scaffold = await launchWebScaffold({})
|
||||
}
|
||||
for (const seed of options.seeds) await seedSession(scaffold, seed.fixture.log, seed.id)
|
||||
const events: SessionEvent[] = []
|
||||
scaffold.ctx.on('session/event', (_session, event: SessionEvent) => { events.push(event) })
|
||||
page = await newEnglishPage(browser, 900)
|
||||
const tripwire = watchConsole(page)
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
// Session-list bootstrap can replace the controlled search state. Wait
|
||||
// for the seeded baseline before openSeed starts the lazy content query.
|
||||
await page.getByText(/^\d+ sessions?$/, { exact: true }).waitFor({ timeout: 30_000 })
|
||||
return {
|
||||
events,
|
||||
page,
|
||||
scaffold,
|
||||
tripwire,
|
||||
...(replayDir === undefined ? {} : { replayDir }),
|
||||
}
|
||||
} catch (error) {
|
||||
const failures: unknown[] = [error]
|
||||
if (page !== undefined) await page.context().close().catch((cleanupError: unknown) => failures.push(cleanupError))
|
||||
if (scaffold !== undefined) await scaffold.close().catch((cleanupError: unknown) => failures.push(cleanupError))
|
||||
if (replayDir !== undefined) {
|
||||
await rm(replayDir, { recursive: true, force: true }).catch((cleanupError: unknown) => failures.push(cleanupError))
|
||||
}
|
||||
if (failures.length === 1) throw error
|
||||
throw new AggregateError(failures, 'chat-scroll browser world setup failed and cleanup was incomplete')
|
||||
}
|
||||
}
|
||||
|
||||
async function closeScrollWorld(world: ScrollWorld): Promise<void> {
|
||||
const failures: unknown[] = []
|
||||
// newEnglishPage/browser.newPage owns an isolated context. Close the whole
|
||||
// context so its SSE connection and cache cannot leak into the next world
|
||||
// in this file's shared Chromium process.
|
||||
await world.page.context().close().catch((error: unknown) => failures.push(error))
|
||||
await world.scaffold.close().catch((error: unknown) => failures.push(error))
|
||||
if (world.replayDir !== undefined) {
|
||||
await rm(world.replayDir, { recursive: true, force: true }).catch((error: unknown) => failures.push(error))
|
||||
}
|
||||
if (failures.length === 1) throw failures[0]
|
||||
if (failures.length > 1) throw new AggregateError(failures, 'chat-scroll browser world cleanup failed')
|
||||
}
|
||||
|
||||
async function withScrollWorld(
|
||||
options: ScrollWorldOptions,
|
||||
run: (world: ScrollWorld) => Promise<void>,
|
||||
): Promise<void> {
|
||||
const world = await launchScrollWorld(options)
|
||||
let runFailure: unknown
|
||||
try {
|
||||
await run(world)
|
||||
} catch (error) {
|
||||
runFailure = error
|
||||
try {
|
||||
await saveFailureShot(world.page, options.failureShot)
|
||||
} catch {
|
||||
// Best-effort evidence must never prevent cleanup of the owned world.
|
||||
}
|
||||
}
|
||||
let cleanupFailure: unknown
|
||||
try {
|
||||
await closeScrollWorld(world)
|
||||
} catch (error) {
|
||||
cleanupFailure = error
|
||||
}
|
||||
if (runFailure !== undefined && cleanupFailure !== undefined) {
|
||||
throw new AggregateError([runFailure, cleanupFailure], 'chat-scroll scenario and cleanup both failed')
|
||||
}
|
||||
if (runFailure !== undefined) throw runFailure
|
||||
if (cleanupFailure !== undefined) throw cleanupFailure
|
||||
}
|
||||
|
||||
async function nextPaint(page: Page): Promise<void> {
|
||||
await page.evaluate(async () => {
|
||||
await document.fonts.ready
|
||||
await new Promise<void>(resolve => requestAnimationFrame(() => {
|
||||
requestAnimationFrame(() => { resolve() })
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
function scrollGeometry(page: Page): Promise<ScrollGeometry> {
|
||||
return page.locator('[data-conversation-scroll]').evaluate(host => ({
|
||||
distanceFromBottom: host.scrollHeight - host.clientHeight - host.scrollTop,
|
||||
scrollTop: host.scrollTop,
|
||||
}))
|
||||
}
|
||||
|
||||
async function conversationTurns(page: Page): Promise<number> {
|
||||
const stats = page.getByText(/\d+ turns · \d+ steps/, { exact: true }).last()
|
||||
await stats.waitFor({ timeout: 15_000 })
|
||||
const value = await stats.textContent()
|
||||
const match = value?.match(/^(\d+) turns · \d+ steps$/)
|
||||
if (match?.[1] === undefined) throw new Error(`unexpected conversation stats ${JSON.stringify(value)}`)
|
||||
return Number(match[1])
|
||||
}
|
||||
|
||||
async function openSeed(page: Page, fixture: ChatScrollFixture, tailMarker?: string): Promise<void> {
|
||||
const search = page.getByRole('textbox', { name: 'Search name, keywords...', exact: true })
|
||||
// Cold summaries initially show the temporary workspace basename, so the
|
||||
// persisted first-message marker is the stable user-facing identity. The
|
||||
// query itself triggers lazy content-index reconciliation; no transient
|
||||
// empty-state paint is used as a barrier.
|
||||
await search.fill(fixture.markers.user(1))
|
||||
const results = page.getByRole('tree', { name: 'Search results' }).getByRole('treeitem')
|
||||
await expect.poll(() => results.count(), { timeout: 60_000 }).toBe(1)
|
||||
await results.click()
|
||||
await page.getByRole('tab', { name: 'Chat', exact: true }).waitFor({ timeout: 30_000 })
|
||||
if (tailMarker !== undefined) {
|
||||
await page.getByText(tailMarker, { exact: false }).last().waitFor({ timeout: 30_000 })
|
||||
}
|
||||
await nextPaint(page)
|
||||
}
|
||||
|
||||
async function wheelTranscript(page: Page, deltaY: number): Promise<void> {
|
||||
const box = await page.locator('[data-conversation-scroll]').boundingBox()
|
||||
if (box === null) throw new Error('conversation scrollport has no layout box')
|
||||
await page.mouse.move(box.x + box.width / 2, box.y + Math.min(140, box.height / 3))
|
||||
await page.mouse.wheel(0, deltaY)
|
||||
await nextPaint(page)
|
||||
}
|
||||
|
||||
async function wheelToHistoryStart(page: Page): Promise<void> {
|
||||
for (let attempt = 0; attempt < 12; attempt += 1) {
|
||||
if ((await scrollGeometry(page)).scrollTop <= 1) break
|
||||
await wheelTranscript(page, -2_400)
|
||||
}
|
||||
await expect.poll(async () => (await scrollGeometry(page)).scrollTop, { timeout: 10_000 })
|
||||
.toBeLessThanOrEqual(1)
|
||||
}
|
||||
|
||||
async function wheelUntilMounted(page: Page, selector: string, deltaY: number): Promise<void> {
|
||||
for (let attempt = 0; attempt < 16; attempt += 1) {
|
||||
if (await page.locator(selector).count() > 0) return
|
||||
await wheelTranscript(page, deltaY)
|
||||
}
|
||||
throw new Error(`selector did not mount during transcript wheel: ${selector}`)
|
||||
}
|
||||
|
||||
function visibleFlowAnchor(page: Page): Promise<FlowAnchor> {
|
||||
return page.locator('[data-conversation-scroll]').evaluate((host) => {
|
||||
const rows = [...host.querySelectorAll<HTMLElement>('[data-chat-anchor-key]')]
|
||||
const viewport = host.getBoundingClientRect()
|
||||
const composer = host.querySelector<HTMLElement>('[data-composer-seat]')
|
||||
const visibleBottom = composer?.getBoundingClientRect().top ?? viewport.bottom
|
||||
const visible = rows.filter((candidate) => {
|
||||
const rect = candidate.getBoundingClientRect()
|
||||
return rect.bottom > viewport.top && rect.top < visibleBottom
|
||||
})
|
||||
const row = visible[0]
|
||||
if (row?.dataset.chatAnchorKey === undefined) {
|
||||
throw new Error(`no visible settled Chat row: ${JSON.stringify({
|
||||
composerTop: visibleBottom,
|
||||
host: { bottom: viewport.bottom, top: viewport.top },
|
||||
rows: rows.slice(0, 4).map(candidate => ({
|
||||
callId: candidate.dataset.chatCallId,
|
||||
key: candidate.dataset.chatAnchorKey,
|
||||
rect: {
|
||||
bottom: candidate.getBoundingClientRect().bottom,
|
||||
top: candidate.getBoundingClientRect().top,
|
||||
},
|
||||
})),
|
||||
totalRows: rows.length,
|
||||
})}`)
|
||||
}
|
||||
return {
|
||||
key: row.dataset.chatAnchorKey,
|
||||
top: row.getBoundingClientRect().top - viewport.top,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
function flowTop(page: Page, key: string): Promise<number> {
|
||||
return page.locator('[data-chat-anchor-key]').evaluateAll((rows, anchorKey) => {
|
||||
const row = rows.find(candidate => (candidate as HTMLElement).dataset.chatAnchorKey === anchorKey)
|
||||
if (!(row instanceof HTMLElement)) throw new Error(`stable Chat anchor ${anchorKey} is not mounted`)
|
||||
const host = row.closest('[data-conversation-scroll]')
|
||||
if (!(host instanceof HTMLElement)) throw new Error('flow row has no conversation scrollport')
|
||||
return row.getBoundingClientRect().top - host.getBoundingClientRect().top
|
||||
}, key)
|
||||
}
|
||||
|
||||
async function expectSameFlowTop(page: Page, anchor: FlowAnchor): Promise<void> {
|
||||
await expect.poll(async () => Math.abs((await flowTop(page, anchor.key)) - anchor.top), {
|
||||
timeout: 10_000,
|
||||
message: `flow row ${anchor.key} moved relative to the transcript viewport`,
|
||||
}).toBeLessThanOrEqual(GEOMETRY_TOLERANCE)
|
||||
}
|
||||
|
||||
async function expectBottom(page: Page): Promise<void> {
|
||||
await expect.poll(async () => Math.abs((await scrollGeometry(page)).distanceFromBottom), {
|
||||
timeout: 10_000,
|
||||
}).toBeLessThanOrEqual(1)
|
||||
}
|
||||
|
||||
async function expectMarkerAboveComposer(page: Page, marker: string): Promise<void> {
|
||||
const geometry = await page.getByText(marker, { exact: false }).last().evaluate((node) => {
|
||||
const row = node.closest('[data-chat-flow-key], [data-streaming]')
|
||||
const composer = node.closest('[data-conversation-scroll]')?.querySelector('[data-composer-seat]')
|
||||
if (!(row instanceof HTMLElement) || !(composer instanceof HTMLElement)) {
|
||||
throw new Error('latest marker or composer geometry is unavailable')
|
||||
}
|
||||
return {
|
||||
composerTop: composer.getBoundingClientRect().top,
|
||||
rowBottom: row.getBoundingClientRect().bottom,
|
||||
}
|
||||
})
|
||||
expect(geometry.rowBottom).toBeLessThanOrEqual(geometry.composerTop + GEOMETRY_TOLERANCE)
|
||||
}
|
||||
|
||||
async function loadEarlierWithAnchor(page: Page): Promise<void> {
|
||||
await wheelToHistoryStart(page)
|
||||
const older = page.getByRole('button', { name: 'Load earlier', exact: true })
|
||||
await older.waitFor({ timeout: 10_000 })
|
||||
const anchor = await visibleFlowAnchor(page)
|
||||
const before = await conversationTurns(page)
|
||||
await older.click()
|
||||
await expect.poll(() => conversationTurns(page), { timeout: 30_000 }).toBeGreaterThan(before)
|
||||
await nextPaint(page)
|
||||
await expectSameFlowTop(page, anchor)
|
||||
}
|
||||
|
||||
async function fileExists(path: string): Promise<boolean> {
|
||||
try {
|
||||
await access(path)
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function eventCarries(event: SessionEvent, marker: string): boolean {
|
||||
return JSON.stringify(event).includes(marker)
|
||||
}
|
||||
|
||||
function assertClean(world: ScrollWorld): void {
|
||||
expect(world.tripwire.pageErrors).toEqual([])
|
||||
expect(world.tripwire.warnings).toEqual([])
|
||||
}
|
||||
|
||||
let browser: Browser
|
||||
|
||||
describe('web e2e: long Chat scroll contract', () => {
|
||||
beforeAll(async () => {
|
||||
browser = await chromium.launch()
|
||||
})
|
||||
|
||||
afterAll(async () => {
|
||||
await browser?.close()
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('preserves the reader anchor when history and streaming arrive concurrently', async () => {
|
||||
await withScrollWorld({
|
||||
failureShot: 'web-e2e-chat-scroll-history-stream',
|
||||
replay: [replayEntry(textStream(LIVE_TEXT_FIRST, LIVE_TEXT_DONE, 120))],
|
||||
seeds: [{ fixture: HISTORY_FIXTURE, id: HISTORY_SESSION_ID }],
|
||||
}, async (world) => {
|
||||
await openSeed(
|
||||
world.page,
|
||||
HISTORY_FIXTURE,
|
||||
HISTORY_FIXTURE.markers.assistant(HISTORY_FIXTURE.turns),
|
||||
)
|
||||
await expectBottom(world.page)
|
||||
|
||||
let releaseHistory = (): void => {}
|
||||
let held = false
|
||||
let releaseGate: (() => void) | undefined
|
||||
const gate = new Promise<void>((resolve) => { releaseGate = resolve })
|
||||
releaseHistory = () => { releaseGate?.() }
|
||||
await world.page.route('**/api/session.history', async (route) => {
|
||||
const request = route.request().postDataJSON() as {
|
||||
method?: string
|
||||
payload?: { beforeSeq?: number }
|
||||
}
|
||||
if (!held && request.method === 'session.history' && request.payload?.beforeSeq !== undefined) {
|
||||
held = true
|
||||
await gate
|
||||
}
|
||||
await route.continue()
|
||||
})
|
||||
|
||||
const settled = world.scaffold.whenTurnSettled(60_000)
|
||||
try {
|
||||
const composer = world.page.locator('textarea:enabled').last()
|
||||
await composer.fill(LIVE_TEXT_PROMPT)
|
||||
await world.page.getByRole('button', { name: 'Send message', exact: true }).click()
|
||||
await world.page.getByText(LIVE_TEXT_FIRST, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
await wheelToHistoryStart(world.page)
|
||||
const beforeTurns = await conversationTurns(world.page)
|
||||
await world.page.getByRole('button', { name: 'Load earlier', exact: true }).click()
|
||||
await expect.poll(() => held, { timeout: 10_000 }).toBe(true)
|
||||
|
||||
await wheelTranscript(world.page, 420)
|
||||
const readerAnchor = await visibleFlowAnchor(world.page)
|
||||
const chunksAfterAnchor = world.events.filter(event => event.type === 'assistant/chunk').length
|
||||
await expect.poll(
|
||||
() => world.events.filter(event => event.type === 'assistant/chunk').length,
|
||||
{ timeout: 10_000 },
|
||||
).toBeGreaterThan(chunksAfterAnchor + 5)
|
||||
|
||||
releaseHistory()
|
||||
await expect.poll(() => conversationTurns(world.page), { timeout: 30_000 }).toBeGreaterThan(beforeTurns)
|
||||
await nextPaint(world.page)
|
||||
await expectSameFlowTop(world.page, readerAnchor)
|
||||
} finally {
|
||||
releaseHistory()
|
||||
}
|
||||
|
||||
await settled
|
||||
await expect.poll(() => world.page.locator('[data-streaming="true"]').count(), { timeout: 15_000 }).toBe(0)
|
||||
await world.page.getByText(LIVE_TEXT_DONE, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
await world.page.unroute('**/api/session.history')
|
||||
|
||||
let additionalPages = 0
|
||||
while (additionalPages < 8) {
|
||||
await wheelToHistoryStart(world.page)
|
||||
if (await world.page.getByRole('button', { name: 'Load earlier', exact: true }).count() === 0) break
|
||||
await loadEarlierWithAnchor(world.page)
|
||||
additionalPages += 1
|
||||
}
|
||||
expect(additionalPages).toBeGreaterThan(0)
|
||||
expect(await conversationTurns(world.page)).toBe(HISTORY_FIXTURE.turns + 1)
|
||||
expect(await world.page.getByRole('button', { name: 'Load earlier', exact: true }).count()).toBe(0)
|
||||
assertClean(world)
|
||||
})
|
||||
}, 180_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('keeps streaming ownership and tool disclosure state across a long scroll-away cycle', async () => {
|
||||
await withScrollWorld({
|
||||
failureShot: 'web-e2e-chat-scroll-live-tool',
|
||||
replay: [
|
||||
replayEntry(toolStream()),
|
||||
replayEntry(textStream(LIVE_TOOL_FIRST, LIVE_TOOL_DONE, 84)),
|
||||
],
|
||||
seeds: [{ fixture: TOOL_FIXTURE, id: TOOL_SESSION_ID }],
|
||||
}, async (world) => {
|
||||
const readyPath = join(world.scaffold.workspaceCwd, TOOL_READY_FILE)
|
||||
const releasePath = join(world.scaffold.workspaceCwd, TOOL_RELEASE_FILE)
|
||||
await openSeed(world.page, TOOL_FIXTURE, TOOL_FIXTURE.markers.assistant(TOOL_FIXTURE.turns))
|
||||
const settled = world.scaffold.whenTurnSettled(60_000)
|
||||
let released = false
|
||||
try {
|
||||
const composer = world.page.locator('textarea:enabled').last()
|
||||
await composer.fill(LIVE_TOOL_PROMPT)
|
||||
await world.page.getByRole('button', { name: 'Send message', exact: true }).click()
|
||||
await expect.poll(() => fileExists(readyPath), { timeout: 15_000 }).toBe(true)
|
||||
const liveRow = world.page.locator(`[data-chat-call-id="${LIVE_TOOL_CALL_ID}"] [data-sample="bash-global"]`)
|
||||
await liveRow.waitFor({ timeout: 15_000 })
|
||||
expect(await liveRow.getAttribute('data-state')).toBe('running')
|
||||
await expectBottom(world.page)
|
||||
|
||||
await wheelTranscript(world.page, -1_200)
|
||||
await world.page.getByRole('button', { name: 'Back to bottom', exact: true }).waitFor({ timeout: 10_000 })
|
||||
const awayAnchor = await visibleFlowAnchor(world.page)
|
||||
const chunksBeforeRelease = world.events.filter(event => event.type === 'assistant/chunk').length
|
||||
await writeFile(releasePath, 'release\n')
|
||||
released = true
|
||||
await expect.poll(
|
||||
() => world.events.some(event => event.type === 'tool/result'),
|
||||
{ timeout: 15_000 },
|
||||
).toBe(true)
|
||||
await expect.poll(
|
||||
() => world.events.some(event => eventCarries(event, LIVE_TOOL_FIRST)),
|
||||
{ timeout: 15_000 },
|
||||
).toBe(true)
|
||||
await expect.poll(
|
||||
() => world.events.filter(event => event.type === 'assistant/chunk').length,
|
||||
{ timeout: 15_000 },
|
||||
).toBeGreaterThan(chunksBeforeRelease + 5)
|
||||
await expectSameFlowTop(world.page, awayAnchor)
|
||||
|
||||
const chunksAtRepin = world.events.filter(event => event.type === 'assistant/chunk').length
|
||||
await world.page.getByRole('button', { name: 'Back to bottom', exact: true }).click()
|
||||
await expectBottom(world.page)
|
||||
await expect.poll(
|
||||
() => world.events.filter(event => event.type === 'assistant/chunk').length,
|
||||
{ timeout: 15_000 },
|
||||
).toBeGreaterThan(chunksAtRepin + 5)
|
||||
await expectBottom(world.page)
|
||||
} finally {
|
||||
if (!released) await writeFile(releasePath, 'release\n').catch(() => {})
|
||||
}
|
||||
|
||||
await settled
|
||||
await expect.poll(() => world.page.locator('[data-streaming="true"]').count(), { timeout: 15_000 }).toBe(0)
|
||||
await world.page.getByText(LIVE_TOOL_DONE, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
await expectBottom(world.page)
|
||||
await expectMarkerAboveComposer(world.page, LIVE_TOOL_DONE)
|
||||
|
||||
const liveRowSelector = `[data-chat-call-id="${LIVE_TOOL_CALL_ID}"] [data-sample="bash-global"]`
|
||||
const liveRow = world.page.locator(liveRowSelector)
|
||||
await liveRow.scrollIntoViewIfNeeded()
|
||||
await nextPaint(world.page)
|
||||
const toolAnchor = await liveRow.evaluate((row) => {
|
||||
const flow = row.closest<HTMLElement>('[data-chat-anchor-key]')
|
||||
const host = row.closest<HTMLElement>('[data-conversation-scroll]')
|
||||
if (flow?.dataset.chatAnchorKey === undefined || host === null) {
|
||||
throw new Error('live tool row has no settled flow identity')
|
||||
}
|
||||
return {
|
||||
key: flow.dataset.chatAnchorKey,
|
||||
top: flow.getBoundingClientRect().top - host.getBoundingClientRect().top,
|
||||
}
|
||||
})
|
||||
await liveRow.click()
|
||||
await expect.poll(() => liveRow.getAttribute('aria-expanded'), { timeout: 10_000 }).toBe('true')
|
||||
await expectSameFlowTop(world.page, toolAnchor)
|
||||
await wheelToHistoryStart(world.page)
|
||||
await world.page.getByRole('button', { name: 'Back to bottom', exact: true }).click()
|
||||
await expectBottom(world.page)
|
||||
await wheelUntilMounted(world.page, liveRowSelector, -1_100)
|
||||
const restoredRow = world.page.locator(liveRowSelector)
|
||||
await restoredRow.waitFor({ timeout: 10_000 })
|
||||
expect(await restoredRow.getAttribute('aria-expanded')).toBe('true')
|
||||
expect(await world.page.getByText(LIVE_TOOL_RESULT, { exact: false }).count()).toBeGreaterThan(0)
|
||||
assertClean(world)
|
||||
})
|
||||
}, 180_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('restores tab/session position and keeps composer resizing on the correct scroll owner', async () => {
|
||||
await withScrollWorld({
|
||||
failureShot: 'web-e2e-chat-scroll-restore-composer',
|
||||
seeds: [
|
||||
{ fixture: RESTORE_FIXTURE_A, id: RESTORE_SESSION_A_ID },
|
||||
{ fixture: RESTORE_FIXTURE_B, id: RESTORE_SESSION_B_ID },
|
||||
],
|
||||
}, async (world) => {
|
||||
await openSeed(
|
||||
world.page,
|
||||
RESTORE_FIXTURE_A,
|
||||
RESTORE_FIXTURE_A.markers.assistant(RESTORE_FIXTURE_A.turns),
|
||||
)
|
||||
await loadEarlierWithAnchor(world.page)
|
||||
await loadEarlierWithAnchor(world.page)
|
||||
await wheelToHistoryStart(world.page)
|
||||
await wheelTranscript(world.page, 1_300)
|
||||
const sessionAnchor = await visibleFlowAnchor(world.page)
|
||||
|
||||
await world.page.getByRole('tab', { name: 'Trajectory', exact: true }).click()
|
||||
await world.page.getByLabel('Trajectory timeline').waitFor({ timeout: 30_000 })
|
||||
await world.page.setViewportSize({ width: 700, height: 900 })
|
||||
await world.page.getByRole('tab', { name: 'Chat', exact: true }).click()
|
||||
await nextPaint(world.page)
|
||||
await expectSameFlowTop(world.page, sessionAnchor)
|
||||
|
||||
await openSeed(
|
||||
world.page,
|
||||
RESTORE_FIXTURE_B,
|
||||
RESTORE_FIXTURE_B.markers.assistant(RESTORE_FIXTURE_B.turns),
|
||||
)
|
||||
await openSeed(
|
||||
world.page,
|
||||
RESTORE_FIXTURE_A,
|
||||
)
|
||||
await expectSameFlowTop(world.page, sessionAnchor)
|
||||
|
||||
const backToBottom = world.page.getByRole('button', { name: 'Back to bottom', exact: true })
|
||||
await backToBottom.evaluate((button) => {
|
||||
if (!(button instanceof HTMLElement)) throw new Error('Back-to-bottom control is not an HTML element')
|
||||
button.click()
|
||||
const trajectory = [...document.querySelectorAll<HTMLElement>('[role="tab"]')]
|
||||
.find(tab => tab.textContent?.trim() === 'Trajectory')
|
||||
if (!(trajectory instanceof HTMLElement)) {
|
||||
throw new Error('Trajectory tab is unavailable during pinned remount')
|
||||
}
|
||||
trajectory.click()
|
||||
})
|
||||
await world.page.getByLabel('Trajectory timeline').waitFor({ timeout: 30_000 })
|
||||
await world.page.getByRole('tab', { name: 'Chat', exact: true }).click()
|
||||
await expectBottom(world.page)
|
||||
await openSeed(
|
||||
world.page,
|
||||
RESTORE_FIXTURE_B,
|
||||
RESTORE_FIXTURE_B.markers.assistant(RESTORE_FIXTURE_B.turns),
|
||||
)
|
||||
await openSeed(
|
||||
world.page,
|
||||
RESTORE_FIXTURE_A,
|
||||
RESTORE_FIXTURE_A.markers.assistant(RESTORE_FIXTURE_A.turns),
|
||||
)
|
||||
await expectBottom(world.page)
|
||||
const composer = world.page.locator('textarea:enabled').last()
|
||||
const longDraft = Array.from(
|
||||
{ length: 18 },
|
||||
(_, index) => `composer resize line ${String(index + 1).padStart(2, '0')}`,
|
||||
).join('\n')
|
||||
await composer.fill(longDraft)
|
||||
await nextPaint(world.page)
|
||||
await expectBottom(world.page)
|
||||
await expectMarkerAboveComposer(
|
||||
world.page,
|
||||
RESTORE_FIXTURE_A.markers.assistant(RESTORE_FIXTURE_A.turns),
|
||||
)
|
||||
|
||||
await composer.fill('short draft')
|
||||
await nextPaint(world.page)
|
||||
await wheelTranscript(world.page, -900)
|
||||
const resizeAnchor = await visibleFlowAnchor(world.page)
|
||||
await composer.fill(longDraft)
|
||||
await nextPaint(world.page)
|
||||
await expectSameFlowTop(world.page, resizeAnchor)
|
||||
await composer.fill('short draft')
|
||||
await nextPaint(world.page)
|
||||
await expectSameFlowTop(world.page, resizeAnchor)
|
||||
|
||||
const beforeChain = await scrollGeometry(world.page)
|
||||
await composer.hover()
|
||||
await world.page.mouse.wheel(0, -320)
|
||||
await expect.poll(async () => (await scrollGeometry(world.page)).scrollTop, { timeout: 10_000 })
|
||||
.toBeLessThan(beforeChain.scrollTop)
|
||||
assertClean(world)
|
||||
})
|
||||
}, 180_000)
|
||||
})
|
||||
@@ -0,0 +1,235 @@
|
||||
// Synthetic long-chat history for browser behavior contracts. The fixture is
|
||||
// generated through Session so pagination exercises the same event shapes as
|
||||
// persisted conversations, while unique markers let tests identify semantic
|
||||
// rows without depending on CSS-module names or the eventual virtualizer DOM.
|
||||
import {
|
||||
CallId,
|
||||
createAssistantMessage,
|
||||
createToolResultMessage,
|
||||
createUserMessage,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
SESSION_FORMAT_VERSION,
|
||||
Session,
|
||||
SessionId,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
// Carries the session/title event declaration into this fixture builder.
|
||||
import type {} from '@deepseek-ai/dsh-session-title'
|
||||
|
||||
/** Options for one deterministic long-chat fixture. */
|
||||
export interface ChatScrollFixtureOptions {
|
||||
/** Marker namespace, used when two sessions share one browser world. */
|
||||
readonly markerPrefix: string
|
||||
/** Searchable title projected into the sidebar. */
|
||||
readonly title: string
|
||||
/** Number of closed turns to generate. */
|
||||
readonly turns?: number
|
||||
}
|
||||
|
||||
/** Semantic marker helpers returned with a generated fixture. */
|
||||
export interface ChatScrollMarkers {
|
||||
/** Marker painted in the human message for a turn. */
|
||||
user(turn: number): string
|
||||
/** Marker painted in the final assistant message for a turn. */
|
||||
assistant(turn: number): string
|
||||
/** Marker painted in one seeded bash call and result. */
|
||||
tool(turn: number, index: number): string
|
||||
}
|
||||
|
||||
/** Generated JSONL plus the stable facts browser scenarios assert. */
|
||||
export interface ChatScrollFixture {
|
||||
readonly log: string
|
||||
readonly markers: ChatScrollMarkers
|
||||
readonly title: string
|
||||
readonly turns: number
|
||||
}
|
||||
|
||||
const DEFAULT_TURNS = 88
|
||||
const TOOL_INTERVAL = 8
|
||||
const CODE_INTERVAL = 11
|
||||
|
||||
function text(value: string): { type: 'text'; text: string }[] {
|
||||
return [{ type: 'text', text: value }]
|
||||
}
|
||||
|
||||
function suffix(turn: number): string {
|
||||
return String(turn).padStart(3, '0')
|
||||
}
|
||||
|
||||
function markerHelpers(prefix: string): ChatScrollMarkers {
|
||||
return {
|
||||
user: turn => `CHAT_SCROLL_${prefix}_USER_${suffix(turn)}`,
|
||||
assistant: turn => `CHAT_SCROLL_${prefix}_ASSISTANT_${suffix(turn)}`,
|
||||
tool: (turn, index) => `CHAT_SCROLL_${prefix}_TOOL_${suffix(turn)}_${String(index)}`,
|
||||
}
|
||||
}
|
||||
|
||||
function appendRequestHeader(session: Session, turn: number, step: number): void {
|
||||
session.append('request/header', {
|
||||
header: {
|
||||
config: { provider: 'deepseek-official', model: 'deepseek-v4-flash' },
|
||||
system: `Synthetic chat-scroll request for turn ${String(turn)}, step ${String(step)}.`,
|
||||
},
|
||||
reason: turn === 1 && step === 1 ? 'initial' : 'change',
|
||||
})
|
||||
}
|
||||
|
||||
function appendAssistant(session: Session, turn: number, step: number, body: string): void {
|
||||
session.append('assistant/message', {
|
||||
turn,
|
||||
step,
|
||||
message: createAssistantMessage({
|
||||
content: text(body),
|
||||
source: { provider: 'deepseek-official', model: 'deepseek-v4-flash' },
|
||||
}),
|
||||
usage: {
|
||||
inputTokens: 2_000 + turn * 7,
|
||||
outputTokens: 180 + step * 20,
|
||||
},
|
||||
}, { surfaceOp: 'append' })
|
||||
}
|
||||
|
||||
function codeBlock(turn: number): string {
|
||||
if (turn % CODE_INTERVAL !== 0) return ''
|
||||
const lines = Array.from(
|
||||
{ length: 30 },
|
||||
(_, index) => `const scroll_case_${suffix(turn)}_${String(index).padStart(2, '0')} = ${String(turn + index)}`,
|
||||
)
|
||||
return `\n\n\`\`\`ts\n${lines.join('\n')}\n\`\`\``
|
||||
}
|
||||
|
||||
function appendToolStep(
|
||||
session: Session,
|
||||
markers: ChatScrollMarkers,
|
||||
turn: number,
|
||||
): void {
|
||||
const calls = [1, 2].map((index) => {
|
||||
const marker = markers.tool(turn, index)
|
||||
const callId = CallId(`chat-scroll-${suffix(turn)}-${String(index)}`)
|
||||
const args = JSON.stringify({
|
||||
command: `printf '${marker}\\n'`,
|
||||
description: marker,
|
||||
})
|
||||
return { args, callId, marker }
|
||||
})
|
||||
|
||||
session.append('assistant/message', {
|
||||
turn,
|
||||
step: 1,
|
||||
message: createAssistantMessage({
|
||||
content: [
|
||||
{ type: 'reasoning', text: `Inspecting two scroll fixtures for turn ${String(turn)}.` },
|
||||
...calls.map(call => ({
|
||||
type: 'tool-call' as const,
|
||||
id: call.callId,
|
||||
name: 'bash',
|
||||
arguments: call.args,
|
||||
})),
|
||||
],
|
||||
source: { provider: 'deepseek-official', model: 'deepseek-v4-flash' },
|
||||
}),
|
||||
usage: { inputTokens: 2_000 + turn * 7, outputTokens: 240, reasoningTokens: 30 },
|
||||
}, { surfaceOp: 'append' })
|
||||
|
||||
for (const call of calls) {
|
||||
const source = session.append('tool/call', {
|
||||
turn,
|
||||
step: 1,
|
||||
callId: call.callId,
|
||||
name: 'bash',
|
||||
arguments: call.args,
|
||||
})
|
||||
session.append('tool/result', {
|
||||
turn,
|
||||
step: 1,
|
||||
message: createToolResultMessage({
|
||||
callId: call.callId,
|
||||
content: text(Array.from(
|
||||
{ length: 12 },
|
||||
(_, line) => `${call.marker} output line ${String(line + 1).padStart(2, '0')}`,
|
||||
).join('\n')),
|
||||
isError: false,
|
||||
}),
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [source.seq] })
|
||||
}
|
||||
}
|
||||
|
||||
function fixtureLog(session: Session): string {
|
||||
return [
|
||||
JSON.stringify({
|
||||
type: 'session',
|
||||
version: SESSION_FORMAT_VERSION,
|
||||
id: '{{sessionId}}',
|
||||
createdAt: Date.now() - 60_000,
|
||||
cwd: '{{cwd}}',
|
||||
delegationDepth: 0,
|
||||
}),
|
||||
...session.events.map(event => JSON.stringify(event)),
|
||||
'',
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a multi-page conversation with prose, fenced code, and paired bash
|
||||
* calls/results. Every turn is closed, so cold resume cannot repair or mutate
|
||||
* the seed before the browser observes it.
|
||||
* @param options - Fixture identity and optional turn count.
|
||||
* @returns Canonical JSONL and semantic marker helpers.
|
||||
*/
|
||||
export function createChatScrollFixture(options: ChatScrollFixtureOptions): ChatScrollFixture {
|
||||
const turns = options.turns ?? DEFAULT_TURNS
|
||||
const markers = markerHelpers(options.markerPrefix)
|
||||
const session = new Session(SessionId(`chat-scroll-${options.markerPrefix.toLowerCase()}-template`))
|
||||
|
||||
for (let turn = 1; turn <= turns; turn += 1) {
|
||||
session.append('turn/start', {
|
||||
turn,
|
||||
trigger: { kind: 'message', source: { kind: 'user' } },
|
||||
})
|
||||
const user = session.append('user/message', createUserMessage({
|
||||
content: text(
|
||||
`${markers.user(turn)} Review the long-running conversation state for turn ${String(turn)}. `
|
||||
+ 'Keep the visible message stable while history, tools, and new output change around it.',
|
||||
),
|
||||
source: { kind: 'user' },
|
||||
}), { surfaceOp: 'append' })
|
||||
if (turn === 1) {
|
||||
session.append('session/title', {
|
||||
title: options.title,
|
||||
messageSeqs: [user.seq],
|
||||
source: { kind: 'fallback' },
|
||||
})
|
||||
}
|
||||
|
||||
session.append('step/start', { turn, step: 1 })
|
||||
appendRequestHeader(session, turn, 1)
|
||||
if (turn % TOOL_INTERVAL === 0) {
|
||||
appendToolStep(session, markers, turn)
|
||||
session.append('step/end', { turn, step: 1 })
|
||||
session.append('step/start', { turn, step: 2 })
|
||||
appendRequestHeader(session, turn, 2)
|
||||
appendAssistant(
|
||||
session,
|
||||
turn,
|
||||
2,
|
||||
`${markers.assistant(turn)} Both tool results are accounted for. `
|
||||
+ `This settled response keeps turn ${String(turn)} identifiable after paging.${codeBlock(turn)}`,
|
||||
)
|
||||
session.append('step/end', { turn, step: 2 })
|
||||
} else {
|
||||
appendAssistant(
|
||||
session,
|
||||
turn,
|
||||
1,
|
||||
`${markers.assistant(turn)} The conversation remains readable after several paragraphs.\n\n`
|
||||
+ `Turn ${String(turn)} deliberately carries enough prose to wrap at narrower viewport widths. `
|
||||
+ 'The semantic marker stays near the start so geometry probes can find the same rendered row.\n\n'
|
||||
+ `The closing paragraph makes this a realistic assistant response rather than a one-line list item.${codeBlock(turn)}`,
|
||||
)
|
||||
session.append('step/end', { turn, step: 1 })
|
||||
}
|
||||
session.append('turn/end', { turn, reason: { kind: 'completed' } })
|
||||
}
|
||||
|
||||
return { log: fixtureLog(session), markers, title: options.title, turns }
|
||||
}
|
||||
@@ -50,6 +50,7 @@ const COMPARISON_TURNS = 8
|
||||
const COMPARISON_DELTA_COUNT = 24
|
||||
const COMPARISON_TOOL_INTERVAL = 3
|
||||
const SOAK_TURNS = 100
|
||||
const POST_SOAK_RENDER_TURN = SOAK_TURNS + 1
|
||||
const SOAK_DELTA_COUNT = 8
|
||||
const SOAK_TOOL_INTERVAL = 10
|
||||
const SOAK_CHECKPOINT_INTERVAL = 10
|
||||
@@ -107,6 +108,15 @@ interface MutationProbeResult {
|
||||
readonly records: number
|
||||
}
|
||||
|
||||
interface UserRenderProbeResult {
|
||||
readonly trustedClick: boolean
|
||||
readonly sendToDomMs: number
|
||||
readonly sendToPaintMs: number
|
||||
readonly domToPaintMs: number
|
||||
readonly mutationBatches: number
|
||||
readonly mutationRecords: number
|
||||
}
|
||||
|
||||
interface RetainedBrowserState {
|
||||
readonly domElements: number
|
||||
readonly nodes: number
|
||||
@@ -608,6 +618,168 @@ async function stopMutationProbe(page: Page): Promise<MutationProbeResult> {
|
||||
})
|
||||
}
|
||||
|
||||
async function startUserRenderProbe(
|
||||
page: Page,
|
||||
marker: string,
|
||||
): Promise<void> {
|
||||
const send = page.getByRole('button', { name: 'Send message', exact: true })
|
||||
await send.waitFor({ timeout: 15_000 })
|
||||
await expect.poll(() => send.isEnabled(), { timeout: 15_000 }).toBe(true)
|
||||
await page.evaluate((expectedMarker) => {
|
||||
const target = document.querySelector('[data-conversation-scroll]')
|
||||
if (target === null) throw new Error('user render probe target is unavailable')
|
||||
const probe: {
|
||||
sendAt?: number
|
||||
domAt?: number
|
||||
paintAt?: number
|
||||
trustedClick?: boolean
|
||||
batches: number
|
||||
records: number
|
||||
observer?: MutationObserver
|
||||
pollForMarker?: () => void
|
||||
} = { batches: 0, records: 0 }
|
||||
const transcriptContainsMarker = (): boolean => {
|
||||
// The composer projects the draft into a backdrop and hidden sizing
|
||||
// mirror; only a matching text node outside that card proves delivery.
|
||||
const walker = document.createTreeWalker(target, NodeFilter.SHOW_TEXT)
|
||||
for (let node = walker.nextNode(); node !== null; node = walker.nextNode()) {
|
||||
if (!node.textContent?.includes(expectedMarker)) continue
|
||||
const parent = node.parentElement
|
||||
if (parent !== null && parent.closest('[data-composer-card]') === null) return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
const markDomVisible = (): void => {
|
||||
if (
|
||||
probe.sendAt === undefined
|
||||
|| probe.domAt !== undefined
|
||||
|| !transcriptContainsMarker()
|
||||
) return
|
||||
probe.domAt = globalThis.performance.now()
|
||||
// The second callback runs only after a render opportunity for the DOM
|
||||
// insertion observed before the first callback.
|
||||
requestAnimationFrame(() => {
|
||||
requestAnimationFrame(() => {
|
||||
probe.paintAt = globalThis.performance.now()
|
||||
observer.disconnect()
|
||||
})
|
||||
})
|
||||
}
|
||||
const pollForMarker = (): void => {
|
||||
markDomVisible()
|
||||
if (probe.domAt === undefined) requestAnimationFrame(pollForMarker)
|
||||
}
|
||||
const observer = new MutationObserver((records) => {
|
||||
if (probe.sendAt === undefined) return
|
||||
probe.batches += 1
|
||||
probe.records += records.length
|
||||
markDomVisible()
|
||||
})
|
||||
probe.observer = observer
|
||||
probe.pollForMarker = pollForMarker
|
||||
observer.observe(target, {
|
||||
attributes: true,
|
||||
attributeFilter: ['data-streaming'],
|
||||
characterData: true,
|
||||
childList: true,
|
||||
subtree: true,
|
||||
})
|
||||
Reflect.set(globalThis, '__dshPerfUserRenderProbe', probe)
|
||||
}, marker)
|
||||
await send.evaluate((button) => {
|
||||
button.addEventListener('click', (event) => {
|
||||
const probe = Reflect.get(globalThis, '__dshPerfUserRenderProbe') as
|
||||
| { sendAt?: number; trustedClick?: boolean; pollForMarker?: () => void }
|
||||
| undefined
|
||||
if (probe?.pollForMarker === undefined) throw new Error('user render probe was not started')
|
||||
probe.trustedClick = event.isTrusted
|
||||
probe.sendAt = globalThis.performance.now()
|
||||
requestAnimationFrame(probe.pollForMarker)
|
||||
}, { capture: true, once: true })
|
||||
})
|
||||
}
|
||||
|
||||
async function triggerUserRenderProbe(page: Page): Promise<void> {
|
||||
const send = page.getByRole('button', { name: 'Send message', exact: true })
|
||||
await send.click()
|
||||
}
|
||||
|
||||
async function stopUserRenderProbe(
|
||||
page: Page,
|
||||
marker: string,
|
||||
): Promise<UserRenderProbeResult> {
|
||||
try {
|
||||
await page.waitForFunction(() => {
|
||||
const probe = Reflect.get(globalThis, '__dshPerfUserRenderProbe') as
|
||||
| { paintAt?: number }
|
||||
| undefined
|
||||
return probe?.paintAt !== undefined
|
||||
}, undefined, { timeout: 15_000 })
|
||||
} catch (error) {
|
||||
const diagnostic = await page.evaluate((expectedMarker) => {
|
||||
const probe = Reflect.get(globalThis, '__dshPerfUserRenderProbe') as
|
||||
| {
|
||||
sendAt?: number
|
||||
domAt?: number
|
||||
paintAt?: number
|
||||
trustedClick?: boolean
|
||||
batches: number
|
||||
records: number
|
||||
}
|
||||
| undefined
|
||||
const target = document.querySelector('[data-conversation-scroll]')
|
||||
let markerOutsideComposer = false
|
||||
if (target !== null) {
|
||||
const walker = document.createTreeWalker(target, NodeFilter.SHOW_TEXT)
|
||||
for (let node = walker.nextNode(); node !== null; node = walker.nextNode()) {
|
||||
if (!node.textContent?.includes(expectedMarker)) continue
|
||||
const parent = node.parentElement
|
||||
if (parent !== null && parent.closest('[data-composer-card]') === null) {
|
||||
markerOutsideComposer = true
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return {
|
||||
probe,
|
||||
markerOutsideComposer,
|
||||
markerInComposer: document.querySelector('[data-composer-card]')
|
||||
?.textContent?.includes(expectedMarker) ?? false,
|
||||
}
|
||||
}, marker)
|
||||
throw new Error(`user render probe timed out: ${JSON.stringify(diagnostic)}`, { cause: error })
|
||||
}
|
||||
return page.evaluate(() => {
|
||||
const probe = Reflect.get(globalThis, '__dshPerfUserRenderProbe') as
|
||||
| {
|
||||
sendAt?: number
|
||||
domAt?: number
|
||||
paintAt?: number
|
||||
trustedClick?: boolean
|
||||
batches: number
|
||||
records: number
|
||||
}
|
||||
| undefined
|
||||
Reflect.deleteProperty(globalThis, '__dshPerfUserRenderProbe')
|
||||
if (
|
||||
probe?.trustedClick !== true
|
||||
|| probe.sendAt === undefined
|
||||
|| probe.domAt === undefined
|
||||
|| probe.paintAt === undefined
|
||||
) {
|
||||
throw new Error('user render probe did not observe a trusted click, DOM insertion, and paint')
|
||||
}
|
||||
return {
|
||||
trustedClick: probe.trustedClick,
|
||||
sendToDomMs: probe.domAt - probe.sendAt,
|
||||
sendToPaintMs: probe.paintAt - probe.sendAt,
|
||||
domToPaintMs: probe.paintAt - probe.domAt,
|
||||
mutationBatches: probe.batches,
|
||||
mutationRecords: probe.records,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
async function stableCount(
|
||||
locator: Locator,
|
||||
accepts: (count: number) => boolean,
|
||||
@@ -887,6 +1059,77 @@ async function continueConversation(
|
||||
}
|
||||
}
|
||||
|
||||
async function measurePostSoakUserRender(
|
||||
world: PerformanceWorld,
|
||||
cdp: CDPSession,
|
||||
): Promise<object> {
|
||||
const spec = soakTurn(POST_SOAK_RENDER_TURN)
|
||||
if (spec.toolResultMarker !== undefined) {
|
||||
throw new Error('post-soak render probe must remain a text-only turn')
|
||||
}
|
||||
const composer = world.page.locator('textarea:enabled').last()
|
||||
const composerFill = await measure(cdp, async () => {
|
||||
await composer.fill(spec.prompt)
|
||||
await expect.poll(() => composer.inputValue()).toBe(spec.prompt)
|
||||
return (await composer.inputValue()).length
|
||||
})
|
||||
expect(composerFill.value).toBe(spec.prompt.length)
|
||||
|
||||
const eventStart = world.sessionEvents.length
|
||||
await startUserRenderProbe(world.page, spec.userMarker)
|
||||
const browserBefore = await chromiumMetrics(cdp)
|
||||
const fullTurnStarted = performance.now()
|
||||
const settled = world.scaffold.whenTurnSettled(60_000)
|
||||
await triggerUserRenderProbe(world.page)
|
||||
const browserTiming = await stopUserRenderProbe(world.page, spec.userMarker)
|
||||
const browserAfter = await chromiumMetrics(cdp)
|
||||
const browserAfterPaintSample = metricDelta(
|
||||
browserBefore,
|
||||
browserAfter,
|
||||
performance.now() - fullTurnStarted,
|
||||
)
|
||||
await world.page.getByText(spec.firstMarker, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
const settledSessionId = await settled
|
||||
await expect.poll(
|
||||
() => world.page.locator('[data-streaming="true"]').count(),
|
||||
{ timeout: 15_000 },
|
||||
).toBe(0)
|
||||
await world.page.getByText(spec.doneMarker, { exact: false }).last().waitFor({ timeout: 15_000 })
|
||||
const fullTurnMs = performance.now() - fullTurnStarted
|
||||
|
||||
const turnEvents = world.sessionEvents.slice(eventStart)
|
||||
const chunks = turnEvents.filter(event => event.type === 'assistant/chunk')
|
||||
const user = turnEvents.find(
|
||||
event => event.type === 'user/message' && event.data.source.kind === 'user',
|
||||
)
|
||||
if (user?.type !== 'user/message') throw new Error('post-soak render probe did not log its user message')
|
||||
expect(user.data.content
|
||||
.filter(block => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')).toBe(spec.prompt)
|
||||
expect(chunks).toHaveLength(spec.deltas.length + 4)
|
||||
expect(turnEvents.filter(event => event.type === 'tool/call')).toHaveLength(0)
|
||||
expect(turnEvents.filter(event => event.type === 'tool/result')).toHaveLength(0)
|
||||
expect(world.scaffold.ctx.agents.get(settledSessionId)).toBeDefined()
|
||||
expect(await conversationTurns(world.page)).toBe(POST_SOAK_RENDER_TURN)
|
||||
|
||||
return {
|
||||
ordinal: POST_SOAK_RENDER_TURN,
|
||||
resultingTurns: POST_SOAK_RENDER_TURN,
|
||||
promptChars: spec.prompt.length,
|
||||
composerFill: composerFill.measurement,
|
||||
trustedClick: browserTiming.trustedClick,
|
||||
sendToDomMs: rounded(browserTiming.sendToDomMs),
|
||||
sendToPaintMs: rounded(browserTiming.sendToPaintMs),
|
||||
domToPaintMs: rounded(browserTiming.domToPaintMs),
|
||||
mutationBatchesThroughPaint: browserTiming.mutationBatches,
|
||||
mutationRecordsThroughPaint: browserTiming.mutationRecords,
|
||||
browserAfterPaintSample,
|
||||
fullTurnMs: rounded(fullTurnMs),
|
||||
persistedChunks: chunks.length,
|
||||
}
|
||||
}
|
||||
|
||||
function average(values: readonly number[]): number {
|
||||
return rounded(values.reduce((sum, value) => sum + value, 0) / values.length)
|
||||
}
|
||||
@@ -1142,11 +1385,12 @@ describe('manual web performance: complex workspace and history', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('reports one hundred continuously generated turns with retained checkpoints', async () => {
|
||||
it('reports one hundred generated turns and the next user-message paint', async () => {
|
||||
const world = await launchPerformanceWorld({
|
||||
browser,
|
||||
replay: performanceReplayOverride(SOAK_TURNS, soakTurn),
|
||||
replay: performanceReplayOverride(POST_SOAK_RENDER_TURN, soakTurn),
|
||||
})
|
||||
let testFailure: unknown
|
||||
try {
|
||||
await world.page.goto(world.scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await world.page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
@@ -1160,6 +1404,7 @@ describe('manual web performance: complex workspace and history', () => {
|
||||
checkpointInterval: SOAK_CHECKPOINT_INTERVAL,
|
||||
})
|
||||
expect(conversation.toolTurns).toBe(SOAK_TURNS / SOAK_TOOL_INTERVAL)
|
||||
const postSoakUserRender = await measurePostSoakUserRender(world, cdp)
|
||||
const { turns, ...retained } = conversation
|
||||
console.info(`WEB_PERF_RESULT ${JSON.stringify({
|
||||
scenario: 'continuous-100-turn-soak',
|
||||
@@ -1167,11 +1412,23 @@ describe('manual web performance: complex workspace and history', () => {
|
||||
replayContextWindow: PERF_REPLAY_CONTEXT_WINDOW,
|
||||
...retained,
|
||||
windows: summarizeTurnWindows(turns, SOAK_CHECKPOINT_INTERVAL),
|
||||
postSoakUserRender,
|
||||
}, null, 2)}`)
|
||||
expect(world.tripwire.warnings).toEqual([])
|
||||
expect(world.tripwire.pageErrors).toEqual([])
|
||||
} catch (error) {
|
||||
testFailure = error
|
||||
throw error
|
||||
} finally {
|
||||
await closePerformanceWorld(world)
|
||||
try {
|
||||
await closePerformanceWorld(world)
|
||||
} catch (cleanupError) {
|
||||
if (testFailure === undefined) throw cleanupError
|
||||
throw new AggregateError(
|
||||
[testFailure, cleanupError],
|
||||
'continuous conversation performance test and teardown failed',
|
||||
)
|
||||
}
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -9,9 +9,9 @@
|
||||
import { mkdir, readFile, writeFile } from 'node:fs/promises'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { join } from 'node:path'
|
||||
import type { Browser, Page } from 'playwright'
|
||||
import type { Browser, Page, Response } from 'playwright'
|
||||
import { chromium } from 'playwright'
|
||||
import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
@@ -34,12 +34,46 @@ const SEED_ID = 'navigation-panes-web-e2e'
|
||||
const PROMPT_TURN1 = 'NavScenario: first run bash to print exactly NAVIGATION_OK, then read nav-a.md and nav-b.md using two read calls in ONE assistant message, then reply with the single word FIRST_DONE and stop.'
|
||||
const PROMPT_TURN2 = 'Reply in markdown with: a level-2 heading "Navigation Summary", a bulleted list of exactly two items, and a fenced code block containing echo WATERFALL. Then stop.'
|
||||
|
||||
async function baselineResponse(
|
||||
page: Page,
|
||||
method: 'session.list' | 'workspace.list',
|
||||
): Promise<Response> {
|
||||
return page.waitForResponse(response => (
|
||||
response.request().method() === 'POST'
|
||||
&& new URL(response.url()).pathname === `/api/${method}`
|
||||
), { timeout: 30_000 })
|
||||
}
|
||||
|
||||
async function assertBaselineSucceeded(response: Response, method: string): Promise<void> {
|
||||
expect(response.ok(), `${method} baseline HTTP response`).toBe(true)
|
||||
const body = await response.json() as { result?: { ok?: unknown } }
|
||||
expect(body.result?.ok, `${method} baseline RPC result`).toBe(true)
|
||||
}
|
||||
|
||||
async function ensureSeedOpen(page: Page): Promise<void> {
|
||||
const chat = page.getByRole('tab', { name: 'Chat', exact: true })
|
||||
const search = page.getByPlaceholder('Search name, keywords', { exact: false })
|
||||
if (await chat.count() === 0) {
|
||||
await search.fill('WATERFALL')
|
||||
const result = page.getByRole('tree', { name: 'Search results' }).getByRole('treeitem')
|
||||
await expect.poll(() => result.count(), { timeout: 15_000 }).toBe(1)
|
||||
await result.click()
|
||||
await chat.waitFor({ timeout: 15_000 })
|
||||
}
|
||||
await chat.click()
|
||||
await page.getByText('FIRST_DONE', { exact: true }).waitFor({ timeout: 15_000 })
|
||||
if (await search.inputValue() !== '') {
|
||||
await search.fill('')
|
||||
await expect.poll(() => search.inputValue(), { timeout: 5_000 }).toBe('')
|
||||
}
|
||||
}
|
||||
|
||||
describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
let scaffold: WebScaffold
|
||||
let browser: Browser
|
||||
let page: Page
|
||||
let tripwire: ReturnType<typeof watchConsole>
|
||||
let slotErrors: string[]
|
||||
let tripwire: ReturnType<typeof watchConsole> = { warnings: [], pageErrors: [] }
|
||||
let slotErrors: string[] = []
|
||||
|
||||
beforeAll(async () => {
|
||||
scaffold = await launchWebScaffold({})
|
||||
@@ -57,6 +91,9 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
await seedSession(scaffold, raw, SEED_ID)
|
||||
}
|
||||
browser = await chromium.launch()
|
||||
}, 120_000)
|
||||
|
||||
beforeEach(async () => {
|
||||
page = await newEnglishPage(browser)
|
||||
tripwire = watchConsole(page)
|
||||
slotErrors = []
|
||||
@@ -65,7 +102,19 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
slotErrors.push(message.text())
|
||||
}
|
||||
})
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
// Initial navigation and list ownership settle only after both independent
|
||||
// RPC baselines succeed; arm before navigation so neither response is missed.
|
||||
const sessionBaseline = baselineResponse(page, 'session.list')
|
||||
const workspaceBaseline = baselineResponse(page, 'workspace.list')
|
||||
const [, sessionResponse, workspaceResponse] = await Promise.all([
|
||||
page.goto(scaffold.baseUrl, { waitUntil: 'load' }),
|
||||
sessionBaseline,
|
||||
workspaceBaseline,
|
||||
])
|
||||
await Promise.all([
|
||||
assertBaselineSucceeded(sessionResponse, 'session.list'),
|
||||
assertBaselineSucceeded(workspaceResponse, 'workspace.list'),
|
||||
])
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
// The frame mounts before the asynchronous session-list baseline lands.
|
||||
// Search must target the settled seeded row, not the startup input that
|
||||
@@ -73,9 +122,32 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
await page.getByText('1 session', { exact: true }).waitFor({ timeout: 30_000 })
|
||||
}, 120_000)
|
||||
|
||||
afterEach(async () => {
|
||||
const failures: unknown[] = []
|
||||
try {
|
||||
expect({
|
||||
pageErrors: tripwire.pageErrors,
|
||||
slotErrors,
|
||||
warnings: tripwire.warnings,
|
||||
}).toEqual({
|
||||
pageErrors: [],
|
||||
slotErrors: [],
|
||||
warnings: [],
|
||||
})
|
||||
} catch (error) {
|
||||
failures.push(error)
|
||||
}
|
||||
await page?.close().catch((error: unknown) => failures.push(error))
|
||||
if (failures.length === 1) throw failures[0]
|
||||
if (failures.length > 1) throw new AggregateError(failures, 'navigation case cleanup failed')
|
||||
})
|
||||
|
||||
afterAll(async () => {
|
||||
await browser?.close()
|
||||
await scaffold?.close()
|
||||
const failures: unknown[] = []
|
||||
await browser?.close().catch((error: unknown) => failures.push(error))
|
||||
await scaffold?.close().catch((error: unknown) => failures.push(error))
|
||||
if (failures.length === 1) throw failures[0]
|
||||
if (failures.length > 1) throw new AggregateError(failures, 'navigation e2e cleanup failed')
|
||||
})
|
||||
|
||||
it.skipIf(MODE !== 'record')('records the two-turn seed live through the composer', async () => {
|
||||
@@ -102,6 +174,9 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
|
||||
it.skipIf(MODE === 'record')('finds an unopened seeded session by message content and opens it', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-navigation-search'))
|
||||
// The API baselines can settle before React commits their projection. The
|
||||
// seeded count is the final user-visible barrier before editing search.
|
||||
await page.getByText('1 session', { exact: true }).waitFor({ timeout: 30_000 })
|
||||
const search = page.getByPlaceholder('Search name, keywords', { exact: false })
|
||||
// The cold row has not been opened, so only the persisted log can satisfy
|
||||
// this query. First search lazily reconciles the SQLite content index.
|
||||
@@ -136,6 +211,7 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
|
||||
it.skipIf(MODE === 'record')('renders the trajectory ledger and opens its local record inspector', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-navigation-trajectory'))
|
||||
await ensureSeedOpen(page)
|
||||
await page.getByRole('tab', { name: 'Trajectory' }).click()
|
||||
await page.waitForTimeout(100)
|
||||
const overlayLayout = await page.getByRole('table').evaluate((table) => {
|
||||
@@ -200,7 +276,10 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
|
||||
it.skipIf(MODE === 'record')('focuses the ledger by dragging an overview interval', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-navigation-timeline'))
|
||||
await ensureSeedOpen(page)
|
||||
await page.getByRole('tab', { name: 'Trajectory' }).click()
|
||||
const plot = page.getByLabel('Timeline overview; drag horizontally to focus events')
|
||||
await plot.waitFor({ timeout: 15_000 })
|
||||
const before = await page.locator('tr[data-kind]').count()
|
||||
const box = await plot.boundingBox()
|
||||
if (box === null) throw new Error('trajectory timeline plot has no layout box')
|
||||
@@ -217,7 +296,7 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
|
||||
it.skipIf(MODE === 'record')('bash and file-path rows leave the default details column closed', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-navigation-details'))
|
||||
await page.getByRole('tab', { name: 'Chat' }).click()
|
||||
await ensureSeedOpen(page)
|
||||
const bashRow = page.locator('[data-sample="bash"]').first()
|
||||
await bashRow.waitFor({ timeout: 15_000 })
|
||||
const frame = page.locator('[style*="grid-template-columns"]').first()
|
||||
@@ -239,9 +318,9 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
|
||||
it.skipIf(MODE === 'record')('renders the bash row as a terminal card in the real browser', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-navigation-terminal'))
|
||||
await page.getByRole('tab', { name: 'Chat' }).click()
|
||||
await ensureSeedOpen(page)
|
||||
// The card is expand-gated behind the whole-row toggle (the unified
|
||||
// tool-row interaction): open it if a previous case left it collapsed.
|
||||
// tool-row interaction): open it if this fresh view leaves it collapsed.
|
||||
// Expanded, the recorded command's own output sits in the message flow,
|
||||
// derived from the logged call/result presentations alone.
|
||||
const bashRow = page.locator('[data-sample="bash"]').first()
|
||||
@@ -322,10 +401,7 @@ describe('web e2e: navigation & panes over a rich seeded session', () => {
|
||||
expect(await page.evaluate(() => navigator.clipboard.readText())).toContain('NAVIGATION_OK')
|
||||
}, 60_000)
|
||||
|
||||
it.skipIf(MODE === 'record')('issued zero model calls and stayed clean', async () => {
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(slotErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
it.skipIf(MODE === 'record')('keeps the recorded fixture inventory exact', async () => {
|
||||
await assertFixtureInventory(SNAPSHOT_DIR, [
|
||||
'seed.jsonl', 'search-results.expected.md', 'trajectory.expected.md',
|
||||
'terminal-card.expected.md',
|
||||
|
||||
@@ -57,6 +57,10 @@
|
||||
"tests/startup-auto-selection.e2e.ts",
|
||||
"tests/subagent-conversation.e2e.ts",
|
||||
"tests/bash-abort-row.e2e.ts",
|
||||
"tests/chat-scroll-fixture.ts",
|
||||
"tests/chat-scroll-contract.e2e.ts",
|
||||
"tests/chat-long-interactions.e2e.ts",
|
||||
"tests/chat-continuous-conversation.e2e.ts",
|
||||
"tests/complex-history.perf.ts"
|
||||
],
|
||||
"references": [
|
||||
|
||||
@@ -44,6 +44,10 @@
|
||||
"apps/web/tests/startup-auto-selection.e2e.ts",
|
||||
"apps/web/tests/subagent-conversation.e2e.ts",
|
||||
"apps/web/tests/bash-abort-row.e2e.ts",
|
||||
"apps/web/tests/chat-scroll-fixture.ts",
|
||||
"apps/web/tests/chat-scroll-contract.e2e.ts",
|
||||
"apps/web/tests/chat-long-interactions.e2e.ts",
|
||||
"apps/web/tests/chat-continuous-conversation.e2e.ts",
|
||||
"apps/web/tests/complex-history.perf.ts",
|
||||
"apps/cli/tests/**/*.ts",
|
||||
"examples/*/src/**/*.ts",
|
||||
|
||||
Reference in New Issue
Block a user