Yichen Jiang
96c44c83d4
The TTFT and tok/s readings divide by measured wall time, so they are not
reproducible: the same replayed scenario yielded 69 and 70 tok/s on
consecutive local runs, and a 3 ms replayed stream reads 26333 tok/s. Baking
those into committed goldens made the Web lane flaky by construction, and the
goldens for the readings themselves were never refreshed.
Three fixes, then a refresh:
The footer's decorative dots are `aria-hidden`, so the readings concatenated
into one accessible string — `Ran for 13sTTFT 0.2s12 tok/s`. That is a real
defect on its own (a reader hears one run-on instead of three facts) and it
also denied `{{duration}}` the word boundary it matches on, so even the
previously-stable `Ran for` duration started leaking raw. The separators now
carry flanking spaces.
`normalizeAria` gains `{{throughput}}` beside `{{duration}}`, and its duration
alternation accepts the stats line's compact `2m42s` as well as the
message-chrome template's `2m 42s` — the compact form had no pattern at all,
which is why `LLM 382m39s` survived the first refresh.
Refreshed 17 goldens. They also record that the stats line's `LLM` group now
renders at all: it folds assistant `timing`, which the live transcript adapter
only began attaching in this branch, so the group was previously dead in Chat.
Verified by running the lane in replay three times after the refresh: 41/41
files green each time, goldens untouched. Before this change two consecutive
runs disagreed on both the values and the failure count.
2.1 KiB
2.1 KiB
- banner:
- navigation "Session hierarchy":
- button "Use the read tool twice" [disabled]
- tablist:
- tab "Chat" [selected]
- tab "Trajectory"
- navigation "Session hierarchy":
- text: "Use the read tool twice in one assistant message: read a.txt and b.txt. Then reply with the single word DONE and stop. 7/25 {{clock}}"
- button "Copy":
- img
- tooltip "Copy"
- button "Branch into a new conversation" [disabled]:
- img
- text: Available only on the last message of a completed turn
- button "Think The user wants me to read a.txt and b.txt, then reply with "DONE". Let me do both reads in parallel.":
- img
- img
- text: Think The user wants me to read a.txt and b.txt, then reply with "DONE". Let me do both reads in parallel.
- paragraph: I will read both files before answering.
- button "Copy":
- img
- button "Branch into a new conversation" [disabled]:
- img
- text: Available only on the last message of a completed turn 7/25 {{clock}} Ran for {{duration}} TTFT {{duration}} {{throughput}} tok/s
- button "Read a.txt":
- img
- img
- text: Read
- button "a.txt"
- button "Read b.txt":
- img
- img
- text: Read
- button "b.txt"
- button "Think Both files have been read. a.txt contains "alpha" and b.txt contains "beta". I'll now reply with DONE as instructed.":
- img
- img
- text: Think Both files have been read. a.txt contains "alpha" and b.txt contains "beta". I'll now reply with DONE as instructed.
- text: Stopped Now give the final answer. 7/25 {{clock}}
- button "Copy":
- img
- button "Branch into a new conversation" [disabled]:
- img
- text: Available only on the last message of a completed turn
- paragraph: DONE
- button "Copy":
- img
- button "Branch into a new conversation":
- img
- text: 7/25 {{clock}} Ran for {{duration}}
- textbox "Message the agent"
- button "Commands":
- img
- 'button "Access mode, current: Workspace Write"': Workspace Write
- button "Select model":
- text: Select model
- img
- button "Send message" [disabled]
- text: 2 turns · 3 steps LLM {{duration}} · Tool call {{duration}} TTFT avg {{duration}} · {{throughput}} tok/s Cache hit 98% Input 7.8K tok · Output 103 tok