fix: address markdown renderer review findings
Restore the replaced pipeline's synthetic fence newline so a real trailing blank line inside a fence survives CodeBlock's display trim, and pin it plus header-only tables in the DOM-parity corpus. Route extractMarkdownPlainText through parseGfm so the grammar has one source. Document the fixture provenance check (all 46 fixtures reproduce byte-identically from react-markdown at 9e8101b800), the deliberate O(prefix) divergence memcmp, the per-arm scope of grammar consistency, the React 18 MathML namespace limitation, and the prefix-equivalence corpus constraint.
This commit is contained in:
07akioni
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-06-web-markdown-incremental-ast-renderer.md
|
||||
2026-08-06-web-markdown-incremental-ast-renderer.md: 98d28fa27f8e6e4b5f2ac21831c8f80f8c0c3631
|
||||
2026-08-06-web-markdown-incremental-ast-renderer.zh.md: c2c19fe36b8290c1802f6e2b321336e1a68d686f
|
||||
2026-08-06-web-markdown-incremental-ast-renderer.md: 3599bfcc78dc4eefe5e82f461a469bdba15f3aae
|
||||
2026-08-06-web-markdown-incremental-ast-renderer.zh.md: 2e00977da58ef29a77c45abcecf0f3bb62737929
|
||||
+1
-1
@@ -14,7 +14,7 @@ English | [中文](2026-08-06-web-markdown-incremental-ast-renderer.zh.md)
|
||||
|
||||
- **Grammars** ([parse.ts](../../../../packages/client/ui-primitives/src/markdown/parse.ts)): `parseGfm` (streaming arm and `extractMarkdownPlainText`) and `parseGfmWithMath` (settled arm) call `mdast-util-from-markdown` with the same micromark extensions the replaced remark plugins wrapped, so block boundaries are identical everywhere. `mathCompatibility` (ex `remarkMathCompatibility`) now exports its micromark extension directly.
|
||||
- **Incremental parsing** ([incremental.ts](../../../../packages/client/ui-primitives/src/markdown/incremental.ts)): CommonMark block parsing is line-based, so appended text reshapes only the parse frontier. `IncrementalMarkdownParser` keeps the trailing two blocks unstable (the last block is the frontier; the second-to-last is safety margin), freezes everything before them, and re-parses only the source tail from the last frozen block's `position.end.offset` — the parser's own offsets, no bespoke source scanning. Each source region parses O(1) times per stream instead of once per chunk; a single giant block (an unclosed fence) degrades to the old full-reparse cost and no worse. Non-append input resets the state under a bumped generation.
|
||||
- **Rendering** ([render.tsx](../../../../packages/client/ui-primitives/src/markdown/render.tsx), [katex.tsx](../../../../packages/client/ui-primitives/src/markdown/katex.tsx)): one switch over mdast node types replaces remark-rehype + react-markdown, reproducing the replaced pipeline's DOM byte-for-byte — table alignment as `text-align` styles, tight-list paragraph unwrapping, task-list classes and checkbox spacing, the footnote section (whose in-page anchors the protocol allowlist already reduced to plain text), literal raw HTML, the separator newlines that surface next to literal HTML text, and rehype-katex's three-arm error chain with KaTeX HTML mapped to React through the browser's own `DOMParser` (no wrapper element, so first/last-child margin rules still reach `.katex-display`). Frozen blocks cache their React elements and keep source-offset keys, so crossing the freeze boundary reconciles instead of remounting; `MarkdownText` is memoized.
|
||||
- **Rendering** ([render.tsx](../../../../packages/client/ui-primitives/src/markdown/render.tsx), [katex.tsx](../../../../packages/client/ui-primitives/src/markdown/katex.tsx)): one switch over mdast node types replaces remark-rehype + react-markdown, reproducing the replaced pipeline's DOM byte-for-byte — table alignment as `text-align` styles, tight-list paragraph unwrapping, task-list classes and checkbox spacing, the footnote section (whose in-page anchors the protocol allowlist already reduced to plain text), literal raw HTML, the separator newlines that surface next to literal HTML text, and rehype-katex's three-arm error chain with KaTeX HTML mapped to React through the browser's own `DOMParser` (no wrapper element, so first/last-child margin rules still reach `.katex-display`; React 18 puts the `.katex-mathml` subtree in the HTML namespace exactly as the replaced pipeline did — a pre-existing limitation outside this parity contract, invisible to the visual `.katex-html` arm). Frozen blocks cache their React elements and keep source-offset keys, so crossing the freeze boundary reconciles instead of remounting; `MarkdownText` is memoized.
|
||||
|
||||
The DOM is pinned by `tests/fixtures/markdown-dom`: fixtures recorded from the react-markdown implementation before the swap, which the new renderer must reproduce under a whitespace-normalizing serializer. A fixture diff is a user-visible markdown style change to review, never to re-record for a refactor. `tests/markdown-incremental.spec.tsx` holds the equivalence property — at every appended prefix, chunked at 1/3/7/16 bytes, the live component's DOM equals a fresh mount's — plus freeze-boundary DOM-node identity and reset behavior.
|
||||
|
||||
|
||||
+1
-1
@@ -14,7 +14,7 @@ Status: implemented
|
||||
|
||||
- **语法**([parse.ts](../../../../packages/client/ui-primitives/src/markdown/parse.ts)):`parseGfm`(流式臂与 `extractMarkdownPlainText`)和 `parseGfmWithMath`(定稿臂)以被替换的 remark 插件所包装的同一组 micromark 扩展调用 `mdast-util-from-markdown`,因此各处块边界完全一致。`mathCompatibility`(原 `remarkMathCompatibility`)现在直接导出其 micromark 扩展。
|
||||
- **增量解析**([incremental.ts](../../../../packages/client/ui-primitives/src/markdown/incremental.ts)):CommonMark 块解析按行推进,追加文本只会重塑解析前沿。`IncrementalMarkdownParser` 保留末尾两个块不稳定(最后一块是前沿;倒数第二块是安全裕量),冻结其前的所有块,只从最后一个冻结块的 `position.end.offset` 起重新解析源尾部——用的是解析器自己的偏移量,没有任何自制源扫描。每个源区间在整个流中解析 O(1) 次而非每 chunk 一次;单个巨型块(未闭合围栏)退化为旧的全量重解析成本,不会更差。非追加输入在递增的 generation 下重置状态。
|
||||
- **渲染**([render.tsx](../../../../packages/client/ui-primitives/src/markdown/render.tsx)、[katex.tsx](../../../../packages/client/ui-primitives/src/markdown/katex.tsx)):一个对 mdast 节点类型的 switch 取代 remark-rehype + react-markdown,逐字节复刻被替换管线的 DOM——表格对齐渲染为 `text-align` 样式、紧凑列表段落解包、任务列表类名与复选框空格、脚注区(其页内锚点本就被协议白名单降为纯文本)、字面 raw HTML、会与字面 HTML 文本相邻显形的分隔换行,以及 rehype-katex 的三臂容错链,KaTeX HTML 经浏览器自带的 `DOMParser` 映射为 React(无包裹元素,首/末子元素的 margin 规则仍能作用于 `.katex-display`)。冻结块缓存其 React 元素并保持源偏移 key,跨过冻结边界时走 reconcile 而非重挂载;`MarkdownText` 已 memo 化。
|
||||
- **渲染**([render.tsx](../../../../packages/client/ui-primitives/src/markdown/render.tsx)、[katex.tsx](../../../../packages/client/ui-primitives/src/markdown/katex.tsx)):一个对 mdast 节点类型的 switch 取代 remark-rehype + react-markdown,逐字节复刻被替换管线的 DOM——表格对齐渲染为 `text-align` 样式、紧凑列表段落解包、任务列表类名与复选框空格、脚注区(其页内锚点本就被协议白名单降为纯文本)、字面 raw HTML、会与字面 HTML 文本相邻显形的分隔换行,以及 rehype-katex 的三臂容错链,KaTeX HTML 经浏览器自带的 `DOMParser` 映射为 React(无包裹元素,首/末子元素的 margin 规则仍能作用于 `.katex-display`;React 18 会把 `.katex-mathml` 子树放进 HTML 命名空间,与被替换管线完全一致——既有限制,不在本对等性契约范围内,对承担视觉渲染的 `.katex-html` 臂不可见)。冻结块缓存其 React 元素并保持源偏移 key,跨过冻结边界时走 reconcile 而非重挂载;`MarkdownText` 已 memo 化。
|
||||
|
||||
DOM 由 `tests/fixtures/markdown-dom` 钉死:fixture 录制自替换前的 react-markdown 实现,新渲染器必须在空白规整序列化器下复现。fixture 差异即用户可见的 markdown 样式变更,必须按此评审,绝不能为重构而重录。`tests/markdown-incremental.spec.tsx` 承载等价性性质——以 1/3/7/16 字节分块,在每个追加前缀处,常驻组件的 DOM 都等于全新挂载——外加冻结边界的 DOM 节点同一性与重置行为。
|
||||
|
||||
|
||||
@@ -58,7 +58,10 @@ export interface IncrementalBlocks {
|
||||
/**
|
||||
* A block's render key: its absolute source start offset. A position-less
|
||||
* node (a grammar is free to omit positions) falls back to a negative
|
||||
* list-index key, which keeps sibling keys unique without inventing offsets.
|
||||
* list-index key — unique within one update's tail, which is the only place
|
||||
* the fallback can occur: freezing requires the cut block's position, so a
|
||||
* position-less parse keeps every block in the tail (real grammars always
|
||||
* stamp positions and never take this path).
|
||||
*/
|
||||
function blockKey(node: RootContent, base: number, index: number): number {
|
||||
const offset = node.position?.start.offset
|
||||
@@ -88,6 +91,12 @@ export class IncrementalMarkdownParser {
|
||||
*/
|
||||
update(text: string): IncrementalBlocks {
|
||||
if (this.cached !== null && text === this.prevText) return this.cached
|
||||
// Deliberate O(prefix) memcmp per update: sound divergence detection has
|
||||
// to verify the whole retained prefix, and startsWith compares bytes two
|
||||
// orders of magnitude faster than parsing them — the cost this class
|
||||
// exists to remove. Passing append/reset deltas instead would push
|
||||
// append bookkeeping across the session-projection seam for a check
|
||||
// that stays sub-millisecond at realistic reply sizes.
|
||||
if (!text.startsWith(this.prevText)) {
|
||||
this.prevText = ''
|
||||
this.tailStart = 0
|
||||
|
||||
@@ -7,6 +7,12 @@
|
||||
* adjustments KaTeX output relies on) turns it into a tree this module maps
|
||||
* onto React elements — KaTeX output is a static span/MathML/SVG vocabulary
|
||||
* with no raw user HTML, the same trust shiki's tree gets in CodeBlock.
|
||||
*
|
||||
* React 18 has no MathML support, so the `.katex-mathml` subtree's elements
|
||||
* land in the HTML namespace — exactly as they did under the replaced
|
||||
* hast-util-to-jsx-runtime pipeline. The visual arm is the `.katex-html`
|
||||
* span tree; the MathML arm serves assistive technology, which reads it by
|
||||
* tag name regardless of namespace.
|
||||
*/
|
||||
|
||||
import { createElement } from 'react'
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
/**
|
||||
* The markdown renderer's two mdast grammars, one per rendering arm. Both are
|
||||
* built from the same micromark extensions, so block boundaries and inline
|
||||
* semantics are identical wherever a document (or a document tail) is parsed:
|
||||
* the incremental streaming path, the settled path, and the plain-text
|
||||
* projection all agree on where blocks start and end.
|
||||
* The markdown renderer's two mdast grammars, one per rendering arm. Each
|
||||
* arm is internally consistent — the incremental tail parses, the one-shot
|
||||
* parses, and the plain-text projection of a given grammar always agree on
|
||||
* where blocks start and end — and the settled grammar is the streaming one
|
||||
* plus the math extensions, so the arms differ only where TeX delimiters
|
||||
* begin a math construct (a `$$` block is a paragraph while streaming and a
|
||||
* math block once settled, by design).
|
||||
*/
|
||||
|
||||
import type { Root } from 'mdast'
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
/**
|
||||
* Markdown-to-plain-text projection for compact summaries and labels.
|
||||
* Parsing shares the renderer's GFM grammar; raw HTML stays literal, links
|
||||
* keep their labels, images keep alt text, and code keeps its source text.
|
||||
* Parsing shares the renderer's streaming GFM grammar ({@link parseGfm}), so
|
||||
* the projection strips exactly the markup the renderer would draw; raw HTML
|
||||
* stays literal, links keep their labels, images keep alt text, and code
|
||||
* keeps its source text.
|
||||
*/
|
||||
|
||||
import { fromMarkdown } from 'mdast-util-from-markdown'
|
||||
import { gfmFromMarkdown } from 'mdast-util-gfm'
|
||||
import { gfm } from 'micromark-extension-gfm'
|
||||
import { parseGfm } from './parse.ts'
|
||||
|
||||
/** Amount of parsed Markdown content returned by the extractor. */
|
||||
export type MarkdownPlainTextMode = 'all' | 'first-line' | 'first-paragraph'
|
||||
@@ -108,10 +108,7 @@ export function extractMarkdownPlainText(
|
||||
options: MarkdownPlainTextOptions = {},
|
||||
): string {
|
||||
const { mode = 'all' } = options
|
||||
const root = fromMarkdown(markdown, {
|
||||
extensions: [gfm()],
|
||||
mdastExtensions: [gfmFromMarkdown()],
|
||||
}) as MarkdownNode
|
||||
const root = parseGfm(markdown) as MarkdownNode
|
||||
const all = fullText(root)
|
||||
switch (mode) {
|
||||
case 'all':
|
||||
|
||||
@@ -279,7 +279,10 @@ function renderCode(node: Md.Code, key: Key, context: MarkdownRenderContext): Re
|
||||
return (
|
||||
<CodeBlock
|
||||
key={key}
|
||||
code={node.value}
|
||||
// The replaced hast pipeline appended one synthetic newline that
|
||||
// CodeBlock's display trim removes; feeding the bare value would make
|
||||
// that trim eat a REAL trailing blank line inside the fence instead.
|
||||
code={`${node.value}\n`}
|
||||
lang={context.streaming ? undefined : lang}
|
||||
copyLabel={context.codeLabels?.copyLabel}
|
||||
copiedLabel={context.codeLabels?.copiedLabel}
|
||||
|
||||
Vendored
+37
@@ -0,0 +1,37 @@
|
||||
<div class="_markdown_404681">
|
||||
<div class="_block_9aea57 md-code-block">
|
||||
<div class="_bannerWrap_9aea57">
|
||||
<div class="_banner_9aea57">
|
||||
<div class="_infostring_9aea57">
|
||||
<div class="_action_9aea57">
|
||||
<button class="_copyButton_9aea57" type="button">
|
||||
#text "复制"
|
||||
<pre class="_plain_9aea57">
|
||||
<code>
|
||||
#text "kept blank line follows\n"
|
||||
<div class="_block_9aea57 md-code-block">
|
||||
<div class="_bannerWrap_9aea57">
|
||||
<div class="_banner_9aea57">
|
||||
<div class="_infostring_9aea57">
|
||||
#text "ts"
|
||||
<div class="_action_9aea57">
|
||||
<button class="_copyButton_9aea57" type="button">
|
||||
#text "复制"
|
||||
<div>
|
||||
<pre class="shiki css-variables" style="background-color:var(--shiki-background);color:var(--shiki-foreground)" tabindex="0">
|
||||
<code>
|
||||
<span class="line">
|
||||
<span style="color:var(--shiki-token-keyword)">
|
||||
#text "const"
|
||||
<span style="color:var(--shiki-token-constant)">
|
||||
#text " doubled"
|
||||
<span style="color:var(--shiki-token-keyword)">
|
||||
#text " ="
|
||||
<span style="color:var(--shiki-token-constant)">
|
||||
#text " true"
|
||||
#text "\n"
|
||||
<span class="line">
|
||||
#text "\n"
|
||||
<span class="line">
|
||||
<p>
|
||||
#text "after"
|
||||
Vendored
+23
@@ -0,0 +1,23 @@
|
||||
<div class="_markdown_404681">
|
||||
<div class="_block_9aea57 md-code-block">
|
||||
<div class="_bannerWrap_9aea57">
|
||||
<div class="_banner_9aea57">
|
||||
<div class="_infostring_9aea57">
|
||||
<div class="_action_9aea57">
|
||||
<button class="_copyButton_9aea57" type="button">
|
||||
#text "复制"
|
||||
<pre class="_plain_9aea57">
|
||||
<code>
|
||||
#text "kept blank line follows\n"
|
||||
<div class="_block_9aea57 md-code-block">
|
||||
<div class="_bannerWrap_9aea57">
|
||||
<div class="_banner_9aea57">
|
||||
<div class="_infostring_9aea57">
|
||||
<div class="_action_9aea57">
|
||||
<button class="_copyButton_9aea57" type="button">
|
||||
#text "复制"
|
||||
<pre class="_plain_9aea57">
|
||||
<code>
|
||||
#text "const doubled = true\n\n"
|
||||
<p>
|
||||
#text "after"
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
<div class="_markdown_404681">
|
||||
<div class="_tableScroll_404681">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>
|
||||
#text "a"
|
||||
<th>
|
||||
#text "b"
|
||||
<p>
|
||||
#text "after"
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
<div class="_markdown_404681">
|
||||
<div class="_tableScroll_404681">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>
|
||||
#text "a"
|
||||
<th>
|
||||
#text "b"
|
||||
<p>
|
||||
#text "after"
|
||||
@@ -6,6 +6,15 @@
|
||||
// normalization), so a fixture diff means a user-visible markdown style
|
||||
// change and must be reviewed as such — never re-record to silence a
|
||||
// refactor.
|
||||
//
|
||||
// Provenance is reproducible: the replaced pipeline last lived at commit
|
||||
// 9e8101b800 (origin/master before the renderer swap merged). Checking out
|
||||
// that ref in a worktree, copying this spec, and running it records all
|
||||
// fixtures from react-markdown byte-identical to the ones committed here:
|
||||
// git worktree add /tmp/parity origin/master --detach && cd /tmp/parity
|
||||
// pnpm install && cp <this spec> packages/client/ui-primitives/tests/
|
||||
// npx vitest run packages/client/ui-primitives/tests/markdown-dom-parity.spec.tsx
|
||||
// diff -r <recorded fixtures> <this branch's fixtures> # byte-identical
|
||||
import { cleanup, render } from '@testing-library/react'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { MarkdownText } from '@deepseek-ai/dsh-client-ui-primitives'
|
||||
@@ -139,6 +148,21 @@ const CORPUS: Record<string, string> = {
|
||||
' indented code block',
|
||||
' second line',
|
||||
].join('\n'),
|
||||
'fence-trailing-blank-lines': [
|
||||
'```',
|
||||
'kept blank line follows',
|
||||
'',
|
||||
'```',
|
||||
'',
|
||||
'```ts',
|
||||
'const doubled = true',
|
||||
'',
|
||||
'',
|
||||
'```',
|
||||
'',
|
||||
'after',
|
||||
].join('\n'),
|
||||
'table-header-only': '| a | b |\n| --- | --- |\n\nafter',
|
||||
'inline-code-with-newline': 'Spans `a\nb` across a line.',
|
||||
'links-and-autolinks': [
|
||||
'[https ok](https://example.com "with title") and [mailto ok](mailto:dev@example.com).',
|
||||
|
||||
@@ -12,7 +12,15 @@ import { parseGfm } from '../src/markdown/parse.ts'
|
||||
|
||||
afterEach(cleanup)
|
||||
|
||||
/** A many-block document exercising every freeze-sensitive construct. */
|
||||
/**
|
||||
* A many-block document exercising every freeze-sensitive construct. The
|
||||
* prefix-equivalence property below holds only while no reference or
|
||||
* footnote definition lands on the far side of a freeze boundary from its
|
||||
* use: a fresh mount parses everything in one tree while the live stream's
|
||||
* frozen blocks are already baked (the fingerprint test demonstrates the
|
||||
* documented deviation). Keep definitions adjacent to their references when
|
||||
* extending this corpus.
|
||||
*/
|
||||
const STREAM_DOC = [
|
||||
'# Title',
|
||||
'',
|
||||
|
||||
Reference in New Issue
Block a user