fix(web): render CJK-adjacent strong emphasis
This commit is contained in:
Yichen Jiang
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-23-web-assistant-markdown.md
|
||||
2026-07-23-web-assistant-markdown.md: 8a8778351911bcb3448366c718aa124c4a89de58
|
||||
2026-07-23-web-assistant-markdown.zh.md: 2ac024e24ff95b4eb296112562c93f343b832187
|
||||
2026-07-23-web-assistant-markdown.md: 39f01c272a48a2f7e7ae89c15c2edc4efac4a0cb
|
||||
2026-07-23-web-assistant-markdown.zh.md: 50688aadecb88cb8d92263087eb88e86598af85e
|
||||
@@ -12,7 +12,7 @@ The Web conversation preserves assistant Markdown source through session events,
|
||||
|
||||
`@deepseek-ai/dsh-client-ui-primitives` exports `MarkdownText` as the untrusted assistant-text renderer, and `ui-conversation` selects it only for assistant `text` blocks. Finalized history, the streaming tail, and interrupted partials already share `AssistantMarkdown`, so they receive the same renderer without changing events or snapshots. User and steering messages keep `MessageText` and remain literal.
|
||||
|
||||
`MarkdownText` uses `react-markdown` with `remark-gfm` to build React elements from an AST. It covers CommonMark blocks plus GFM tables, task lists, strikethrough, and autolinks without raw-HTML parsing. Fenced code routes through the shared `CodeBlock`, which highlights registered grammars with the client's shiki singleton (`--shiki-*` tokens) and falls back to plain monospace otherwise. While a turn streams, fences stay on the plain arm so growing fences are not retokenized every chunk.
|
||||
`MarkdownText` uses `react-markdown` with `remark-gfm` to build React elements from an AST. It covers CommonMark blocks plus GFM tables, task lists, strikethrough, and autolinks without raw-HTML parsing. A micromark attention extension reuses the CommonMark resolver while letting runs of at least two asterisks close after Unicode punctuation when followed immediately by CJK text. This exception covers punctuation-terminated strong emphasis in whitespace-free CJK prose during streaming and after settlement; single-asterisk emphasis, non-CJK adjacency, escaped source, code, and math retain upstream parsing. Fenced code routes through the shared `CodeBlock`, which highlights registered grammars with the client's shiki singleton (`--shiki-*` tokens) and falls back to plain monospace otherwise. While a turn streams, fences stay on the plain arm so growing fences are not retokenized every chunk.
|
||||
|
||||
Visual spacing, tables, links, blockquotes, inline code, and code-block chrome follow deepsuite `@deepseek/md` (`markdown.css` / `code-block.css`) and the same `--dsw-alias-markdown-*`, `--dsw-font-markdown-*`, `--dsw-alias-border-l*`, and `--dsw-alias-label-*` tokens. Links use `--dsw-alias-state-business-primary` (deepsuite's sheet uses `--dsw-alias-brand-text`, which is blue only under newDesign; design-platform keeps brand-text near-black and is not retuned here). `CodeBlock` ships a language banner and a copy control (`复制` / `复制成功`). Finalized text renders KaTeX through `remark-math` and `rehype-katex`; `remarkMathCompatibility` maps `\(...\)`, `\[...\]`, and block-level same-line `$$...$$` to the same standard math AST nodes. This is a narrow parser compatibility layer, not a regex rewrite or malformed-model-output repair. Streaming stays literal until finalization so incomplete formulae do not flash errors. Citation pills, heading anchors, the thinking-small markdown variant, and custom □/☑ task markers remain out of scope; GFM task lists keep native checkboxes.
|
||||
|
||||
@@ -36,6 +36,8 @@ Fenced code and GFM tables own horizontal overflow so long content cannot widen
|
||||
|
||||
**Port deepsuite Prism `highlight.css` and the mdast pipeline.** Appearance parity is owned by CSS Modules and shared `--dsw-*` tokens; highlighting stays on the existing shiki allowlist so the client does not take a second highlighter or Prism class contract.
|
||||
|
||||
**Preprocess Markdown source or repair text nodes after parsing for CJK punctuation boundaries.** A source rewrite must reproduce escape, code, math, and delimiter rules before the parser owns those distinctions, while a text-node repair has already lost some source intent and cannot compose with parsed inline nodes. Extending attention at the tokenizer boundary preserves the upstream resolver and limits the divergence to delimiter eligibility.
|
||||
|
||||
## Consequences
|
||||
|
||||
Assistant replies render semantic Markdown consistently during streaming and replay, while tool cards, reasoning rows, interactions, user bubbles, and the host protocol remain unchanged. Streaming reparses the current text after each accumulated update; incomplete Markdown can temporarily change structure, but the isolated tail bounds React invalidation and the final event does not switch renderers. Code fences share one chrome and copy path with tool and details surfaces. The initial Web shell includes the Markdown parser, GFM runtime, KaTeX, and shiki allowlist; citation, anchor, and thinking-small surfaces remain deferred.
|
||||
@@ -12,7 +12,7 @@ Web 对话通过会话事件、历史回放与流式累积保留 assistant Markd
|
||||
|
||||
`@deepseek-ai/dsh-client-ui-primitives` 导出 `MarkdownText`，用作不受信任的 assistant 文本渲染器；`ui-conversation` 仅为 assistant `text` 块选择该渲染器。已完成的历史消息、流式输出尾部与被中断的部分输出已经共用 `AssistantMarkdown`，因此无需更改事件或快照，它们便会采用同一渲染器。用户消息与 steering 消息继续使用 `MessageText`，并保持按字面渲染。
|
||||
|
||||
`MarkdownText` 使用 `react-markdown` 与 `remark-gfm`，从 AST 构建 React 元素。它覆盖 CommonMark 块，以及 GFM 表格、任务列表、删除线与自动链接，且不解析原始 HTML。围栏代码经共享的 `CodeBlock` 路由；该组件用客户端的 shiki 单例（`--shiki-*` token）高亮已注册语法，否则回退为纯等宽文本。轮次流式输出期间，围栏停留在纯文本分支，以免每收到一个分片就对增长中的围栏重新分词。
|
||||
`MarkdownText` 使用 `react-markdown` 与 `remark-gfm`，从 AST 构建 React 元素。它覆盖 CommonMark 块，以及 GFM 表格、任务列表、删除线与自动链接，且不解析原始 HTML。一个 micromark attention 扩展复用 CommonMark resolver，同时允许至少两个星号组成的连续序列在 Unicode 标点后闭合，前提是其后紧邻 CJK 文本。这一例外涵盖流式输出期间与完成后无空格 CJK 文本中以标点结尾的粗体；单星号强调、紧邻非 CJK 文本的情况、已转义源文本、代码与数学公式仍沿用上游解析行为。围栏代码经共享的 `CodeBlock` 路由；该组件用客户端的 shiki 单例（`--shiki-*` token）高亮已注册语法，否则回退为纯等宽文本。轮次流式输出期间，围栏停留在纯文本分支，以免每收到一个分片就对增长中的围栏重新分词。
|
||||
|
||||
视觉间距、表格、链接、引用块、行内代码与代码块外框遵循 deepsuite `@deepseek/md`（`markdown.css` / `code-block.css`），并使用同一套 `--dsw-alias-markdown-*`、`--dsw-font-markdown-*`、`--dsw-alias-border-l*` 与 `--dsw-alias-label-*` token。链接使用 `--dsw-alias-state-business-primary`（deepsuite 的样式表使用 `--dsw-alias-brand-text`，仅在 newDesign 下为蓝色；design-platform 将 brand-text 保持为近黑色，此处不做重新调色）。`CodeBlock` 提供语言横幅与复制控件（`复制` / `复制成功`）。已完成的文本通过 `remark-math` 和 `rehype-katex` 渲染 KaTeX；`remarkMathCompatibility` 将 `\(...\)`、`\[...\]` 和块级同一行 `$$...$$` 映射为同一套标准数学 AST 节点。这是一层小范围的解析器兼容层，不是正则重写，也不修复格式错误的模型输出。流式输出在完成前保持按字面渲染，避免不完整公式闪现错误。引用胶囊、标题锚点、thinking-small markdown 变体，以及自定义 □/☑ 任务标记仍不在范围内；GFM 任务列表继续使用原生复选框。
|
||||
|
||||
@@ -36,6 +36,8 @@ assistant 生成的链接目标地址仅限绝对 HTTP、HTTPS 与 mailto URL。
|
||||
|
||||
**移植 deepsuite 的 Prism `highlight.css` 与 mdast 管线。**外观一致性由 CSS Modules 与共享的 `--dsw-*` token 负责；高亮仍走现有的 shiki 允许列表，使客户端不必引入第二套高亮器或 Prism class 契约。
|
||||
|
||||
**为处理 CJK 标点边界而预处理 Markdown 源文本，或在解析后修复文本节点。**源文本重写必须在解析器掌握这些区别之前复现转义、代码、数学公式与定界符规则；文本节点修复则已经丢失部分源文本意图，也无法与已解析的行内节点组合。在分词器边界扩展 attention 可保留上游 resolver，并将差异限制在定界符的适用条件上。
|
||||
|
||||
## 后果
|
||||
|
||||
assistant 回复在流式输出与回放期间都会一致地渲染为语义化 Markdown，而工具卡片、推理行、交互、用户气泡和宿主协议保持不变。每次累积更新后，流式输出都会重新解析当前文本；未完成的 Markdown 可能暂时改变结构，但独立的尾部会限定 React 失效范围，最终事件也不会切换渲染器。代码围栏与工具及详情表层共用同一外框与复制路径。初始 Web shell 包含 Markdown 解析器、GFM 运行时、KaTeX 与 shiki 允许列表；citation、anchor 和 thinking-small 表层仍暂缓。
|
||||
@@ -63,10 +63,12 @@ External packages that a workspace package resolves at runtime. `scripts/install
|
||||
| [`koffi`](https://github.com/Koromix/koffi) | MIT |
|
||||
| [`mdast-util-from-markdown`](https://github.com/syntax-tree/mdast-util-from-markdown) | MIT |
|
||||
| [`mdast-util-gfm`](https://github.com/syntax-tree/mdast-util-gfm) | MIT |
|
||||
| [`micromark-core-commonmark`](https://github.com/micromark/micromark/tree/main/packages/micromark-core-commonmark) | MIT |
|
||||
| [`micromark-extension-gfm`](https://github.com/micromark/micromark-extension-gfm) | MIT |
|
||||
| [`micromark-extension-math`](https://github.com/micromark/micromark-extension-math) | MIT |
|
||||
| [`micromark-factory-space`](https://github.com/micromark/micromark/tree/main/packages/micromark-factory-space) | MIT |
|
||||
| [`micromark-util-character`](https://github.com/micromark/micromark/tree/main/packages/micromark-util-character) | MIT |
|
||||
| [`micromark-util-classify-character`](https://github.com/micromark/micromark/tree/main/packages/micromark-util-classify-character) | MIT |
|
||||
| [`micromark-util-symbol`](https://github.com/micromark/micromark/tree/main/packages/micromark-util-symbol) | MIT |
|
||||
| [`micromark-util-types`](https://github.com/micromark/micromark/tree/main/packages/micromark-util-types) | MIT |
|
||||
| [`node-addon-require-builtin`](https://www.npmjs.com/package/node-addon-require-builtin) | MIT |
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import type { Browser, Page } from 'playwright'
|
||||
import { chromium } from 'playwright'
|
||||
import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest'
|
||||
import { createMessage, createUserMessage } from '@deepseek-ai/dsh-llm'
|
||||
import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type {} from '@deepseek-ai/dsh-session-title'
|
||||
import {
|
||||
assertFixtureInventory,
|
||||
captureStableAria,
|
||||
compareOrRefreshGolden,
|
||||
launchWebScaffold,
|
||||
seedSession,
|
||||
watchConsole,
|
||||
webSnapshotMode,
|
||||
type WebScaffold,
|
||||
} from './scaffold.ts'
|
||||
import { newEnglishPage, saveFailureShot } from './support.ts'
|
||||
|
||||
const SNAPSHOT_DIR = fileURLToPath(new URL('./snapshots/markdown-cjk-strong', import.meta.url))
|
||||
const UI_EXPECTED = fileURLToPath(new URL('./snapshots/markdown-cjk-strong/ui.expected.md', import.meta.url))
|
||||
const MODE = webSnapshotMode()
|
||||
const SEED_ID = 'markdown-cjk-strong-web-e2e'
|
||||
const DONE = 'CJK_STRONG_DONE'
|
||||
const CASES = [
|
||||
['**注意：**内容', '注意：', '注意：内容'],
|
||||
['**Notice:**内容', 'Notice:', 'Notice:内容'],
|
||||
['**事件中间件（waterfall）**实现', '事件中间件（waterfall）', '事件中间件（waterfall）实现'],
|
||||
['**事件中间件(waterfall)**实现', '事件中间件(waterfall)', '事件中间件(waterfall)实现'],
|
||||
['**句号。**后续', '句号。', '句号。后续'],
|
||||
['**Period.**后续', 'Period.', 'Period.后续'],
|
||||
['**提醒！**继续', '提醒！', '提醒！继续'],
|
||||
['**Warning!**继续', 'Warning!', 'Warning!继续'],
|
||||
] as const
|
||||
|
||||
/** Build one settled assistant reply covering CJK-adjacent strong punctuation boundaries. */
|
||||
function markdownFixture(): string {
|
||||
const session = Session.create(SessionId('markdown-cjk-strong-source'))
|
||||
session.append('turn/start', { turn: 1 })
|
||||
const user = session.append('user/message', createUserMessage({
|
||||
content: [{ type: 'text', text: 'Render adjacent CJK strong emphasis.' }],
|
||||
source: { kind: 'user' },
|
||||
}), { surfaceOp: 'append' })
|
||||
session.append('session/title', {
|
||||
title: 'CJK strong emphasis',
|
||||
messageSeqs: [user.seq],
|
||||
source: { kind: 'fallback' },
|
||||
})
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('assistant/message', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
message: createMessage({
|
||||
role: 'assistant',
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: [
|
||||
'## CJK strong emphasis',
|
||||
'',
|
||||
...CASES.flatMap(([markdown]) => [markdown, '']),
|
||||
DONE,
|
||||
].join('\n'),
|
||||
}],
|
||||
source: { kind: 'model', provider: 'fixture', model: 'fixture' },
|
||||
}),
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
return [
|
||||
JSON.stringify({
|
||||
type: 'session',
|
||||
version: SESSION_FORMAT_VERSION,
|
||||
id: '{{sessionId}}',
|
||||
createdAt: 0,
|
||||
cwd: '{{cwd}}',
|
||||
}),
|
||||
...session.events.map(event => JSON.stringify(event)),
|
||||
'',
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
describe('web e2e: CJK-adjacent Markdown strong emphasis', () => {
|
||||
let scaffold: WebScaffold
|
||||
let browser: Browser
|
||||
let page: Page
|
||||
let tripwire: ReturnType<typeof watchConsole>
|
||||
|
||||
beforeAll(async () => {
|
||||
scaffold = await launchWebScaffold({})
|
||||
await seedSession(scaffold, markdownFixture(), SEED_ID)
|
||||
browser = await chromium.launch()
|
||||
page = await newEnglishPage(browser)
|
||||
tripwire = watchConsole(page)
|
||||
await page.goto(scaffold.baseUrl, { waitUntil: 'load' })
|
||||
await page.waitForSelector('[class*="frame"]', { timeout: 30_000 })
|
||||
}, 120_000)
|
||||
|
||||
afterAll(async () => {
|
||||
await browser?.close()
|
||||
await scaffold?.close()
|
||||
})
|
||||
|
||||
it.skipIf(MODE === 'record')('renders punctuation-terminated strong spans before adjacent CJK text', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-markdown-cjk-strong'))
|
||||
const groupRow = page.locator('[role="treeitem"]').first()
|
||||
await groupRow.waitFor({ timeout: 15_000 })
|
||||
await groupRow.click()
|
||||
const sessionRow = page.locator('[role="treeitem"]').nth(1)
|
||||
await sessionRow.waitFor({ timeout: 10_000 })
|
||||
await sessionRow.click()
|
||||
await expect.poll(() => page.getByText(DONE, { exact: true }).count(), { timeout: 15_000 }).toBe(1)
|
||||
|
||||
const strong = page.locator('[class*="markdown"] strong')
|
||||
await expect.poll(() => strong.count(), { timeout: 10_000 }).toBe(CASES.length)
|
||||
expect(await strong.allTextContents()).toEqual(CASES.map(([, expected]) => expected))
|
||||
for (const [, , paragraph] of CASES) {
|
||||
expect(await page.getByText(paragraph, { exact: true }).count()).toBe(1)
|
||||
}
|
||||
|
||||
const snapshot = (await captureStableAria(page, '[class*="centerCol"]', scaffold.workspaceCwd))
|
||||
.split(SEED_ID).join('{{seededId}}')
|
||||
await compareOrRefreshGolden(UI_EXPECTED, snapshot, MODE)
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
expect(tripwire.warnings).toEqual([])
|
||||
await assertFixtureInventory(SNAPSHOT_DIR, ['ui.expected.md'])
|
||||
}, 60_000)
|
||||
})
|
||||
@@ -0,0 +1,52 @@
|
||||
- banner:
|
||||
- navigation "Session hierarchy":
|
||||
- button "CJK strong emphasis" [disabled]
|
||||
- tablist:
|
||||
- tab "Chat" [selected]
|
||||
- tab "Trajectory"
|
||||
- text: Render adjacent CJK strong emphasis. {{clock}}
|
||||
- button "Copy":
|
||||
- img
|
||||
- button "Branch into a new conversation" [disabled]:
|
||||
- img
|
||||
- text: Available only on the last message of a completed turn
|
||||
- heading "CJK strong emphasis" [level=2]
|
||||
- paragraph:
|
||||
- strong: 注意：
|
||||
- text: 内容
|
||||
- paragraph:
|
||||
- strong: "Notice:"
|
||||
- text: 内容
|
||||
- paragraph:
|
||||
- strong: 事件中间件（waterfall）
|
||||
- text: 实现
|
||||
- paragraph:
|
||||
- strong: 事件中间件(waterfall)
|
||||
- text: 实现
|
||||
- paragraph:
|
||||
- strong: 句号。
|
||||
- text: 后续
|
||||
- paragraph:
|
||||
- strong: Period.
|
||||
- text: 后续
|
||||
- paragraph:
|
||||
- strong: 提醒！
|
||||
- text: 继续
|
||||
- paragraph:
|
||||
- strong: Warning!
|
||||
- text: 继续
|
||||
- paragraph: CJK_STRONG_DONE
|
||||
- button "Copy":
|
||||
- img
|
||||
- button "Branch into a new conversation":
|
||||
- img
|
||||
- text: {{clock}}Ran for {{duration}}
|
||||
- textbox "Message the agent"
|
||||
- button "Commands":
|
||||
- img
|
||||
- 'button "Access mode, current: Workspace Write"': Workspace Write
|
||||
- button "Select model":
|
||||
- text: Select model
|
||||
- img
|
||||
- button "Send message" [disabled]
|
||||
- text: 1 turns · 1 steps Input 0 tok · Output 0 tok
|
||||
@@ -51,6 +51,7 @@
|
||||
"tests/message-actions.e2e.ts",
|
||||
"tests/markdown-images.e2e.ts",
|
||||
"tests/math-rendering.e2e.ts",
|
||||
"tests/markdown-cjk-strong.e2e.ts",
|
||||
"tests/queue-actions.e2e.ts",
|
||||
"tests/skill-invocation-policy.e2e.ts",
|
||||
"tests/permission-policy-context.e2e.ts",
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/client/ui-primitives/README.md
|
||||
README.md: 03e7e3649fd0913fb48579aa87634153f67f5baf
|
||||
README.zh.md: 090ecc34e8d514e38853de8ed52e82d3bf019b43
|
||||
README.md: aebcc06bf95b5bd843c2e8add6f89908d7bfa1fa
|
||||
README.zh.md: 7e141148cee23416e344f57089d425c272e86db4
|
||||
@@ -10,7 +10,7 @@ Pure React atoms (zero cordis): StateDot, ic_ds_* icons, Button/Pill/Menu/Modal/
|
||||
|
||||
## Markdown rendering
|
||||
|
||||
`MarkdownText` renders GFM and `$…$`, `$$…$$`, `\(…\)`, and `\[…\]` TeX math from untrusted assistant output through React elements, with math typeset by KaTeX and trusted commands disabled; block-level same-line `$$…$$` is display math, including `\tag{}`. It omits raw HTML, neutralizes relative and non-HTTP(S)/mailto links, opens HTTP(S) links with safe external-link attributes, and renders absolute HTTP(S) images without a referrer; relative paths, absolute local paths, `file:` URLs, and unsupported schemes retain their alt text. `MessageText` remains the literal-text primitive for user-authored content. `extractMarkdownPlainText` removes Markdown presentation markup for compact labels while preserving raw HTML as literal text. Element spacing, responsive images, tables, links, and inline code use the same `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` tokens as deepsuite `@deepseek/md`. Fenced blocks render through `CodeBlock` (language banner, copy control, shiki for the registered grammars).
|
||||
`MarkdownText` renders GFM and `$…$`, `$$…$$`, `\(…\)`, and `\[…\]` TeX math from untrusted assistant output through React elements, with math typeset by KaTeX and trusted commands disabled; block-level same-line `$$…$$` is display math, including `\tag{}`. A narrow micromark extension lets asterisk strong emphasis ending in punctuation close before adjacent CJK text, where prose normally omits the whitespace CommonMark requires; single-asterisk emphasis, non-CJK adjacency, escapes, code, and math retain upstream parsing. It omits raw HTML, neutralizes relative and non-HTTP(S)/mailto links, opens HTTP(S) links with safe external-link attributes, and renders absolute HTTP(S) images without a referrer; relative paths, absolute local paths, `file:` URLs, and unsupported schemes retain their alt text. `MessageText` remains the literal-text primitive for user-authored content. `extractMarkdownPlainText` removes Markdown presentation markup for compact labels while preserving raw HTML as literal text. Element spacing, responsive images, tables, links, and inline code use the same `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` tokens as deepsuite `@deepseek/md`. Fenced blocks render through `CodeBlock` (language banner, copy control, shiki for the registered grammars).
|
||||
|
||||
## Terminal output
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
## Markdown 渲染
|
||||
|
||||
`MarkdownText` 通过 React 元素渲染来自不受信任 assistant 输出的 GFM 与 `$…$`、`$$…$$`、`\(…\)` 和 `\[…\]` TeX 公式，公式由 KaTeX 排版并禁用受信任命令；块级同一行 `$$…$$` 是显示公式并支持 `\tag{}`。它会省略原始 HTML，使相对链接及非 HTTP(S)/mailto 链接失效，以安全的外部链接属性打开 HTTP(S) 链接，并在不发送 referrer 的情况下渲染采用绝对 HTTP(S) URL 的图片；相对路径、绝对本地路径、`file:` URL 与不受支持的 scheme 会保留其 alt 文本。`MessageText` 仍是用户创作内容使用的字面文本原语。`extractMarkdownPlainText` 会移除 Markdown 呈现标记以用于紧凑标签，同时将原始 HTML 保留为字面文本。元素间距、响应式图片、表格、链接与行内代码使用与 deepsuite `@deepseek/md` 相同的 `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` token。围栏代码块通过 `CodeBlock` 渲染（语言横幅、复制控件，以及对已注册语法使用 shiki）。
|
||||
`MarkdownText` 通过 React 元素渲染来自不受信任 assistant 输出的 GFM 与 `$…$`、`$$…$$`、`\(…\)` 和 `\[…\]` TeX 公式，公式由 KaTeX 排版并禁用受信任命令；块级同一行 `$$…$$` 是显示公式并支持 `\tag{}`。一个小范围的 micromark 扩展允许由星号标记、以标点结尾的粗体在紧邻的 CJK 文本前闭合，以适应 CJK 文本通常省略 CommonMark 所要求空格的写法；单星号强调、紧邻非 CJK 文本的情况、转义、代码与数学公式仍沿用上游解析行为。它会省略原始 HTML，使相对链接及非 HTTP(S)/mailto 链接失效，以安全的外部链接属性打开 HTTP(S) 链接，并在不发送 referrer 的情况下渲染采用绝对 HTTP(S) URL 的图片；相对路径、绝对本地路径、`file:` URL 与不受支持的 scheme 会保留其 alt 文本。`MessageText` 仍是用户创作内容使用的字面文本原语。`extractMarkdownPlainText` 会移除 Markdown 呈现标记以用于紧凑标签，同时将原始 HTML 保留为字面文本。元素间距、响应式图片、表格、链接与行内代码使用与 deepsuite `@deepseek/md` 相同的 `--dsw-alias-markdown-*` / `--dsw-font-markdown-*` token。围栏代码块通过 `CodeBlock` 渲染（语言横幅、复制控件，以及对已注册语法使用 shiki）。
|
||||
|
||||
## 终端输出
|
||||
|
||||
|
||||
@@ -26,10 +26,12 @@
|
||||
"katex": "^0.16.47",
|
||||
"mdast-util-from-markdown": "^2.0.3",
|
||||
"mdast-util-gfm": "^3.1.0",
|
||||
"micromark-core-commonmark": "^2.0.3",
|
||||
"micromark-extension-gfm": "^3.0.0",
|
||||
"micromark-extension-math": "^3.1.0",
|
||||
"micromark-factory-space": "^2.0.1",
|
||||
"micromark-util-character": "^2.1.1",
|
||||
"micromark-util-classify-character": "^2.0.1",
|
||||
"micromark-util-symbol": "^2.0.1",
|
||||
"micromark-util-types": "^2.0.2",
|
||||
"react": "^18.2.0",
|
||||
|
||||
@@ -5,13 +5,15 @@ import rehypeKatex from 'rehype-katex'
|
||||
import remarkGfm from 'remark-gfm'
|
||||
import remarkMath from 'remark-math'
|
||||
import { CodeBlock } from './CodeBlock.tsx'
|
||||
import { remarkCjkFriendlyStrong } from './remarkCjkFriendlyStrong.ts'
|
||||
import { remarkMathCompatibility } from './remarkMathCompatibility.ts'
|
||||
import 'katex/dist/katex.min.css'
|
||||
import css from './MarkdownText.module.css'
|
||||
|
||||
const streamingRemarkPlugins = [remarkGfm]
|
||||
const streamingRemarkPlugins = [remarkGfm, remarkCjkFriendlyStrong]
|
||||
const settledRemarkPlugins = [
|
||||
remarkGfm,
|
||||
remarkCjkFriendlyStrong,
|
||||
remarkMathCompatibility,
|
||||
remarkMath,
|
||||
]
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
/** Let asterisk strong emphasis close after punctuation when CJK prose continues without whitespace. */
|
||||
|
||||
import { attention } from 'micromark-core-commonmark'
|
||||
import { unicodePunctuation } from 'micromark-util-character'
|
||||
import { classifyCharacter } from 'micromark-util-classify-character'
|
||||
import { codes, constants } from 'micromark-util-symbol'
|
||||
import type { Construct, Extension, State, Tokenizer } from 'micromark-util-types'
|
||||
|
||||
interface RemarkProcessor {
|
||||
data(): { micromarkExtensions?: Extension[] }
|
||||
}
|
||||
|
||||
const cjkCharacter = new RegExp([
|
||||
'\\p{Script_Extensions=Han}',
|
||||
'\\p{Script_Extensions=Hiragana}',
|
||||
'\\p{Script_Extensions=Katakana}',
|
||||
'\\p{Script_Extensions=Hangul}',
|
||||
'\\p{Script_Extensions=Bopomofo}',
|
||||
].join('|'), 'u')
|
||||
|
||||
function isCjkCharacter(code: number | null): boolean {
|
||||
return code !== null && code >= 0 && cjkCharacter.test(String.fromCodePoint(code))
|
||||
}
|
||||
|
||||
const tokenizeCjkFriendlyAttention: Tokenizer = function (effects, ok, nok) {
|
||||
const configuredAttentionMarkers = this.parser.constructs.attentionMarkers.null
|
||||
if (configuredAttentionMarkers === undefined) {
|
||||
throw new Error('micromark CommonMark attention markers are unavailable')
|
||||
}
|
||||
const attentionMarkers = configuredAttentionMarkers
|
||||
const previous = this.previous
|
||||
const before = classifyCharacter(previous)
|
||||
let marker: number | null = codes.eof
|
||||
|
||||
return start
|
||||
|
||||
function start(code: number | null): State | undefined {
|
||||
/* v8 ignore next -- this text construct is dispatched only for an asterisk. */
|
||||
if (code !== codes.asterisk) return nok(code)
|
||||
marker = code
|
||||
effects.enter('attentionSequence')
|
||||
return inside(code)
|
||||
}
|
||||
|
||||
function inside(code: number | null): State | undefined {
|
||||
if (code === marker) {
|
||||
effects.consume(code)
|
||||
return inside
|
||||
}
|
||||
|
||||
const token = effects.exit('attentionSequence')
|
||||
const after = classifyCharacter(code)
|
||||
const open = !after || (after === constants.characterGroupPunctuation && Boolean(before))
|
||||
|| attentionMarkers.includes(code)
|
||||
const commonMarkClose = !before
|
||||
|| (before === constants.characterGroupPunctuation && Boolean(after))
|
||||
|| attentionMarkers.includes(previous)
|
||||
const markerCount = token.end.offset - token.start.offset
|
||||
const cjkStrongClose = markerCount >= 2
|
||||
&& unicodePunctuation(previous)
|
||||
&& isCjkCharacter(code)
|
||||
const close = commonMarkClose || cjkStrongClose
|
||||
|
||||
token._open = open
|
||||
token._close = close
|
||||
return ok(code)
|
||||
}
|
||||
}
|
||||
|
||||
const cjkFriendlyAttention: Construct = {
|
||||
name: 'cjkFriendlyAttention',
|
||||
resolveAll: attention.resolveAll,
|
||||
tokenize: tokenizeCjkFriendlyAttention,
|
||||
}
|
||||
|
||||
const cjkFriendlyStrong: Extension = {
|
||||
text: { [codes.asterisk]: cjkFriendlyAttention },
|
||||
}
|
||||
|
||||
/**
|
||||
* Extend CommonMark asterisk strong emphasis for punctuation-delimited CJK prose.
|
||||
* @returns Nothing.
|
||||
*/
|
||||
export function remarkCjkFriendlyStrong(this: RemarkProcessor): undefined {
|
||||
const data = this.data()
|
||||
const extensions = data.micromarkExtensions ?? (data.micromarkExtensions = [])
|
||||
extensions.push(cjkFriendlyStrong)
|
||||
}
|
||||
@@ -3,6 +3,7 @@ import { cleanup, fireEvent, render, screen } from '@testing-library/react'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import type { Extension } from 'micromark-util-types'
|
||||
import { JsonBlock, MarkdownText, MessageText } from '@deepseek-ai/dsh-client-ui-primitives'
|
||||
import { remarkCjkFriendlyStrong } from '../src/markdown/remarkCjkFriendlyStrong.ts'
|
||||
import { remarkMathCompatibility } from '../src/markdown/remarkMathCompatibility.ts'
|
||||
|
||||
afterEach(cleanup)
|
||||
@@ -68,6 +69,70 @@ describe('MarkdownText', () => {
|
||||
expect(screen.getByRole('link', { name: 'https://deepseek.com' })).toBeTruthy()
|
||||
})
|
||||
|
||||
it('closes punctuation-terminated strong emphasis before adjacent CJK text', () => {
|
||||
const cases = [
|
||||
['**注意：**内容', '注意：'],
|
||||
['**Notice:**内容', 'Notice:'],
|
||||
['**事件中间件（waterfall）**实现', '事件中间件（waterfall）'],
|
||||
['**事件中间件(waterfall)**实现', '事件中间件(waterfall)'],
|
||||
['**句号。**后续', '句号。'],
|
||||
['**Period.**后续', 'Period.'],
|
||||
['**提醒！**继续', '提醒！'],
|
||||
['**Warning!**继续', 'Warning!'],
|
||||
] as const
|
||||
const source = cases.map(([markdown]) => markdown).join('\n\n')
|
||||
|
||||
for (const streaming of [false, true]) {
|
||||
const rendered = render(<MarkdownText text={source} streaming={streaming} />)
|
||||
expect([...rendered.container.querySelectorAll('strong')].map(node => node.textContent))
|
||||
.toEqual(cases.map(([, strong]) => strong))
|
||||
rendered.unmount()
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps the CJK strong extension out of escaped, code, math, and ASCII contexts', () => {
|
||||
const source = [
|
||||
String.raw`\**注意：**内容`,
|
||||
'`**注意：**内容`',
|
||||
'**Notice:**text',
|
||||
'*提醒！*继续',
|
||||
'$**注意：**内容$',
|
||||
'```md',
|
||||
'**注意：**内容',
|
||||
'```',
|
||||
'**普通**内容',
|
||||
'*普通*内容',
|
||||
].join('\n\n')
|
||||
const { container } = render(<MarkdownText text={source} />)
|
||||
|
||||
expect([...container.querySelectorAll('strong')].map(node => node.textContent)).toEqual(['普通'])
|
||||
expect([...container.querySelectorAll('em')].map(node => node.textContent)).toEqual(['普通'])
|
||||
expect(container.querySelector('code')?.textContent).toBe('**注意：**内容')
|
||||
expect(container.querySelector('.katex annotation')?.textContent).toBe('**注意：**内容')
|
||||
expect(container.querySelector('pre code')?.textContent).toContain('**注意：**内容')
|
||||
expect(container.textContent).toContain('**Notice:**text')
|
||||
expect(container.textContent).toContain('*提醒！*继续')
|
||||
expect(container.textContent).toContain('**注意：**内容')
|
||||
})
|
||||
|
||||
it('registers the CJK strong extension and rejects a parser without CommonMark attention markers', () => {
|
||||
const data: { micromarkExtensions?: Extension[] } = {}
|
||||
const processor = { data: () => data }
|
||||
remarkCjkFriendlyStrong.call(processor)
|
||||
remarkCjkFriendlyStrong.call(processor)
|
||||
|
||||
expect(data.micromarkExtensions).toHaveLength(2)
|
||||
const construct = data.micromarkExtensions?.[0]?.text?.[42]
|
||||
const tokenizer = Array.isArray(construct) ? construct[0]?.tokenize : construct?.tokenize
|
||||
expect(tokenizer).toBeTypeOf('function')
|
||||
expect(() => tokenizer?.call({
|
||||
parser: { constructs: { attentionMarkers: {} } },
|
||||
previous: null,
|
||||
} as never, {} as never, () => undefined, () => undefined)).toThrow(
|
||||
'micromark CommonMark attention markers are unavailable',
|
||||
)
|
||||
})
|
||||
|
||||
it('a fence labeled with an inherited object key renders plain, never crashing shiki', () => {
|
||||
for (const label of ['constructor', '__proto__', 'toString', 'hasOwnProperty']) {
|
||||
const { container, unmount } = render(<MarkdownText text={'```' + label + '\ncode body\n```'} />)
|
||||
|
||||
Generated
+6
@@ -1703,6 +1703,9 @@ importers:
|
||||
mdast-util-gfm:
|
||||
specifier: ^3.1.0
|
||||
version: 3.1.0
|
||||
micromark-core-commonmark:
|
||||
specifier: ^2.0.3
|
||||
version: 2.0.3
|
||||
micromark-extension-gfm:
|
||||
specifier: ^3.0.0
|
||||
version: 3.0.0
|
||||
@@ -1715,6 +1718,9 @@ importers:
|
||||
micromark-util-character:
|
||||
specifier: ^2.1.1
|
||||
version: 2.1.1
|
||||
micromark-util-classify-character:
|
||||
specifier: ^2.0.1
|
||||
version: 2.0.1
|
||||
micromark-util-symbol:
|
||||
specifier: ^2.0.1
|
||||
version: 2.0.1
|
||||
|
||||
@@ -38,6 +38,7 @@
|
||||
"apps/web/tests/message-actions.e2e.ts",
|
||||
"apps/web/tests/markdown-images.e2e.ts",
|
||||
"apps/web/tests/math-rendering.e2e.ts",
|
||||
"apps/web/tests/markdown-cjk-strong.e2e.ts",
|
||||
"apps/web/tests/queue-actions.e2e.ts",
|
||||
"apps/web/tests/skill-invocation-policy.e2e.ts",
|
||||
"apps/web/tests/permission-policy-context.e2e.ts",
|
||||
|
||||
Reference in New Issue
Block a user