diff --git a/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.i18n.yaml new file mode 100644 index 0000000000..08398b5440 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.md +2026-07-29-atomic-web-image-admission.md: 7b2f7aeb43ca1393cdab3abe35916964bc47f0c9 +2026-07-29-atomic-web-image-admission.zh.md: 5a04d2e599744dfe9f92eb64a6c828161c46bf6e diff --git a/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.md b/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.md new file mode 100644 index 0000000000..7b2f7aeb43 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.md @@ -0,0 +1,29 @@ +# Agent Note: Atomic Web image admission + +Status: implemented + +English | [中文](2026-07-29-atomic-web-image-admission.zh.md) + +## Problem + +Image prompt admission and `session.selectModel` each read session modality state across asynchronous model and attachment lookups. Without one ordering boundary, an image prompt could validate an image-capable target while a concurrent selection installed a text-only target, or selection could miss a prompt after inbox dequeue but before its durable message event. Scanning the immutable event log avoided the second race but permanently blocked a text-only selection even after compaction removed the image from current model history. + +## Decision + +Each live Web agent has one private promise chain shared by image-bearing prompt admission and model selection. A failed operation settles its caller normally and leaves the chain usable. Text-only prompts bypass the chain because they cannot change the modality constraint. + +The pending-inbox mirror marks a prompt as claimed at dequeue and retains it until the matching `user/message` or `steering/message` event publishes. If admission ends without publishing, the next dequeue or the transition to idle retires the claimed entry; inbox discard retires the listed work, and session disposal retires every remaining entry. Model selection checks that mirror plus `Session.deriveMessages()`, which is the current model-visible history after compaction. + +Provider adapters remain the final enforcement boundary. The host ordering only prevents its mutable route and pending image state from contradicting each other before request assembly. + +## Alternatives considered + +**Scan every immutable session event.** This catches published images but treats compacted-away content as permanently model-visible, preventing a valid later switch to a text-only route. + +**Retire the pending mirror at inbox dequeue.** Dequeue precedes the durable message append and leaves the exact interval in which model selection can miss both pending and published state. + +**Serialize every prompt and session mutation.** Text-only prompts and unrelated session operations cannot introduce an image requirement. A broader lock would add latency and ownership without closing another modality race. + +## Consequences + +An image prompt and a concurrent model selection have deterministic order, and a text-only target cannot strand an image that has been admitted but not yet published. Selection may wait for an in-flight image admission, while unrelated prompts retain their existing concurrency. Compaction can make a text-only target valid once no pending or derived image remains. diff --git a/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.zh.md b/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.zh.md new file mode 100644 index 0000000000..5a04d2e599 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.zh.md @@ -0,0 +1,29 @@ +# Agent Note: Web 图片准入的原子性 + +Status: implemented + +[English](2026-07-29-atomic-web-image-admission.md) | 中文 + +## 问题 + +包含图片的提示词准入与 `session.selectModel` 都会在跨越异步模型查询与附件查询的过程中读取会话模态状态。如果没有统一的顺序边界，包含图片的提示词可能在支持图片的目标上通过校验，并发的选择操作却设置了纯文本目标；选择操作也可能在提示词已从 inbox 出队、但其持久消息事件尚未发布时漏掉该提示词。扫描不可变事件日志可以避免第二种竞态，但即使压缩（compaction）已经从当前模型历史中移除图片，仍会永久阻止选择纯文本目标。 + +## 决策 + +每个活跃 Web agent（智能体）都有一条私有 promise 链，由包含图片的提示词准入与模型选择共享。操作失败会照常传递给调用方，且不会使该链失效。纯文本提示词绕过该链，因为它们不会改变模态约束。 + +待处理 inbox 镜像会在提示词出队时将其标记为已认领，并保留到匹配的 `user/message` 或 `steering/message` 事件发布。若准入结束时未发布事件，下一次出队或转为空闲状态会移除已认领的条目；inbox 丢弃会移除列出的工作项，会话 dispose（资源释放）则会移除所有剩余条目。模型选择会检查该镜像以及 `Session.deriveMessages()`；后者表示压缩后模型当前可见的历史。 + +提供方适配器仍是最终的强制检查边界。宿主的顺序控制仅用于避免其可变路由与待发布图片状态在请求组装前彼此矛盾。 + +## 曾考虑的替代方案 + +**扫描每个不可变会话事件。** 这能捕获已发布的图片，但会把经压缩移除的内容视为永久对模型可见，从而阻止之后合法切换到纯文本路由。 + +**在 inbox 出队时退役待处理镜像。** 出队早于持久消息追加，因此恰好会留下一个时间区间，让模型选择既看不到待处理状态，也看不到已发布状态。 + +**序列化每个提示词和会话变更。** 纯文本提示词和无关的会话操作无法引入图片要求。更宽的锁会增加延迟与所有权复杂度，却不会再消除任何模态竞态。 + +## 后果 + +包含图片的提示词准入与并发模型选择之间具有确定的先后顺序，纯文本目标无法使已获准入但尚未发布的图片搁浅。模型选择可能等待正在进行的图片准入完成，而无关提示词仍按现有方式并发处理。当没有图片等待发布，且派生历史经过压缩后也不再含图片时，纯文本目标可以变得有效。 diff --git a/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.i18n.yaml b/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.i18n.yaml index 4d1bec3b32..04783dff62 100644 --- a/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md -2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 7965f661dd232c035d986eead08bad0e61fecaf7 -2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 07586464874c18cc7121ae6cdee07dec379703b0 +2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 50ba92ed503126fc26859d7646774a5b25bcc4eb +2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 93fff2a550fdcfe013110eab28ddba0a38ec7490 diff --git a/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md b/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md index 7965f661dd..50ba92ed50 100644 --- a/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md +++ b/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md @@ -44,7 +44,9 @@ The persistence boundary is message acceptance, not paste: Each session's `InputMachine` state keeps the ordered runtime-only attachment identifiers alongside the live draft. The framework-owned chat store receives only the draft's plain-text persistence mirror, while `ConversationService` owns the corresponding browser-only `File` and object-URL registry: ```ts -export {} +import type { Branded } from '@deepseek-ai/dsh-brand' + +type DraftAttachmentId = Branded<'DraftAttachmentId'> interface ChatStoreState { selection: object | null @@ -54,12 +56,12 @@ interface ChatStoreState { interface InputState { draft: string - imageIds: readonly string[] + imageIds: readonly DraftAttachmentId[] } interface ComposerAttachment { kind: 'image' - id: string + id: DraftAttachmentId file: File previewUrl: string } @@ -67,7 +69,7 @@ interface ComposerAttachment { This split uses the session provide channel's input hook and actions as the single subscription path for live composer state while keeping non-serializable browser objects out of persisted JSON. Only the plain-text draft mirror uses `localStorage`; attachment identifiers, browser `File` objects, and object URLs remain scoped to the live session input shell. Unsent images therefore do not survive reload or session-scope disposal. A native client may stage input in an OS temporary directory, but it must treat that path exactly like the browser object URL: delete it when no longer needed and copy the bytes into the durable store before message acceptance. -The local attachment backend resolves an explicit `dshHome`, then `$DSH_HOME`, then `~/.dsh`. It stores content-addressed objects below `$DSH_HOME/attachments/v1/objects//` with owner-only directory and file permissions. A temporary file is written, synchronized, atomically published, and made durable with a directory sync on the publication directories (POSIX; Windows relies on filesystem metadata journaling) before the service returns a reference. The content digest is encoded in the opaque `sha256:` identifier, and every read verifies the digest, media type, byte length, width, and height. +The local attachment backend resolves an explicit `dshHome`, then `$DSH_HOME`, then `~/.dsh`. It stores content-addressed objects below `$DSH_HOME/attachments/v1/objects//` with owner-only directory and file permissions. A temporary file is written, synchronized, atomically published, and made durable with a directory sync on the publication directories (POSIX; Windows relies on filesystem metadata journaling) before the service returns a reference. The content digest is encoded in the opaque `sha256:` identifier. Admission and reads fully decode supported rasters before accepting their format and dimensions, and every read also verifies the digest, byte length, and logged metadata. The store performs no automatic deletion in version one. Sent user images and model-generated images remain reachable for history, resume, and fork. Reference-aware garbage collection needs a separate design because an age-only rule can delete data still referenced by a durable session. Deployment byte and pixel limits are admission policy on writes; reads verify the digest and recorded metadata without reapplying current admission limits, so lowering policy does not invalidate older history. @@ -112,17 +114,17 @@ type PromptInputPart = } ``` -Base64 crosses JSON-RPC once and is discarded after persistence. The host validates canonical base64, image count, aggregate bytes, individual bytes, magic-byte MIME, intrinsic dimensions, and decoded-pixel count: it validates the complete batch through the seam's storage-free `validateImage` before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Only after every image succeeds does it call the agent with normalized text and durable image blocks in the submitted order. A failure appends no user event and exposes no attachment path or raw bytes. +Base64 crosses JSON-RPC once and is discarded after persistence. The host validates canonical base64, image count, aggregate bytes, individual bytes, the declared MIME against a fully decoded raster, intrinsic dimensions, and decoded-pixel count. It awaits the seam's storage-free `validateImage` for every batch member before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Storage commits then run in submission order to bound full-raster decoder memory. If a later storage I/O operation fails, the host appends no user event, but an earlier immutable content-addressed object may remain unreferenced; version one leaves cleanup to future reference-aware garbage collection instead of adding destructive rollback to the deduplicated store. Only after every image succeeds does it call the agent with normalized text and durable image blocks in the submitted order. A failure exposes no attachment path or raw bytes. -`session.attachment` is a read-only, session-scoped endpoint. The host serves bytes only when a durable event in that session references the requested attachment identifier. The client deduplicates loads by session and attachment identifier while that session is rendered, revokes resolved URLs on rendered-session disposal, and invalidates late loads so an unmounted session cannot repopulate the cache. +`session.attachment` is a read-only, session-scoped endpoint. The host serves bytes only when a durable event in that session references the requested attachment identifier. The client deduplicates loads by session and attachment identifier while that session is rendered, revokes resolved URLs on rendered-session disposal, and rejects invalidated late loads before allocating an object URL so an unmounted session or disposed service cannot repopulate the cache. ### Model capabilities and provider behavior Model catalog entries gain optional merge-extensible input modality declarations. A missing declaration means unknown; a present list without `image` is an explicit negative capability. -The host is the authoritative preflight boundary. It resolves the session's latest routed provider/model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects the prompt before writing any attachment or event, and the client restores the draft. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. The browser rejects unsupported declared image media types before allocating preview URLs, but it does not snapshot deployment limits or model capability: a handshake snapshot cannot represent a session's current target after `session.selectModel`, and deployment policy may change independently. The host validates the complete batch against current byte, count, aggregate, media, dimension, pixel, and routed-model policy before writing any attachment or event; its rejection renders through the composer error strip. +The host is the authoritative preflight boundary. It resolves the session's latest routed provider/model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects the prompt before writing any attachment or event, and the client restores the draft. Image-bearing prompt admission and model selection share one per-agent serial boundary, and a dequeued prompt remains pending until its durable message event publishes ([ordering decision](../bug-fix/2026-07-29-atomic-web-image-admission.md)). Selection rejects a text-only target while an image is pending publication or remains in the session's current derived history; compaction can remove old images and make a later text-only selection valid. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. The browser rejects unsupported declared image media types before allocating preview URLs, but it does not snapshot deployment limits or model capability: a handshake snapshot cannot represent a session's current target after `session.selectModel`, and deployment policy may change independently. The host validates the complete batch against current byte, count, aggregate, media, dimension, pixel, and routed-model policy before writing any attachment or event; its rejection renders through the composer error strip. -The Pi-AI adapter is the first visual-input route: it resolves `ctx.attachments` at request time, then resolves each durable reference and emits native image content only for models that declare image input. The shipped composition registers Pi-AI OpenAI and Anthropic routes alongside the text-only default DeepSeek route; selecting the active provider/model remains a host composition or profile concern rather than an image-input CLI feature. Request-time service resolution keeps Cordis load order from freezing optional attachment availability. The hand-written DeepSeek adapter throws typed `UNSUPPORTED_CONTENT` for an image anywhere in the request, including nested tool results. No adapter may flatten or skip an image. +The Pi-AI adapter is the first visual-input route: it resolves `ctx.attachments` at request time, recursively converts each durable image reference including references nested inside tool results, and emits native image content only for models that declare image input. The shipped composition registers Pi-AI OpenAI and Anthropic routes alongside the text-only default DeepSeek route; selecting the active provider/model remains a host composition or profile concern rather than an image-input CLI feature. Request-time service resolution keeps Cordis load order from freezing optional attachment availability. The hand-written DeepSeek adapter throws typed `UNSUPPORTED_CONTENT` for an image anywhere in the request, including nested tool results. No adapter may flatten or skip an image. Core supports structured assistant image blocks, but no current production provider route is certified for image output. Any future output-capable adapter must retrieve provider bytes under bounded size and time policy, validate them through the same attachment service, persist them, and only then publish the atomic `ImageBlock`. A URL in assistant Markdown remains text and is never downloaded automatically. @@ -140,14 +142,14 @@ Composer thumbnails and each `MessageImage` own ephemeral original-preview state Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 5 MiB per image, 10 images and 20 MiB aggregate image bytes per message, and 40 million intrinsic pixels per image. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier independently caps buffered API request bodies, deriving the cap from the host attachment service's aggregate image limit plus base64 and envelope expansion; a body without a declared length is rejected the moment it crosses the cap rather than drained to its end. -Malformed base64, unsupported or mismatched media, truncated headers, excess bytes, excess image count, excess pixels, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser. +Malformed base64, unsupported or mismatched media, truncated image payloads, excess bytes, excess image count, excess pixels, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser. ### Package and surface changes | Surface | Responsibility | | --- | --- | | `packages/attachment/attachment` | Opaque attachment identifier, image reference, limits, failures, and `ctx.attachments` service. | -| `packages/attachment/attachment-local` | Private content-addressed storage, image-header validation, integrity verification, and configuration. | +| `packages/attachment/attachment-local` | Private content-addressed storage, complete raster decoding, integrity verification, and configuration. | | `packages/llm/llm` | Role-neutral `ImageBlock` and input-modality metadata. | | `packages/llm/llm-pi-ai` | Resolve durable supported image input into native provider content. | | `packages/llm/llm-deepseek` | Reject image content explicitly. | @@ -194,9 +196,9 @@ UI state can be stale and does not protect direct SDK, ACP, replay, or uncatalog ## Testing - Storage tests cover content-addressed deduplication, private permissions, admission failures, corruption/missing-object failures, and reading history after deployment limits are lowered. -- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, and refusal of a text-only `session.selectModel` once the session log carries an image (an accepted switch would strand every later turn). -- Client unit tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, ordering, and draft/session-scope/application object-URL cleanup; the keyless assembled built-client lane (`apps/web/tests/image-display.snapshot.ts`, `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`) covers the historical user and assistant galleries over the authorized attachment route, the original-size lightbox, and the composer paste rail. -- Adapter and compaction tests cover native Pi-AI image conversion, late attachment-service composition, text-only rejection, nested tool-result images, preserved summary input, and explicit image-output rejection. +- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, image-admission/model-selection races, pending publication, and selection against current derived history after compaction. +- Client unit tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, ordering, draft/session-scope/application object-URL cleanup, and a deferred historical read that completes after disposal; the keyless assembled built-client lane (`apps/web/tests/image-display.snapshot.ts`, `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`) covers the historical user and assistant galleries over the authorized attachment route, the original-size lightbox, and the composer paste rail. +- Adapter and compaction tests cover native Pi-AI image conversion, late attachment-service composition, text-only rejection, recursively nested tool-result images, preserved summary input, and explicit image-output rejection. - A credentialed real-API test sends a PNG through the Anthropic `claude-opus-4-8` route and requires the model to identify its QR code. - The current production adapter set has no certified image-output route; output-provider certification remains outside version one. diff --git a/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md b/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md index 0758646487..93fff2a550 100644 --- a/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md +++ b/.agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md @@ -44,7 +44,9 @@ Status: implemented 每个会话的 `InputMachine` 状态在实时草稿旁保存仅限运行时的有序附件标识符。框架持有的 chat store 只接收草稿的纯文本持久化镜像，`ConversationService` 则持有相应的浏览器专用 `File` 与对象 URL 注册表： ```ts -export {} +import type { Branded } from '@deepseek-ai/dsh-brand' + +type DraftAttachmentId = Branded<'DraftAttachmentId'> interface ChatStoreState { selection: object | null @@ -54,12 +56,12 @@ interface ChatStoreState { interface InputState { draft: string - imageIds: readonly string[] + imageIds: readonly DraftAttachmentId[] } interface ComposerAttachment { kind: 'image' - id: string + id: DraftAttachmentId file: File previewUrl: string } @@ -67,7 +69,7 @@ interface ComposerAttachment { 这一拆分把会话 provide 通道的输入 hook 与 actions 用作实时输入区状态的唯一订阅路径，同时避免把不可序列化的浏览器对象写进持久 JSON。只有纯文本草稿镜像使用 `localStorage`；附件标识符、浏览器 `File` 对象和对象 URL 都限定在实时会话输入外壳的 scope 内。未发送图片因此无法跨重载或会话 scope 释放保留。原生客户端可以在操作系统临时目录中暂存输入，但必须像对待浏览器对象 URL 一样对待该路径：不再需要时删除，并在消息被接受前把字节复制进持久存储。 -本地附件后端依次解析显式 `dshHome`、`$DSH_HOME` 和 `~/.dsh`。它把内容寻址对象存储在 `$DSH_HOME/attachments/v1/objects//` 下，并为目录和文件设置仅所有者可访问的权限。服务先写入并同步临时文件，再以原子方式发布，并对发布目录执行目录同步使其持久（POSIX；Windows 依赖文件系统元数据日志），之后才返回引用。内容摘要编码在不透明的 `sha256:` 标识符中，每次读取都会校验摘要、媒体类型、字节长度、宽度和高度。 +本地附件后端依次解析显式 `dshHome`、`$DSH_HOME` 和 `~/.dsh`。它把内容寻址对象存储在 `$DSH_HOME/attachments/v1/objects//` 下，并为目录和文件设置仅所有者可访问的权限。服务先写入并同步临时文件，再以原子方式发布，并对发布目录执行目录同步使其持久（POSIX；Windows 依赖文件系统元数据日志），之后才返回引用。内容摘要编码在不透明的 `sha256:` 标识符中。写入准入与读取都会完整解码受支持的光栅图片，之后才接受其格式和尺寸；每次读取还会校验摘要、字节长度和已记录的元数据。 第一版不对存储执行自动删除。已发送的用户图片和模型生成图片会一直保留，以供历史记录、恢复和 fork 使用。按引用感知的垃圾回收需要单独设计，因为仅按时间清理可能删除仍被持久会话引用的数据。部署的字节和像素限制是写入时的准入策略；读取时会校验摘要和已记录的元数据，但不重新应用当前准入限制，因此收紧策略不会导致旧历史记录失效。 @@ -112,23 +114,23 @@ type PromptInputPart = } ``` -Base64 只跨越一次 JSON-RPC，并在持久化后丢弃。宿主会校验规范 base64、图片数量、总字节数、单张图片字节数、与魔数字节一致的 MIME、固有尺寸和解码像素数：它会在保存任何成员之前，通过服务边界上不触碰存储的 `validateImage` 校验完整批次，因此一张畸形图片不会把批次中的有效成员留成无引用对象。只有每张图片都成功后，宿主才会用规范化文本和按提交顺序排列的持久图片块调用 agent。失败时不追加用户事件，也不公开任何附件路径或原始字节。 +Base64 只跨越一次 JSON-RPC，并在持久化后丢弃。宿主会校验规范 base64、图片数量、总字节数、单张图片字节数、声明的 MIME 与完整解码后的光栅图片是否一致、固有尺寸和解码像素数。它会在保存任何成员之前，等待服务边界上不触碰存储的 `validateImage` 完成对每个批次成员的校验，因此一张畸形图片不会把批次中的有效成员留成无引用对象。随后按提交顺序执行存储提交，以限制完整光栅解码器的内存占用。如果后续存储 I/O 操作失败，宿主不会追加用户事件，但先前的不可变内容寻址对象可能保持无引用状态；第一版将清理留给未来按引用感知的垃圾回收，而不向去重存储添加破坏性回滚。只有每张图片都成功后，宿主才会用规范化文本和按提交顺序排列的持久图片块调用 agent。失败时不公开任何附件路径或原始字节。 -`session.attachment` 是只读且限定于会话作用域的端点。只有该会话中的持久事件引用了所请求的附件标识符，宿主才提供字节。会话处于渲染状态时，客户端会按会话和附件标识符对加载操作去重；已渲染会话释放时会撤销已解析的 URL，并使延迟完成的加载失效，以免已卸载的会话重新写入缓存。 +`session.attachment` 是只读且限定于会话作用域的端点。只有该会话中的持久事件引用了所请求的附件标识符，宿主才提供字节。会话处于渲染状态时，客户端会按会话和附件标识符对加载操作去重；已渲染会话释放时会撤销已解析的 URL，并在分配对象 URL 前拒绝已失效的延迟加载，以免已卸载的会话或已释放的服务重新写入缓存。 ### 模型能力与提供方行为 模型目录项增加可选且可合并扩展的输入模态声明。缺少声明表示未知；声明存在但不含 `image`，则明确表示不支持图片。 -宿主是权威的前置检查边界。它会解析会话最新路由到的提供方和模型，并在缺失时依次回退到 agent 选项和宿主默认值；如果该模型明确排除图片输入，宿主会在写入任何附件或事件前拒绝提示词，客户端则恢复草稿。能力未知时继续进入适配器强制检查，使未收录的模型标识符仍然可用。浏览器会在分配预览 URL 前拒绝声明不支持的图片媒体类型，但不会为部署限制或模型能力保留快照：握手快照无法表达 `session.selectModel` 之后会话的当前目标，部署策略也可能独立变化。宿主会根据当前的单张字节数、图片数量、总字节数、媒体类型、尺寸、像素数和路由模型策略校验整个批次，再写入任何附件或事件；其拒绝通过 composer 错误条呈现。 +宿主是权威的前置检查边界。它会解析会话最新路由到的提供方和模型，并在缺失时依次回退到 agent 选项和宿主默认值；如果该模型明确排除图片输入，宿主会在写入任何附件或事件前拒绝提示词，客户端则恢复草稿。包含图片的提示词准入与模型选择共用一个逐 agent 的串行边界，而且已经出队的提示词在其持久消息事件发布前仍保持待发布状态（[顺序决策](../bug-fix/2026-07-29-atomic-web-image-admission.md)）。当图片正等待发布或仍存在于会话当前的派生历史中时，模型选择会拒绝纯文本目标；压缩（compaction）可以移除旧图片，使之后选择纯文本目标变得有效。能力未知时继续进入适配器强制检查，使未收录的模型标识符仍然可用。浏览器会在分配预览 URL 前拒绝声明不支持的图片媒体类型，但不会为部署限制或模型能力保留快照：握手快照无法表达 `session.selectModel` 之后会话的当前目标，部署策略也可能独立变化。宿主会根据当前的单张字节数、图片数量、总字节数、媒体类型、尺寸、像素数和路由模型策略校验整个批次，再写入任何附件或事件；其拒绝通过 composer 错误条呈现。 -Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachments`，再解析每个持久引用，并且仅为声明支持图片输入的模型生成提供方原生图片内容。交付的组合会同时注册 Pi-AI OpenAI、Anthropic 路由和仅支持文本的默认 DeepSeek 路由；选择当前提供方／模型仍由宿主组合或配置承担，而不是图片输入 CLI（命令行界面）的功能。在请求时解析服务，可避免 Cordis 加载顺序将可选附件服务的可用性固化。手写 DeepSeek 适配器遇到请求中任何位置的图片时都会抛出类型化的 `UNSUPPORTED_CONTENT` 错误，包括嵌套工具结果中的图片。任何适配器都不得将图片展平或跳过。 +Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachments`，递归转换每个持久图片引用，包括嵌套在工具结果中的引用，并且仅为声明支持图片输入的模型生成提供方原生图片内容。交付的组合会同时注册 Pi-AI OpenAI、Anthropic 路由和仅支持文本的默认 DeepSeek 路由；选择当前提供方／模型仍由宿主组合或配置承担，而不是图片输入 CLI（命令行界面）的功能。在请求时解析服务，可避免 Cordis 加载顺序将可选附件服务的可用性固化。手写 DeepSeek 适配器遇到请求中任何位置的图片时都会抛出类型化的 `UNSUPPORTED_CONTENT` 错误，包括嵌套工具结果中的图片。任何适配器都不得将图片展平或跳过。 核心层支持结构化助手图片块，但当前没有任何生产提供方路径通过图片输出认证。未来任何支持输出的适配器都必须在有界的大小和时间策略下获取提供方字节，通过同一个附件服务校验并持久化字节，之后才能以原子方式发布 `ImageBlock`。助手 Markdown 中的 URL 仍是文本，绝不自动下载。 提供方无关的 token 估算不会根据图片尺寸猜测视觉定价；提供方返回的用量仍是权威值。在 ACP（Agent Client Protocol）接口原生支持图片前，ACP 会渲染明确的图片标记，而不是静默省略该块。 -压缩（compaction）会把选定的会话前缀（包含图片引用）回放到已配置的摘要生成路径中。支持视觉的路径会通过适配器解析这些引用；仅文本路径会明确失败，而不是静默丢弃视觉上下文。合成的检查点仍仅包含文本，`compact-basic` 会以 `UNSUPPORTED_CONTENT` 拒绝包含图片的摘要输出。 +压缩会把选定的会话前缀（包含图片引用）回放到已配置的摘要生成路径中。支持视觉的路径会通过适配器解析这些引用；仅文本路径会明确失败，而不是静默丢弃视觉上下文。合成的检查点仍仅包含文本，`compact-basic` 会以 `UNSUPPORTED_CONTENT` 拒绝包含图片的摘要输出。 ### 历史渲染与原图预览 @@ -140,14 +142,14 @@ Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachme 第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 5 MiB、每条消息 10 张图片和 20 MiB 图片总字节数，以及每张图片 4,000 万个固有像素。这些随部署变化的限制属于经过校验的后端配置，并由宿主在持久化前强制执行。客户端连接载体会独立限制 API 请求体的缓冲大小，并根据宿主附件服务的图片总量限制，加上 base64 和请求封装的膨胀量推导上限；未声明长度的请求体在越过上限的瞬间即被拒绝，而不是先读完再拒。 -格式错误的 base64、不支持或不匹配的媒体、截断的文件头、超出字节限制、超出图片数量、超出像素限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段，控制字符会被移除，并且任何本地路径都不会写入日志或返回浏览器。 +格式错误的 base64、不支持或不匹配的媒体、截断的图片数据、超出字节限制、超出图片数量、超出像素限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段，控制字符会被移除，并且任何本地路径都不会写入日志或返回浏览器。 ### 包与接口变更 | 接口 | 职责 | | --- | --- | | `packages/attachment/attachment` | 不透明附件标识符、图片引用、限制、错误和 `ctx.attachments` 服务。 | -| `packages/attachment/attachment-local` | 私有内容寻址存储、图片头校验、完整性校验和配置。 | +| `packages/attachment/attachment-local` | 私有内容寻址存储、完整光栅解码、完整性校验和配置。 | | `packages/llm/llm` | 角色无关的 `ImageBlock` 和输入模态元数据。 | | `packages/llm/llm-pi-ai` | 将持久且受支持的图片输入解析为提供方原生内容。 | | `packages/llm/llm-deepseek` | 明确拒绝图片内容。 | @@ -177,7 +179,7 @@ Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachme ### 在消息与会话日志中内联 base64 -这种方式会在 RPC、事件、历史分页、fork、压缩（compaction）和浏览器存储中复制二进制数据，还会诱使 token 计量把编码文本当成模型文本。单一不可变对象配合小型引用，可以让持久表示保持有界。 +这种方式会在 RPC、事件、历史分页、fork、压缩和浏览器存储中复制二进制数据，还会诱使 token 计量把编码文本当成模型文本。单一不可变对象配合小型引用，可以让持久表示保持有界。 ### 使用浏览器对象 URL、本地路径或提供方 URL 作为规范内容 @@ -194,9 +196,9 @@ UI 状态可能陈旧，也无法保护直接 SDK、ACP、回放或未收录模 ## 测试 - 存储测试覆盖内容寻址去重、私有权限、准入失败、对象损坏或缺失时的失败，以及收紧部署限制后读取历史数据。 -- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体，以及在会话日志已含图片时拒绝切换到纯文本模型的 `session.selectModel`（接受该切换会让此后每一轮都失败）。 -- 客户端单元测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、顺序，以及草稿、会话作用域和应用层级的对象 URL 清理；keyless 的组装后构建产物通道（`apps/web/tests/image-display.snapshot.ts`，`DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`）覆盖经授权附件路由渲染的历史用户与助手图片画廊、原图 lightbox，以及 composer 粘贴缩略图条。 -- 适配器与压缩测试覆盖 Pi-AI 原生图片转换、后置附件服务组合、仅文本拒绝、嵌套工具结果中的图片、保留摘要输入，以及明确拒绝图片输出。 +- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体、图片准入与模型选择的竞态、待发布状态，以及压缩后依据当前派生历史进行的选择。 +- 客户端单元测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、顺序、草稿、会话作用域和应用层级的对象 URL 清理，以及一项在释放后才完成的延迟历史读取；keyless 的组装后构建产物通道（`apps/web/tests/image-display.snapshot.ts`，`DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`）覆盖经授权附件路由渲染的历史用户与助手图片画廊、原图 lightbox，以及 composer 粘贴缩略图条。 +- 适配器与压缩测试覆盖 Pi-AI 原生图片转换、后置附件服务组合、仅文本拒绝、递归嵌套在工具结果中的图片、保留摘要输入，以及明确拒绝图片输出。 - 需要凭据的实际 API 测试会通过 Anthropic `claude-opus-4-8` 路径发送一张 PNG，并要求模型识别其中的二维码。 - 当前生产适配器集合没有经过认证的图片输出路由；输出提供方认证仍不在第一版范围内。 diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 71d009fea3..f480d9581f 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -264,8 +264,9 @@ Immutable binary attachment service. Implementations validate bytes before publi * Validate one image without persisting it. * Batch callers validate every member before saving any member. * @param input - encoded bytes, declared media type, and optional display name. + * @returns completion after the encoded raster has been fully decoded. */ -abstract validateImage(input: SaveImageAttachment): void +abstract validateImage(input: SaveImageAttachment): Promise /** * Validate and durably commit one image before its owning session event is appended. diff --git a/docs/core-data-structures/attachment.i18n.yaml b/docs/core-data-structures/attachment.i18n.yaml index ccbe788946..247079b07f 100644 --- a/docs/core-data-structures/attachment.i18n.yaml +++ b/docs/core-data-structures/attachment.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/attachment.md -attachment.md: 566f0198ce6e797e270b63e3cfd255a0025d7135 -attachment.zh.md: 95d755407b3b2ea7ab5b109f2e78edea16ccdd9c +attachment.md: 5423891d2be483364d48d8520a596c52d5f6e66e +attachment.zh.md: d1183b5a984cb5f70fce5aa1b739e280b89c3f47 diff --git a/docs/core-data-structures/attachment.md b/docs/core-data-structures/attachment.md index 566f0198ce..5423891d2b 100644 --- a/docs/core-data-structures/attachment.md +++ b/docs/core-data-structures/attachment.md @@ -36,7 +36,7 @@ interface ImageAttachmentRef { ``` ```ts type-equiv -/** Deployment-resolved limits shared by upload consumers and UI preflight. */ +/** Deployment-resolved limits used by upload admission and request buffering. */ interface ImageAttachmentLimits { maxImageBytes: number maxImagesPerMessage: number @@ -54,7 +54,7 @@ The reference records intrinsic dimensions and encoded length so clients can lay /** Request to validate and durably commit one image. */ interface SaveImageAttachment { data: Uint8Array - /** Caller-declared media type, checked against magic bytes. */ + /** Caller-declared media type, checked against fully decoded bytes. */ mediaType: ImageMediaType /** Optional browser/provider display name; it is never interpreted as a path. */ name?: string @@ -69,4 +69,4 @@ interface StoredImageAttachment { } ``` -`saveImage()` validates bytes and atomically commits one object before returning its reference. `validateImage()` runs the same admission checks without persisting anything; batch callers validate every member through it before saving any member, so admission rejection leaves no partial objects behind. `readImage()` accepts a reference from an authorized session path and returns bytes only after integrity verification. The service is deliberately retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to any one session's deletion. +`saveImage()` validates bytes and atomically commits one object before returning its reference. `validateImage()` runs the same admission checks without persisting anything; batch callers validate every member through it before saving any member, so validation rejection leaves no partial objects behind. `readImage()` accepts a reference from an authorized session path and returns bytes only after integrity verification. The service is deliberately retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to any one session's deletion. diff --git a/docs/core-data-structures/attachment.zh.md b/docs/core-data-structures/attachment.zh.md index 95d755407b..d1183b5a98 100644 --- a/docs/core-data-structures/attachment.zh.md +++ b/docs/core-data-structures/attachment.zh.md @@ -36,7 +36,7 @@ interface ImageAttachmentRef { ``` ```ts type-equiv -/** Deployment-resolved limits shared by upload consumers and UI preflight. */ +/** Deployment-resolved limits used by upload admission and request buffering. */ interface ImageAttachmentLimits { maxImageBytes: number maxImagesPerMessage: number @@ -54,7 +54,7 @@ interface ImageAttachmentLimits { /** Request to validate and durably commit one image. */ interface SaveImageAttachment { data: Uint8Array - /** Caller-declared media type, checked against magic bytes. */ + /** Caller-declared media type, checked against fully decoded bytes. */ mediaType: ImageMediaType /** Optional browser/provider display name; it is never interpreted as a path. */ name?: string @@ -69,4 +69,4 @@ interface StoredImageAttachment { } ``` -`saveImage()` 校验字节并以原子方式提交一个对象，之后才返回其引用。`validateImage()` 执行相同的准入检查，但不持久化任何内容；批量调用方会在保存任何成员前通过它校验所有成员，因此准入拒绝不会留下部分对象。`readImage()` 接受来自已授权会话路径的引用，只在完整性校验通过后返回字节。该服务刻意不规定保留策略：恢复和 fork 后的会话可能共享对象，因此基于引用的垃圾回收会延期实现，而不是与任何一个会话的删除绑定。 +`saveImage()` 校验字节并以原子方式提交一个对象，之后才返回其引用。`validateImage()` 执行相同的准入检查，但不持久化任何内容；批量调用方会在保存任何成员前通过它校验所有成员，因此校验拒绝不会留下部分对象。`readImage()` 接受来自已授权会话路径的引用，只在完整性校验通过后返回字节。该服务刻意不规定保留策略：恢复和 fork 后的会话可能共享对象，因此基于引用的垃圾回收会延期实现，而不是与任何一个会话的删除绑定。 diff --git a/packages/attachment/attachment-local/README.i18n.yaml b/packages/attachment/attachment-local/README.i18n.yaml index 3b02e4d79c..77b716173f 100644 --- a/packages/attachment/attachment-local/README.i18n.yaml +++ b/packages/attachment/attachment-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/attachment/attachment-local/README.md -README.md: 38331df827d64c8370d0208f6898971e46010afa -README.zh.md: bc6a471465c7bffc136e8ec2233e200a292f2d1a +README.md: 310120bd1c3573da1e7c60334d5c2712195f3186 +README.zh.md: 9fd3a857eca1c25c90b665735f67d2c27c92334a diff --git a/packages/attachment/attachment-local/README.md b/packages/attachment/attachment-local/README.md index 38331df827..310120bd1c 100644 --- a/packages/attachment/attachment-local/README.md +++ b/packages/attachment/attachment-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The private local implementation of [`@deepseek-ai/dsh-attachment`](../attachment). Objects land at `/attachments/v1/objects//` and are addressed by an opaque `sha256:` id. Writes use a private staging directory, owner-only files, a synced temporary file, an atomic exclusive hard-link publish, and a directory sync on the publication directories (POSIX; Windows relies on filesystem metadata journaling) so the reported reference survives a crash; reads re-check the digest, media signature, dimensions, and logged metadata. Byte and pixel limits are write-time admission policy, so a later policy reduction does not make already-admitted history unreadable. +The private local implementation of [`@deepseek-ai/dsh-attachment`](../attachment). Objects land at `/attachments/v1/objects//` and are addressed by an opaque `sha256:` id. Writes use a private staging directory, owner-only files, a synced temporary file, an atomic exclusive hard-link publish, and a directory sync on the publication directories (POSIX; Windows relies on filesystem metadata journaling) so the reported reference survives a crash. Write admission and reads fully decode the raster before accepting its format and dimensions; reads also re-check the digest and logged metadata. Byte and pixel limits are write-time admission policy, so a later policy reduction does not make already-admitted history unreadable. `DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata, never this host path. diff --git a/packages/attachment/attachment-local/README.zh.md b/packages/attachment/attachment-local/README.zh.md index bc6a471465..9fd3a857ec 100644 --- a/packages/attachment/attachment-local/README.zh.md +++ b/packages/attachment/attachment-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -这是 [`@deepseek-ai/dsh-attachment`](../attachment) 的私有本地实现。对象存放在 `/attachments/v1/objects//`，并通过不透明的 `sha256:` 标识符寻址。写入过程使用私有暂存目录、仅所有者可访问的文件、经过同步的临时文件、原子且排他的硬链接发布，并对发布目录执行同步（适用于 POSIX；Windows 依赖文件系统元数据日志），确保已报告的引用能够在崩溃后继续存在；读取过程会重新校验摘要、媒体签名、尺寸和已记录的元数据。字节和像素限制属于写入时的准入策略，因此后续收紧限制不会导致已经接纳的历史记录变得不可读。 +这是 [`@deepseek-ai/dsh-attachment`](../attachment) 的私有本地实现。对象存放在 `/attachments/v1/objects//`，并通过不透明的 `sha256:` 标识符寻址。写入过程使用私有暂存目录、仅所有者可访问的文件、经过同步的临时文件、原子且排他的硬链接发布，并对发布目录执行同步（适用于 POSIX；Windows 依赖文件系统元数据日志），确保已报告的引用能够在崩溃后继续存在。写入准入与读取都会完整解码光栅图片，之后才接受其格式和尺寸；读取还会重新校验摘要和已记录的元数据。字节和像素限制属于写入时的准入策略，因此后续收紧限制不会导致已经接纳的历史记录变得不可读。 `DSH_HOME` 按共享路径策略解析：显式配置、`$DSH_HOME`，最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据，绝不包含这个宿主路径。 diff --git a/packages/attachment/attachment-local/package.json b/packages/attachment/attachment-local/package.json index b87fd83e79..239063542f 100644 --- a/packages/attachment/attachment-local/package.json +++ b/packages/attachment/attachment-local/package.json @@ -20,7 +20,10 @@ "@deepseek-ai/dsh-paths": "^0.0.1", "cordis": "^4.0.0-rc.6" }, - "dependencies": { "schemastery": "^3.18.0" }, + "dependencies": { + "schemastery": "^3.18.0", + "sharp": "^0.35.3" + }, "devDependencies": { "@deepseek-ai/dsh-attachment": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", diff --git a/packages/attachment/attachment-local/src/image.ts b/packages/attachment/attachment-local/src/image.ts index 4e7ae5aabb..1ef30358a2 100644 --- a/packages/attachment/attachment-local/src/image.ts +++ b/packages/attachment/attachment-local/src/image.ts @@ -1,102 +1,45 @@ -/** Minimal raster header validation used before bytes enter durable storage. */ +/** Raster decoding used before bytes enter durable storage. */ +import sharp from 'sharp' import { AttachmentError } from '@deepseek-ai/dsh-attachment' import type { ImageMediaType } from '@deepseek-ai/dsh-attachment' -/** Decoded metadata from a supported image header. */ +/** Decoded metadata from a supported image. */ export interface DetectedImage { mediaType: ImageMediaType width: number height: number } -function ascii(data: Uint8Array, start: number, value: string): boolean { - /* v8 ignore next -- Every call site establishes the fixed header span before comparing it. */ - if (data.length < start + value.length) return false - for (let i = 0; i < value.length; i++) if (data[start + i] !== value.charCodeAt(i)) return false - return true -} - -function u16be(data: Uint8Array, offset: number): number { - return new DataView(data.buffer, data.byteOffset, data.byteLength).getUint16(offset) -} - -function u16le(data: Uint8Array, offset: number): number { - return new DataView(data.buffer, data.byteOffset, data.byteLength).getUint16(offset, true) -} - -function u24le(data: Uint8Array, offset: number): number { - const view = new DataView(data.buffer, data.byteOffset, data.byteLength) - return view.getUint8(offset) | (view.getUint8(offset + 1) << 8) | (view.getUint8(offset + 2) << 16) -} - -function u32be(data: Uint8Array, offset: number): number { - return new DataView(data.buffer, data.byteOffset, data.byteLength).getUint32(offset) -} - -function u32le(data: Uint8Array, offset: number): number { - return new DataView(data.buffer, data.byteOffset, data.byteLength).getUint32(offset, true) -} - -function dimensions(width: number, height: number, mediaType: ImageMediaType): DetectedImage { - if (width < 1 || height < 1) throw new AttachmentError('Image dimensions must be positive.', 'INVALID_IMAGE') - return { mediaType, width, height } -} - -function jpeg(data: Uint8Array): DetectedImage | null { - if (data[0] !== 0xff || data[1] !== 0xd8) return null - const sof = new Set([0xc0, 0xc1, 0xc2, 0xc3, 0xc5, 0xc6, 0xc7, 0xc9, 0xca, 0xcb, 0xcd, 0xce, 0xcf]) - let offset = 2 - while (offset + 3 < data.length) { - while (data[offset] === 0xff) offset++ - const marker = data[offset] - if (marker === undefined || marker === 0xd9 || marker === 0xda) break - if (marker === 0x01 || (marker >= 0xd0 && marker <= 0xd7)) { - offset++ - continue - } - const length = u16be(data, offset + 1) - if (length < 2 || offset + 1 + length > data.length) throw new AttachmentError('JPEG data is truncated.', 'INVALID_IMAGE') - if (sof.has(marker)) { - if (length < 7) throw new AttachmentError('JPEG dimensions are truncated.', 'INVALID_IMAGE') - return dimensions(u16be(data, offset + 6), u16be(data, offset + 4), 'image/jpeg') - } - offset += length + 1 - } - throw new AttachmentError('JPEG dimensions are missing.', 'INVALID_IMAGE') +const MEDIA_TYPES: Readonly> = { + png: 'image/png', + jpeg: 'image/jpeg', + webp: 'image/webp', + gif: 'image/gif', } /** - * Detect a supported raster type and intrinsic dimensions from encoded bytes. + * Decode a supported raster and return its intrinsic metadata. * @param data - complete encoded image bytes. + * @param maxPixels - optional write-time decoded-pixel limit; reads omit it. * @returns verified format and dimensions. */ -export function detectImage(data: Uint8Array): DetectedImage { - if (data.length >= 24 - && data[0] === 0x89 && ascii(data, 1, 'PNG\r\n\u001a\n') && ascii(data, 12, 'IHDR')) { - return dimensions(u32be(data, 16), u32be(data, 20), 'image/png') - } - if (data.length >= 10 && (ascii(data, 0, 'GIF87a') || ascii(data, 0, 'GIF89a'))) { - return dimensions(u16le(data, 6), u16le(data, 8), 'image/gif') - } - const detectedJpeg = jpeg(data) - if (detectedJpeg !== null) return detectedJpeg - if (data.length >= 30 && ascii(data, 0, 'RIFF') && ascii(data, 8, 'WEBP')) { - const declaredLength = u32le(data, 4) + 8 - if (declaredLength > data.length) throw new AttachmentError('WebP data is truncated.', 'INVALID_IMAGE') - if (ascii(data, 12, 'VP8X')) return dimensions(u24le(data, 24) + 1, u24le(data, 27) + 1, 'image/webp') - if (ascii(data, 12, 'VP8L') && data[20] === 0x2f) { - const view = new DataView(data.buffer, data.byteOffset, data.byteLength) - const b0 = view.getUint8(21) - const b1 = view.getUint8(22) - const b2 = view.getUint8(23) - const b3 = view.getUint8(24) - return dimensions(1 + b0 + ((b1 & 0x3f) << 8), 1 + (b1 >> 6) + (b2 << 2) + ((b3 & 0x0f) << 10), 'image/webp') +export async function detectImage(data: Uint8Array, maxPixels?: number): Promise { + try { + const image = sharp(data, { failOn: 'error', limitInputPixels: false }) + const metadata = await image.metadata() + const mediaType = MEDIA_TYPES[metadata.format as string] + if (mediaType === undefined) { + throw new AttachmentError('Unsupported or malformed image data.', 'INVALID_IMAGE') } - if (ascii(data, 12, 'VP8 ') && data[23] === 0x9d && data[24] === 0x01 && data[25] === 0x2a) { - return dimensions(u16le(data, 26) & 0x3fff, u16le(data, 28) & 0x3fff, 'image/webp') + const { width, height } = metadata + if (maxPixels !== undefined && width * height > maxPixels) { + throw new AttachmentError('Image exceeds the configured decoded-pixel limit.', 'IMAGE_TOO_MANY_PIXELS') } - throw new AttachmentError('WebP dimensions are missing.', 'INVALID_IMAGE') + await image.raw().toBuffer() + return { mediaType, width, height } + } catch (error) { + if (error instanceof AttachmentError) throw error + throw new AttachmentError('Unsupported or malformed image data.', 'INVALID_IMAGE', { cause: error }) } - throw new AttachmentError('Unsupported or malformed image data.', 'INVALID_IMAGE') } diff --git a/packages/attachment/attachment-local/src/index.ts b/packages/attachment/attachment-local/src/index.ts index c015887bfc..7ed4824ef2 100644 --- a/packages/attachment/attachment-local/src/index.ts +++ b/packages/attachment/attachment-local/src/index.ts @@ -60,8 +60,8 @@ export class LocalAttachmentStore extends AttachmentStore { }) } - validateImage(input: SaveImageAttachment): void { - validateImageFile(input, this.imageLimits) + async validateImage(input: SaveImageAttachment): Promise { + await validateImageFile(input, this.imageLimits) } async saveImage(input: SaveImageAttachment): Promise { diff --git a/packages/attachment/attachment-local/src/store.ts b/packages/attachment/attachment-local/src/store.ts index b8a5c6aa3c..f489ae315c 100644 --- a/packages/attachment/attachment-local/src/store.ts +++ b/packages/attachment/attachment-local/src/store.ts @@ -38,27 +38,28 @@ function ensureReference(ref: ImageAttachmentRef): string { return match[1] } -function inspectMetadata(data: Uint8Array, declaredMediaType: ImageAttachmentRef['mediaType']): Omit { +async function inspectMetadata( + data: Uint8Array, + declaredMediaType: ImageAttachmentRef['mediaType'], + maxPixels?: number, +): Promise> { if (data.byteLength === 0) throw new AttachmentError('Image is empty.', 'INVALID_IMAGE') - const detected = detectImage(data) + const detected = await detectImage(data, maxPixels) if (detected.mediaType !== declaredMediaType) throw new AttachmentError('Declared image type does not match its bytes.', 'IMAGE_TYPE_MISMATCH') return { ...detected, bytes: data.byteLength } } -function validateAdmission(metadata: Omit, limits: ImageAttachmentLimits): void { - if (metadata.bytes > limits.maxImageBytes) throw new AttachmentError('Image exceeds the configured byte limit.', 'IMAGE_TOO_LARGE') - if (metadata.width * metadata.height > limits.maxImagePixels) { - throw new AttachmentError('Image exceeds the configured decoded-pixel limit.', 'IMAGE_TOO_MANY_PIXELS') - } -} - /** * Run the full admission policy for one image without touching storage. * @param input - encoded bytes and declared metadata. * @param limits - resolved storage policy. + * @returns completion after the encoded raster has been fully decoded. */ -export function validateImageFile(input: SaveImageAttachment, limits: ImageAttachmentLimits): void { - validateAdmission(inspectMetadata(input.data, input.mediaType), limits) +export async function validateImageFile(input: SaveImageAttachment, limits: ImageAttachmentLimits): Promise { + if (input.data.byteLength > limits.maxImageBytes) { + throw new AttachmentError('Image exceeds the configured byte limit.', 'IMAGE_TOO_LARGE') + } + await inspectMetadata(input.data, input.mediaType, limits.maxImagePixels) } /** @@ -112,8 +113,8 @@ async function ensureDurableDirectory(path: string, boundary: string): Promise { - const metadata = inspectMetadata(input.data, input.mediaType) - validateAdmission(metadata, limits) + if (input.data.byteLength > limits.maxImageBytes) throw new AttachmentError('Image exceeds the configured byte limit.', 'IMAGE_TOO_LARGE') + const metadata = await inspectMetadata(input.data, input.mediaType, limits.maxImagePixels) const sha256 = digest(input.data) const bucket = join(root, 'objects', sha256.slice(0, 2)) const staging = join(root, 'tmp') @@ -187,7 +188,7 @@ export async function readImageFile(root: string, ref: ImageAttachmentRef): Prom throw new AttachmentError('Unable to read image attachment.', 'ATTACHMENT_READ_FAILED', { cause: error }) } if (digest(data) !== sha256) throw new AttachmentError('Stored attachment failed integrity verification.', 'ATTACHMENT_CORRUPT') - const metadata = inspectMetadata(data, ref.mediaType) + const metadata = await inspectMetadata(data, ref.mediaType) if (metadata.bytes !== ref.bytes || metadata.width !== ref.width || metadata.height !== ref.height) { throw new AttachmentError('Stored attachment metadata does not match its reference.', 'ATTACHMENT_CORRUPT') } diff --git a/packages/attachment/attachment-local/tests/image.spec.ts b/packages/attachment/attachment-local/tests/image.spec.ts index 04cbe984fd..ccfe7f62e7 100644 --- a/packages/attachment/attachment-local/tests/image.spec.ts +++ b/packages/attachment/attachment-local/tests/image.spec.ts @@ -1,93 +1,42 @@ +import sharp from 'sharp' import { describe, expect, it } from 'vitest' import { detectImage } from '../src/image.ts' -function bytes(text: string): number[] { - return [...Buffer.from(text, 'ascii')] +async function raster(format: 'png' | 'jpeg' | 'webp' | 'gif'): Promise { + const image = sharp({ + create: { width: 3, height: 2, channels: 4, background: { r: 1, g: 2, b: 3, alpha: 1 } }, + }) + return new Uint8Array(await image.toFormat(format).toBuffer()) } -function webp(chunk: string, mutate: (data: Uint8Array) => void): Uint8Array { - const data = new Uint8Array(30) - data.set(bytes('RIFF'), 0) - data.set([22, 0, 0, 0], 4) - data.set(bytes('WEBP'), 8) - data.set(bytes(chunk), 12) - mutate(data) - return data -} - -describe('raster header detection', () => { - it('detects PNG dimensions', () => { - const data = Uint8Array.from(Buffer.from( - 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=', - 'base64', - )) - expect(detectImage(data)).toEqual({ mediaType: 'image/png', width: 1, height: 1 }) +describe('raster decoding', () => { + it('decodes every supported format and its intrinsic dimensions', async () => { + for (const [format, mediaType] of [ + ['png', 'image/png'], + ['jpeg', 'image/jpeg'], + ['webp', 'image/webp'], + ['gif', 'image/gif'], + ] as const) { + await expect(detectImage(await raster(format))) + .resolves.toEqual({ mediaType, width: 3, height: 2 }) + } }) - it('detects both GIF revisions and rejects zero dimensions', () => { - expect(detectImage(Uint8Array.from([...bytes('GIF87a'), 3, 0, 2, 0]))) - .toEqual({ mediaType: 'image/gif', width: 3, height: 2 }) - expect(detectImage(Uint8Array.from([...bytes('GIF89a'), 4, 0, 5, 0]))) - .toEqual({ mediaType: 'image/gif', width: 4, height: 5 }) - expect(() => detectImage(Uint8Array.from([...bytes('GIF89a'), 0, 0, 1, 0]))) - .toThrow(/positive/) - expect(() => detectImage(Uint8Array.from([...bytes('GIF89a'), 1, 0, 0, 0]))) - .toThrow(/positive/) + it('rejects excess decoded pixels before decoding', async () => { + await expect(detectImage(await raster('png'), 5)) + .rejects.toMatchObject({ code: 'IMAGE_TOO_MANY_PIXELS' }) }) - it('walks JPEG marker forms and reports malformed dimensions', () => { - const sof = [0xff, 0xc0, 0, 7, 8, 0, 2, 0, 3] - expect(detectImage(Uint8Array.from([0xff, 0xd8, ...sof]))) - .toEqual({ mediaType: 'image/jpeg', width: 3, height: 2 }) - expect(detectImage(Uint8Array.from([ - 0xff, 0xd8, - 0xe0, 0, 2, - 0x01, - 0xff, ...sof, - ]))).toEqual({ mediaType: 'image/jpeg', width: 3, height: 2 }) - - expect(() => detectImage(Uint8Array.from([0xff, 0xd8, 0xd9, 0, 0, 0]))) - .toThrow(/missing/) - expect(() => detectImage(Uint8Array.from([0xff, 0xd8, 0xff, 0xff, 0xff, 0xff]))) - .toThrow(/missing/) - expect(() => detectImage(Uint8Array.from([0xff, 0xd8, 0xe0, 0, 1, 0]))) - .toThrow(/truncated/) - expect(() => detectImage(Uint8Array.from([0xff, 0xd8, 0xe0, 0, 9, 0]))) - .toThrow(/truncated/) - expect(() => detectImage(Uint8Array.from([0xff, 0xd8, 0xc0, 0, 6, 0, 0, 0, 0]))) - .toThrow(/dimensions are truncated/) - }) - - it('detects each WebP header and rejects truncated or unknown chunks', () => { - expect(detectImage(webp('VP8X', (data) => { - data.set([2, 0, 0], 24) - data.set([3, 0, 0], 27) - }))).toEqual({ mediaType: 'image/webp', width: 3, height: 4 }) - - expect(detectImage(webp('VP8L', (data) => { - data[20] = 0x2f - data.set([2, 0, 1, 0], 21) - }))).toEqual({ mediaType: 'image/webp', width: 3, height: 5 }) - - expect(detectImage(webp('VP8 ', (data) => { - data.set([0x9d, 0x01, 0x2a], 23) - data.set([6, 0, 7, 0], 26) - }))).toEqual({ mediaType: 'image/webp', width: 6, height: 7 }) - - const truncated = webp('VP8X', () => {}) - truncated[4] = 23 - expect(() => detectImage(truncated)).toThrow(/truncated/) - expect(() => detectImage(webp('NOPE', () => {}))).toThrow(/dimensions are missing/) - expect(() => detectImage(webp('VP8L', () => {}))).toThrow(/dimensions are missing/) - expect(() => detectImage(webp('VP8 ', () => {}))).toThrow(/dimensions are missing/) - }) - - it('rejects unrecognized bytes and near-miss signatures', () => { - expect(() => detectImage(new Uint8Array(0))).toThrow(/Unsupported/) - expect(() => detectImage(Uint8Array.from([...bytes('GIFxxa'), 1, 0, 1, 0]))) - .toThrow(/Unsupported/) - const nearWebp = webp('VP8X', () => {}) - nearWebp[8] = 0 - expect(() => detectImage(nearWebp)).toThrow(/Unsupported/) + it('rejects malformed bytes and truncated payloads with readable headers', async () => { + await expect(detectImage(Uint8Array.of(1, 2, 3))) + .rejects.toMatchObject({ code: 'INVALID_IMAGE' }) + const unsupported = await sharp({ + create: { width: 1, height: 1, channels: 4, background: { r: 0, g: 0, b: 0, alpha: 1 } }, + }).tiff().toBuffer() + await expect(detectImage(unsupported)).rejects.toMatchObject({ code: 'INVALID_IMAGE' }) + const complete = await raster('png') + const truncated = complete.subarray(0, 62) + await expect(sharp(truncated).metadata()).resolves.toMatchObject({ width: 3, height: 2 }) + await expect(detectImage(truncated)).rejects.toMatchObject({ code: 'INVALID_IMAGE' }) }) }) diff --git a/packages/attachment/attachment-local/tests/index.spec.ts b/packages/attachment/attachment-local/tests/index.spec.ts index 22912a2963..7ea71d166b 100644 --- a/packages/attachment/attachment-local/tests/index.spec.ts +++ b/packages/attachment/attachment-local/tests/index.spec.ts @@ -42,13 +42,13 @@ describe('local attachment service', () => { const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-validate-')) try { const service = new LocalAttachmentStore(new Context(), { dshHome }) - expect(() => { service.validateImage({ data: Uint8Array.of(1, 2, 3), mediaType: 'image/png' }) }) - .toThrow(/Unsupported or malformed image data/) + await expect(service.validateImage({ data: Uint8Array.of(1, 2, 3), mediaType: 'image/png' })) + .rejects.toThrow(/Unsupported or malformed image data/) const valid = Uint8Array.from(Buffer.from( 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=', 'base64', )) - expect(() => { service.validateImage({ data: valid, mediaType: 'image/png' }) }).not.toThrow() + await expect(service.validateImage({ data: valid, mediaType: 'image/png' })).resolves.toBeUndefined() expect(existsSync(service.root)).toBe(false) } finally { await rm(dshHome, { recursive: true, force: true }) diff --git a/packages/attachment/attachment-local/tests/store.spec.ts b/packages/attachment/attachment-local/tests/store.spec.ts index 9287c702ad..5838b8748c 100644 --- a/packages/attachment/attachment-local/tests/store.spec.ts +++ b/packages/attachment/attachment-local/tests/store.spec.ts @@ -5,6 +5,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { mkdtemp, rm } from 'node:fs/promises' import { afterEach, describe, expect, it, vi } from 'vitest' +import sharp from 'sharp' import type { ImageAttachmentLimits } from '@deepseek-ai/dsh-attachment' import { readImageFile, saveImageFile } from '../src/store.ts' @@ -122,8 +123,9 @@ describe('local attachment store', () => { data: PNG, mediaType: 'image/png', }, { ...LIMITS, maxImageBytes: 1 })).rejects.toMatchObject({ code: 'IMAGE_TOO_LARGE' }) - const wide = PNG.slice() - wide.set([0, 0, 0, 5, 0, 0, 0, 5], 16) + const wide = new Uint8Array(await sharp({ + create: { width: 5, height: 5, channels: 4, background: { r: 0, g: 0, b: 0, alpha: 1 } }, + }).png().toBuffer()) await expect(saveImageFile(storageRoot, { data: wide, mediaType: 'image/png', }, LIMITS)).rejects.toMatchObject({ code: 'IMAGE_TOO_MANY_PIXELS' }) diff --git a/packages/attachment/attachment/src/index.ts b/packages/attachment/attachment/src/index.ts index 74e280a70f..ebe6ad59c3 100644 --- a/packages/attachment/attachment/src/index.ts +++ b/packages/attachment/attachment/src/index.ts @@ -37,8 +37,9 @@ export abstract class AttachmentStore extends Service { * Validate one image without persisting it. * Batch callers validate every member before saving any member. * @param input - encoded bytes, declared media type, and optional display name. + * @returns completion after the encoded raster has been fully decoded. */ - abstract validateImage(input: SaveImageAttachment): void + abstract validateImage(input: SaveImageAttachment): Promise /** * Validate and durably commit one image before its owning session event is appended. diff --git a/packages/attachment/attachment/src/types.ts b/packages/attachment/attachment/src/types.ts index 88b1dceb52..c443cb8763 100644 --- a/packages/attachment/attachment/src/types.ts +++ b/packages/attachment/attachment/src/types.ts @@ -33,7 +33,7 @@ export interface ImageAttachmentRef { name?: string } -/** Deployment-resolved limits shared by upload consumers and UI preflight. */ +/** Deployment-resolved limits used by upload admission and request buffering. */ export interface ImageAttachmentLimits { maxImageBytes: number maxImagesPerMessage: number @@ -45,7 +45,7 @@ export interface ImageAttachmentLimits { /** Request to validate and durably commit one image. */ export interface SaveImageAttachment { data: Uint8Array - /** Caller-declared media type, checked against magic bytes. */ + /** Caller-declared media type, checked against fully decoded bytes. */ mediaType: ImageMediaType /** Optional browser/provider display name; it is never interpreted as a path. */ name?: string diff --git a/packages/client/ui-conversation/README.i18n.yaml b/packages/client/ui-conversation/README.i18n.yaml index 825072d603..e1d3155591 100644 --- a/packages/client/ui-conversation/README.i18n.yaml +++ b/packages/client/ui-conversation/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/client/ui-conversation/README.md -README.md: 74137edccc3ca68c40afd545350ce3811d6ae2e2 -README.zh.md: c974640ba08451d0a666061c6ec46ca88d66a85e +README.md: ad64511120e3d4308ab03bb45de21b7d577b4335 +README.zh.md: 69926a282dea3f13564fe97e75d0c7c937b86335 diff --git a/packages/client/ui-conversation/README.md b/packages/client/ui-conversation/README.md index 74137edccc..ad64511120 100644 --- a/packages/client/ui-conversation/README.md +++ b/packages/client/ui-conversation/README.md @@ -22,9 +22,9 @@ Per-session UI state for selection and the active view lives in the declared cha The composer bar declares session-scoped single seats for `'conversation.input.plan'` (right of the local access-mode control) and `'conversation.input.model'` (immediately before the pending indicator and send/stop button), plus list slots for overlay, dock, left, and right input extensions. Feature packages own each control and its state; ui-conversation supplies placement, the `locked` owner prop, and the standard slot shares. While the `plan` projection's effective target is plan mode, InputBar swaps its textarea placeholder to the plan-task wording, localized through the `command.hint` locale namespace this package registers and shared verbatim with the claimed `/plan` command hint (a host-folded value read through the standard-kit `useProjection`; owner-supplied placeholders win). A pending composer takeover remains mounted when another conversation view is active so the blocked agent can still receive its answer; without a pending interaction, the active-session composer belongs to Chat. The resident no-session shell uses `DisabledInputBar` and therefore dispatches no session-scoped control seats. -Image drafts keep only ordered runtime ids in that store. `ConversationService` owns the corresponding browser `File` and object URLs, applies the latest host capability and upload-limit snapshot before allocation, and releases draft URLs on removal or send plus historical URLs when their rendered session unmounts. Paste and drop share the same validation path; mixed clipboard text remains native textarea input. +Image drafts keep only ordered `DraftAttachmentId` values in that store. `ConversationService` owns the corresponding browser `File` and object URLs, rejects unsupported declared image media types before allocating previews, and releases draft URLs on removal or send plus historical URLs when their rendered session unmounts. A historical read that completes after its rendered session or the service is disposed rejects before allocating an object URL. Paste and drop share the same validation path; mixed clipboard text remains native textarea input. -`src/client/` is organized for the future package split: `contract/` is the sole inter-domain shared face (`slots.ts` slot declarations + composed slot props including the tool-row contract, `views.ts` shared primitives, `tool-call-model.ts`); the `skeleton/`, `chat/`, and `toolviews/` (sample registrants) domain directories import contract files and never each other; `apply.ts` is the only assembly point allowed to import all three domains. The `/client` export surface is the contract only — `apply`/`inject`, the two service classes, and the `contract/` type families; implementation components (skeleton, chat rows) and the store factory stay internal and reach the page exclusively through apply's slot registrations (tests take them via the `./src/*` subpath). +`src/client/` is organized for the future package split: `contract/` is the sole inter-domain shared face (`slots.ts` slot declarations + composed slot props including the tool-row contract, `views.ts` shared primitives, `tool-call-model.ts`); the `skeleton/`, `chat/`, and `toolviews/` (sample registrants) domain directories import contract files and never each other; `apply.ts` is the only assembly point allowed to import all three domains. The `/client` export surface is limited to `apply`/`inject` and contract types; concrete services, implementation components (skeleton, chat rows), and the store factory stay internal. Same-package tests import those internals through `./src/*`. ## Model Experience diff --git a/packages/client/ui-conversation/README.zh.md b/packages/client/ui-conversation/README.zh.md index c974640ba0..69926a282d 100644 --- a/packages/client/ui-conversation/README.zh.md +++ b/packages/client/ui-conversation/README.zh.md @@ -22,9 +22,9 @@ todo 两个面就是在该形状上的两个注册项，都是普通注册方插 输入栏为 `'conversation.input.plan'`（位于本地 access 模式控件右侧）和 `'conversation.input.model'`（渲染在 pending 指示器与发送／停止按钮之前）声明会话作用域的单实例 seat，并为 overlay、dock、left 和 right 输入扩展声明列表 slot。各功能包拥有相应控件及其状态；ui-conversation 提供放置位置、`locked` owner prop 和标准 slot share。当 `plan` 投影的有效目标为 plan mode 时，InputBar 将文本框 placeholder 切换为 plan 任务措辞，经本包注册的 `command.hint` locale 命名空间本地化，并与已认领 `/plan` 命令的提示逐字共用同一份文案（经标准套件 `useProjection` 读取的 host 折叠值；owner 提供的 placeholder 优先）。另一个会话视图活跃时，待处理的 composer 接管仍保持挂载，使被阻塞的 agent（智能体）仍能收到回答；没有待处理交互时，活跃会话的 composer 归 Chat 所有。常驻无会话壳使用 `DisabledInputBar`，因此不会分发任何会话作用域的控件 seat。 -图片草稿在该 store 中只保留有序的运行时 id。`ConversationService` 持有对应的浏览器 `File` 和对象 URL，在分配前应用最新的宿主能力与上传限制快照，并在图片移除或发送时释放草稿 URL，在所渲染的会话卸载时释放历史 URL。粘贴与拖放共用同一校验路径；混合剪贴板文本仍由 textarea 原生输入。 +图片草稿在该 store 中只保留有序的 `DraftAttachmentId` 值。`ConversationService` 持有对应的浏览器 `File` 和对象 URL，会在分配预览前拒绝声明媒体类型不受支持的图片，并在图片移除或发送时释放草稿 URL，在所渲染的会话卸载时释放历史 URL。一项历史读取如果在其所渲染的会话卸载或该服务释放后才完成，会在分配对象 URL 前被拒绝。粘贴与拖放共用同一校验路径；混合剪贴板文本仍由 textarea 原生输入。 -`src/client/` 按未来的包拆分组织：`contract/` 是唯一的跨领域共享表层（`slots.ts` slot 声明 + 组合后的 slot props，包括工具行契约、`views.ts` 共享原语、`tool-call-model.ts`）；`skeleton/`、`chat/` 和 `toolviews/`（示例注册方）领域目录只导入 contract 文件，彼此绝不导入；`apply.ts` 是唯一允许导入全部三个领域的组装点。`/client` 导出表层只包含契约：`apply`／`inject`、两个服务类和 `contract/` 类型家族；实现组件（骨架、聊天行）与 store factory 保持内部状态，只能通过 apply 的 slot 注册到达页面（测试通过 `./src/*` 子路径获取它们）。 +`src/client/` 按未来的包拆分组织：`contract/` 是唯一的跨领域共享表层（`slots.ts` slot 声明 + 组合后的 slot props，包括工具行契约、`views.ts` 共享原语、`tool-call-model.ts`）；`skeleton/`、`chat/` 和 `toolviews/`（示例注册方）领域目录只导入 contract 文件，彼此绝不导入；`apply.ts` 是唯一允许导入全部三个领域的组装点。`/client` 导出表层仅限 `apply`／`inject` 与契约类型；具体服务、实现组件（骨架、聊天行）和 store factory 均保持内部状态。同包测试通过 `./src/*` 导入这些内部实现。 ## 模型体验 diff --git a/packages/client/ui-conversation/package.json b/packages/client/ui-conversation/package.json index 08c1db5736..d765d8310e 100644 --- a/packages/client/ui-conversation/package.json +++ b/packages/client/ui-conversation/package.json @@ -40,6 +40,7 @@ }, "peerDependencies": { "@deepseek-ai/dsh-attachment": "^0.0.1", + "@deepseek-ai/dsh-brand": "^0.0.1", "@deepseek-ai/dsh-client-locale": "^0.0.1", "@deepseek-ai/dsh-client-runtime": "^0.0.1", "@deepseek-ai/dsh-client-ui-primitives": "^0.0.1", @@ -51,6 +52,7 @@ }, "devDependencies": { "@deepseek-ai/dsh-attachment": "workspace:^", + "@deepseek-ai/dsh-brand": "workspace:^", "@deepseek-ai/dsh-client-locale": "workspace:^", "@deepseek-ai/dsh-client-runtime": "workspace:^", "@deepseek-ai/dsh-goal": "workspace:^", diff --git a/packages/client/ui-conversation/src/client/contract/slots.ts b/packages/client/ui-conversation/src/client/contract/slots.ts index 11937d2974..c66e53378b 100644 --- a/packages/client/ui-conversation/src/client/contract/slots.ts +++ b/packages/client/ui-conversation/src/client/contract/slots.ts @@ -6,14 +6,14 @@ import type { } from '@deepseek-ai/dsh-client-ui-slots' import type { CommandNode, ConversationSnapshot, ObservableSnapshot, PendingInteraction, PendingWait, SessionId, ToolCallBlock, WorkspaceId } from '@deepseek-ai/dsh-client-runtime/client' import type {} from '@deepseek-ai/dsh-client-ui-layout/client' -import type { ComposerKeyboard, InputActions, InputNotice, InputState } from '../input/contract.ts' +import type { ComposerKeyboard, DraftAttachmentId, InputActions, InputNotice, InputState } from '../input/contract.ts' import type { createChatStore } from '../stores.ts' import type { CallId, SelectionTarget, ViewTab } from './views.ts' /** Browser-owned image that has not crossed the durable host boundary. */ export interface ComposerAttachment { kind: 'image' - id: string + id: DraftAttachmentId file: File previewUrl: string } @@ -275,9 +275,9 @@ export interface ComposerBarInjected { /** Create browser previews and append their ids to the session input state. */ addImages: (files: readonly File[]) => string | null /** Release one browser preview and remove its id from the session input state. */ - removeImage: (id: string) => void + removeImage: (id: DraftAttachmentId) => void /** Resolve ordered input-state ids to browser-owned draft attachments. */ - draftImages: (ids: readonly string[]) => readonly ComposerAttachment[] + draftImages: (ids: readonly DraftAttachmentId[]) => readonly ComposerAttachment[] /** Cancel the in-flight turn. */ stop: () => void /** diff --git a/packages/client/ui-conversation/src/client/index.ts b/packages/client/ui-conversation/src/client/index.ts index dca423ff4e..1963dcd72e 100644 --- a/packages/client/ui-conversation/src/client/index.ts +++ b/packages/client/ui-conversation/src/client/index.ts @@ -4,8 +4,8 @@ * owns their slot assembly. */ export { apply, inject } from './apply.ts' -export { ConversationService } from './service.ts' export type { IConversation } from './service.ts' +export type { DraftAttachmentId } from './input/contract.ts' export type { CallId, ChatStoreState, SelectionTarget, ViewTab, diff --git a/packages/client/ui-conversation/src/client/input/contract.ts b/packages/client/ui-conversation/src/client/input/contract.ts index 45ec747f5c..a125348b48 100644 --- a/packages/client/ui-conversation/src/client/input/contract.ts +++ b/packages/client/ui-conversation/src/client/input/contract.ts @@ -6,11 +6,15 @@ * (machine.ts) is package-private and never exported. */ import type { ClientContext, SnapshotStore } from '@deepseek-ai/dsh-client-runtime/client' +import type { Branded } from '@deepseek-ai/dsh-brand' import type { ArbitrateKey, ArbitrateOutcome, CommandClaim, ConsumeTokenRequest, PickOutcome, ReferenceInsert, SubmitOutcome, TokenSpan, } from '@deepseek-ai/dsh-client-ui-slash/client' +/** Browser-runtime identity of one unsent image draft. */ +export type DraftAttachmentId = Branded<'DraftAttachmentId'> + /** * The scoped-event application verbs: the hub's bail listeners call these, * and the boolean answer IS the event's bail value (true ⟺ the machine @@ -28,11 +32,11 @@ export interface SessionInput extends InputTarget { /** Single write path for draft text (all mutation rides machine events). */ setDraft(text: string): void /** Append ordered browser-owned draft attachment ids. */ - addImages(ids: readonly string[]): void + addImages(ids: readonly DraftAttachmentId[]): void /** Remove one browser-owned draft attachment id. */ - removeImage(id: string): void + removeImage(id: DraftAttachmentId): void /** Drop ids whose browser objects no longer exist. */ - pruneImages(ids: readonly string[]): void + pruneImages(ids: readonly DraftAttachmentId[]): void /** THE complexity sink: enter adjudication, submit transaction, and the default sink live inside. */ submit(mode?: 'queue' | 'steer'): void /** @@ -65,11 +69,11 @@ export interface InputActions { /** Single public draft write path (full next draft; occurrence math via diff scan). */ setDraft(text: string): void /** Append ordered browser-owned draft attachment ids. */ - addImages(ids: readonly string[]): void + addImages(ids: readonly DraftAttachmentId[]): void /** Remove one browser-owned draft attachment id. */ - removeImage(id: string): void + removeImage(id: DraftAttachmentId): void /** Drop ids whose browser objects no longer exist. */ - pruneImages(ids: readonly string[]): void + pruneImages(ids: readonly DraftAttachmentId[]): void /** Enter submission (adjudication / claim transaction / default sink inside). */ submit(mode?: 'queue' | 'steer'): void } @@ -198,7 +202,7 @@ export interface InputMachineOptions { export interface InputState { readonly draft: string /** Ordered runtime-only image ids; bytes and object URLs stay in ConversationService. */ - readonly imageIds: readonly string[] + readonly imageIds: readonly DraftAttachmentId[] /** Monotonic draft revision (span CAS compares against this). */ readonly draftRev: number readonly phase: 'plain' | 'adjudicating' | 'claimed' | 'submitting' diff --git a/packages/client/ui-conversation/src/client/input/facade.ts b/packages/client/ui-conversation/src/client/input/facade.ts index 7e3057d8ab..2d566d7632 100644 --- a/packages/client/ui-conversation/src/client/input/facade.ts +++ b/packages/client/ui-conversation/src/client/input/facade.ts @@ -13,7 +13,7 @@ import type { ReferenceInsert, SlashController, TokenSpan, } from '@deepseek-ai/dsh-client-ui-slash/client' import type { - EditRange, EditSelection, InputActions, InputEffect, InputNotice, InputState, + DraftAttachmentId, EditRange, EditSelection, InputActions, InputEffect, InputNotice, InputState, PasteComponent, QueuedMessage, SessionInput, SubmitAttempt, } from './contract.ts' import { InputMachine } from './machine.ts' @@ -39,7 +39,7 @@ export interface SessionInputDeps { /** Queue read face; overlaid onto InputState.queue (absent = empty). */ queue?: ObservableSnapshot | undefined /** The plain-message sink (send choreography / materialize fork — the hub owns it). */ - defaultSink(text: string, mode: 'queue' | 'steer', imageIds: readonly string[]): void + defaultSink(text: string, mode: 'queue' | 'steer', imageIds: readonly DraftAttachmentId[]): void } /** Guard tier from the machine phase. */ @@ -79,7 +79,7 @@ export class SessionInputShell implements SessionInput { private readonly core = new InputMachine({ now: () => Date.now() }) private noticeSeq = 0 private lastDraft = '' - private imageIds: readonly string[] = [] + private imageIds: readonly DraftAttachmentId[] = [] private disposed = false /** Draft persistence mirror (chat store write; receives the clipboard projection, never raw placeholders). */ private mirrorFn: ((text: string) => void) | undefined @@ -102,14 +102,14 @@ export class SessionInputShell implements SessionInput { } /** Append ordered browser-owned draft attachment ids. */ - addImages(ids: readonly string[]): void { + addImages(ids: readonly DraftAttachmentId[]): void { if (ids.length === 0 || this.snapshot.phase === 'adjudicating' || this.snapshot.phase === 'submitting') return this.imageIds = [...this.imageIds, ...ids] this.publish() } /** Remove one browser-owned draft attachment id. */ - removeImage(id: string): void { + removeImage(id: DraftAttachmentId): void { const next = this.imageIds.filter(candidate => candidate !== id) if (next.length === this.imageIds.length) return this.imageIds = next @@ -120,7 +120,7 @@ export class SessionInputShell implements SessionInput { * Drop ids whose browser objects no longer exist. * @param available - ids that still resolve through the browser attachment registry. */ - pruneImages(available: readonly string[]): void { + pruneImages(available: readonly DraftAttachmentId[]): void { const keep = new Set(available) const next = this.imageIds.filter(id => keep.has(id)) if (next.length === this.imageIds.length) return @@ -132,7 +132,7 @@ export class SessionInputShell implements SessionInput { * Restore a failed attempt's ids before any images added after submission. * @param ids - ordered identifiers captured by the failed attempt. */ - restoreImages(ids: readonly string[]): void { + restoreImages(ids: readonly DraftAttachmentId[]): void { const current = new Set(this.imageIds) this.imageIds = [...ids.filter(id => !current.has(id)), ...this.imageIds] this.publish() @@ -144,7 +144,7 @@ export class SessionInputShell implements SessionInput { * (the command path gets the same discipline from submit-settled success). * @param imageIds - identifiers included in the committed attempt. */ - commitSend(imageIds: readonly string[]): void { + commitSend(imageIds: readonly DraftAttachmentId[]): void { const submitted = new Set(imageIds) this.imageIds = this.imageIds.filter(id => !submitted.has(id)) this.run(this.core.dispatch({ type: 'send-committed' })) diff --git a/packages/client/ui-conversation/src/client/input/hub.ts b/packages/client/ui-conversation/src/client/input/hub.ts index e0568ce70f..6f1c68a12d 100644 --- a/packages/client/ui-conversation/src/client/input/hub.ts +++ b/packages/client/ui-conversation/src/client/input/hub.ts @@ -11,7 +11,7 @@ import type { ClientContext, ISessions, SessionBinding, SessionFace, SessionId } from '@deepseek-ai/dsh-client-runtime/client' import type { SlashController } from '@deepseek-ai/dsh-client-ui-slash/client' import { queueReadFaceOf } from '../queue/store.ts' -import type { ComposerKeyboard, InputService, SessionInput } from './contract.ts' +import type { ComposerKeyboard, DraftAttachmentId, InputService, SessionInput } from './contract.ts' import type { PopupDismissFace } from './facade.ts' import { SessionInputShell } from './facade.ts' @@ -26,9 +26,9 @@ interface ConversationAttachmentFace { session: SessionFace, text: string, mode: 'queue' | 'steer', - imageIds: readonly string[], + imageIds: readonly DraftAttachmentId[], ): Promise - releaseDraftImage(id: string): void + releaseDraftImage(id: DraftAttachmentId): void } /** Session-addressed input facade registry (InputService face + composer-layer extras). */ @@ -138,7 +138,7 @@ export class InputHub implements InputService { session: SessionFace, text: string, mode: 'queue' | 'steer', - imageIds: readonly string[], + imageIds: readonly DraftAttachmentId[], ): void { if (text === '' && imageIds.length === 0) return const shell = this.shells.get(session.sessionId) diff --git a/packages/client/ui-conversation/src/client/service.ts b/packages/client/ui-conversation/src/client/service.ts index 81caad16b6..1f2b1e1edb 100644 --- a/packages/client/ui-conversation/src/client/service.ts +++ b/packages/client/ui-conversation/src/client/service.ts @@ -15,7 +15,7 @@ import type { Context } from 'cordis' import type { ISessions, SessionFace, SessionId } from '@deepseek-ai/dsh-client-runtime/client' import type { ImageAttachmentRef, ImageMediaType } from '@deepseek-ai/dsh-attachment' import type { ComposerAttachment } from './contract/slots.ts' -import type { InputService } from './input/contract.ts' +import type { DraftAttachmentId, InputService } from './input/contract.ts' /** * The outward conversation face (`ctx.conversation`): the scope-addressed @@ -46,7 +46,7 @@ export interface IConversation { /** Create one browser-only draft descriptor; only its id enters input state. */ function browserDraftAttachment(file: File): ComposerAttachment { - return { kind: 'image', id: crypto.randomUUID(), previewUrl: URL.createObjectURL(file), file } + return { kind: 'image', id: crypto.randomUUID() as DraftAttachmentId, previewUrl: URL.createObjectURL(file), file } } interface ImageUrlEntry { @@ -59,10 +59,11 @@ interface ImageUrlEntry { export class ConversationService extends Service implements IConversation { /** The per-session input machine registry (InputService face, design §5.2). */ readonly input: InputService - private readonly draftAttachments = new Map() + private readonly draftAttachments = new Map() private readonly imageUrls = new Map() private readonly imageGenerations = new Map() private readonly createdImageUrls = new Set() + private disposed = false /** * @param ctx - owning root context (the plugin apply context; the service @@ -74,6 +75,7 @@ export class ConversationService extends Service implements IConversation { super(ctx, 'conversation') this.input = config.input ctx.effect(() => () => { + this.disposed = true for (const url of this.createdImageUrls) URL.revokeObjectURL(url) this.createdImageUrls.clear() this.draftAttachments.clear() @@ -107,7 +109,7 @@ export class ConversationService extends Service implements IConversation { session: SessionFace, text: string, mode: 'queue' | 'steer', - imageIds: readonly string[], + imageIds: readonly DraftAttachmentId[], ): Promise { const attachments = this.draftImages(imageIds) if (attachments.length !== imageIds.length) { @@ -149,7 +151,7 @@ export class ConversationService extends Service implements IConversation { * @param ids - ordered ids from the per-session input state. * @returns attachments still available in this browser runtime. */ - draftImages(ids: readonly string[]): readonly ComposerAttachment[] { + draftImages(ids: readonly DraftAttachmentId[]): readonly ComposerAttachment[] { const attachments: ComposerAttachment[] = [] for (const id of ids) { const attachment = this.draftAttachments.get(id) @@ -162,7 +164,7 @@ export class ConversationService extends Service implements IConversation { * Release one draft attachment preview. * @param id - draft-local attachment id. */ - releaseDraftImage(id: string): void { + releaseDraftImage(id: DraftAttachmentId): void { const attachment = this.draftAttachments.get(id) if (attachment === undefined) return this.draftAttachments.delete(id) @@ -185,6 +187,7 @@ export class ConversationService extends Service implements IConversation { * @returns a browser URL for inline and original-size display. */ resolveImage(sessionId: SessionId, attachment: ImageAttachmentRef): Promise { + if (this.disposed) return Promise.reject(new Error('conversation.resolveImage: service is disposed')) const key = `${sessionId}:${attachment.attachmentId}` const cached = this.imageUrls.get(key) if (cached !== undefined) return cached.pending @@ -194,6 +197,10 @@ export class ConversationService extends Service implements IConversation { const pending = session.readAttachment(attachment.attachmentId) .then((result) => { if (!result.ok) throw new Error(`${result.error.code}: ${result.error.message}`) + if (this.disposed) throw new Error('conversation.resolveImage: service was disposed before loading completed') + if ((this.imageGenerations.get(sessionId) ?? 0) !== generation) { + throw new Error('historical image scope was released before loading completed') + } if (typeof URL.createObjectURL !== 'function') { return `data:${result.value.attachment.mediaType};base64,${bytesToBase64(result.value.data)}` } @@ -201,10 +208,6 @@ export class ConversationService extends Service implements IConversation { const url = URL.createObjectURL(new Blob([bytes.buffer], { type: result.value.attachment.mediaType, })) - if ((this.imageGenerations.get(sessionId) ?? 0) !== generation) { - revokePreview(url) - throw new Error('historical image scope was released before loading completed') - } this.createdImageUrls.add(url) return url }) diff --git a/packages/client/ui-conversation/tests/input-bar.spec.tsx b/packages/client/ui-conversation/tests/input-bar.spec.tsx index 5a5ea3dd2b..4afc2dc81c 100644 --- a/packages/client/ui-conversation/tests/input-bar.spec.tsx +++ b/packages/client/ui-conversation/tests/input-bar.spec.tsx @@ -13,6 +13,7 @@ import { SessionInputShell } from '../src/client/input/facade.ts' import { InputBar } from '../src/client/skeleton/InputBar.tsx' import type { InputBarProps } from '../src/client/skeleton/InputBar.tsx' import type { ComposerAttachment } from '../src/client/contract/slots.ts' +import type { DraftAttachmentId } from '../src/client/input/contract.ts' afterEach(cleanup) @@ -78,7 +79,7 @@ function bench(over?: BenchOptions) { promptError: over?.promptError ?? null, })) const stop = vi.fn() - const removeImage = vi.fn((id: string) => { shell.removeImage(id) }) + const removeImage = vi.fn((id: DraftAttachmentId) => { shell.removeImage(id) }) const slotCalls: { key: string; owner: unknown }[] = [] const renderSlot = ((key: string, owner: object) => { slotCalls.push({ key, owner }) @@ -523,7 +524,7 @@ describe('image draft rail', () => { it('allows image-only send, removes a thumbnail, and opens original preview on double-click', () => { const file = new File([Uint8Array.of(1)], 'pixel.png', { type: 'image/png' }) - const attachment = { kind: 'image' as const, id: 'draft-1', file, previewUrl: 'blob:draft-1' } + const attachment = { kind: 'image' as const, id: 'draft-1' as DraftAttachmentId, file, previewUrl: 'blob:draft-1' } const { view, textarea, sink, removeImage } = bench({ attachments: [attachment] }) const send = view.getByRole('button', { name: 'Send message' }) as HTMLButtonElement expect(send.disabled).toBe(false) @@ -536,7 +537,7 @@ describe('image draft rail', () => { it('opens the original preview on double-click and closes it with Escape', () => { const file = new File([Uint8Array.of(1)], 'pixel.png', { type: 'image/png' }) - const attachment = { kind: 'image' as const, id: 'draft-1', file, previewUrl: 'blob:draft-1' } + const attachment = { kind: 'image' as const, id: 'draft-1' as DraftAttachmentId, file, previewUrl: 'blob:draft-1' } const { view } = bench({ attachments: [attachment] }) fireEvent.doubleClick(view.getByTitle('双击查看原图')) expect(view.getByRole('dialog', { name: '原图预览' })).toBeTruthy() diff --git a/packages/client/ui-conversation/tests/service-orchestration.spec.ts b/packages/client/ui-conversation/tests/service-orchestration.spec.ts index 5f8d77e335..fc3a23f7dd 100644 --- a/packages/client/ui-conversation/tests/service-orchestration.spec.ts +++ b/packages/client/ui-conversation/tests/service-orchestration.spec.ts @@ -6,17 +6,19 @@ import { Context } from 'cordis' import { describe, expect, it, vi } from 'vitest' import { SlotTestRuntime } from '@deepseek-ai/dsh-client-test-runtime' -import { ConversationService } from '@deepseek-ai/dsh-client-ui-conversation/client' +import { AttachmentId } from '@deepseek-ai/dsh-attachment' +import type { SessionFace } from '@deepseek-ai/dsh-client-runtime/client' import { InputHub } from '../src/client/input/hub.ts' +import { ConversationService } from '../src/client/service.ts' -async function bench() { +async function bench(readAttachment?: SessionFace['readAttachment']) { const runtime = await SlotTestRuntime.create() const prompt = vi.fn(() => Promise.resolve({ ok: true as const, value: { accepted: true as const } })) const cancel = vi.fn(() => Promise.resolve({ ok: true as const, value: { accepted: true as const } })) const loadOlder = vi.fn(() => Promise.resolve()) await runtime.sessions.add({ id: 's1', - session: { prompt, cancel, loadOlder }, + session: { prompt, cancel, loadOlder, ...(readAttachment === undefined ? {} : { readAttachment }) }, }) // config.input is required (the apply shares its hub with the inject // factories); the bench passes its own instance explicitly. @@ -25,7 +27,7 @@ async function bench() { await fiber.await() const root = runtime.ctx.get('conversation') as ConversationService const scoped = runtime.sessions.scope('s1')!.get('conversation') as ConversationService - return { runtime, hub, root, scoped, prompt, cancel, loadOlder } + return { runtime, fiber, hub, root, scoped, prompt, cancel, loadOlder } } describe('ConversationService', () => { @@ -122,6 +124,37 @@ describe('ConversationService', () => { await b.runtime.dispose() }) + it('does not publish a historical image URL after disposal', async () => { + let resolveRead!: (result: Awaited>) => void + const readAttachment: SessionFace['readAttachment'] = vi.fn(() => new Promise>>( + (resolve) => { resolveRead = resolve }, + )) + const b = await bench(readAttachment) + const created = vi.spyOn(URL, 'createObjectURL') + const sessionId = b.runtime.sessions.behavior('s1').sessionId + const attachment = { + attachmentId: AttachmentId('image-1'), + mediaType: 'image/png', + bytes: 1, + width: 1, + height: 1, + } as const + const pending = b.root.resolveImage(sessionId, attachment) + await b.fiber.dispose() + await expect(b.root.resolveImage(sessionId, attachment)).rejects.toThrow('service is disposed') + resolveRead({ + ok: true, + value: { + attachment, + data: Uint8Array.of(1), + }, + }) + await expect(pending).rejects.toThrow('service was disposed before loading completed') + expect(created).not.toHaveBeenCalled() + created.mockRestore() + await b.runtime.dispose() + }) + it('fails loudly from the root scope, on an unbound session, or without SessionsService', async () => { const b = await bench() await expect(b.root.send('x', 'queue')).rejects.toThrow(/requires a session scope/) diff --git a/packages/client/ui-conversation/tsconfig.json b/packages/client/ui-conversation/tsconfig.json index b809c605b1..74fd6f0da7 100644 --- a/packages/client/ui-conversation/tsconfig.json +++ b/packages/client/ui-conversation/tsconfig.json @@ -11,6 +11,9 @@ { "path": "../../attachment/attachment" }, + { + "path": "../../util/brand" + }, { "path": "../../../vendor/cordis" }, diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index b6cfbf882a..06e7036e21 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -161,8 +161,8 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ summary: 'Immutable binary attachment service.', methods: [ { - signature: 'abstract validateImage(input: SaveImageAttachment): void', - jsDoc: '/**\n * Validate one image without persisting it.\n * Batch callers validate every member before saving any member.\n * @param input - encoded bytes, declared media type, and optional display name.\n */', + signature: 'abstract validateImage(input: SaveImageAttachment): Promise', + jsDoc: '/**\n * Validate one image without persisting it.\n * Batch callers validate every member before saving any member.\n * @param input - encoded bytes, declared media type, and optional display name.\n * @returns completion after the encoded raster has been fully decoded.\n */', }, { signature: 'abstract saveImage(input: SaveImageAttachment): Promise', diff --git a/packages/host/apiproxy/README.i18n.yaml b/packages/host/apiproxy/README.i18n.yaml index 18202ed036..4571d984b6 100644 --- a/packages/host/apiproxy/README.i18n.yaml +++ b/packages/host/apiproxy/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/host/apiproxy/README.md -README.md: 1f0daedc54888a1951bc83c474f83287aaf42307 -README.zh.md: abf5417cdbe93f1199c621ac101249986969da93 +README.md: 693b2c26a9ec1e7ea31030a3028f05706adbfc3b +README.zh.md: b1766d901d9ed5743cbc766166bfb310c565ef5f diff --git a/packages/host/apiproxy/README.md b/packages/host/apiproxy/README.md index 1f0daedc54..693b2c26a9 100644 --- a/packages/host/apiproxy/README.md +++ b/packages/host/apiproxy/README.md @@ -14,7 +14,7 @@ The layering/protocol decisions are recorded in the [GUI layering and RPC protoc Session titles ride the generic projection pair like every other domain — the history-tail `projections` block plus `session/projection` frames under the `title` key (the bespoke `session/title` frame is retired). Titles do not join `session.list`; cold sessions remain metadata-only there until opening or resuming attaches their logs. `session.rename` accepts an explicit user title (resuming a cold session first), delegating to `ctx.sessionTitle.rename` — the accepted `session/title` event pins the title against automatic regeneration — and returns the normalized title plus its event seq so a client settles its `title` projection cell ahead of the push frame; a title that normalizes to empty returns `title-invalid`. -Session model routing is a session-domain contract. `session.models` returns the selected provider/model/reasoning target with provider-grouped advisory models, exact-route reasoning metadata, and provider-local lookup failures. `session.selectModel` validates the optional adapter-owned reasoning effort and replaces the complete target selected for the next prompt-assembly boundary. Catalog membership is not validation: an adapter may resolve an unlisted model, while an unavailable route or unsupported effort returns `model-unavailable`. +Session model routing is a session-domain contract. `session.models` returns the selected provider/model/reasoning target with provider-grouped advisory models, exact-route reasoning metadata, and provider-local lookup failures. `session.selectModel` validates the optional adapter-owned reasoning effort and replaces the complete target selected for the next prompt-assembly boundary. Selection is serialized with image-bearing prompt admission and rejects a text-only target while an image is pending publication or remains in the current derived history; an image removed by compaction no longer blocks selection. Catalog membership is not validation: an adapter may resolve an unlisted model, while an unavailable route or unsupported effort returns `model-unavailable`. Workspace and Session lists are separate reconnect baselines. `workspace.create` creates a unique name or adopts an existing directory, `workspace.delete` removes only the Workspace registration, `session.create` accepts an optional preallocated Session id, and `host/workspace-changed`, `host/workspace-removed`, plus `host/session-added` carry committed increments in either arrival order. Registration deletion preserves the directory and session logs; its Sessions remain in `session.list` and become Ungrouped. `SessionSummary.blank` and the `host/session-added` frame carry the derived zero-events bit: clients hide blank sessions and reuse them per workspace, flip blank on the first `host/session-status(running:true)`, and treat `session.list` as the reconnect authority; cold summaries are never blank because lazy persistence keeps never-appended sessions out of `list()`. diff --git a/packages/host/apiproxy/README.zh.md b/packages/host/apiproxy/README.zh.md index abf5417cdb..b1766d901d 100644 --- a/packages/host/apiproxy/README.zh.md +++ b/packages/host/apiproxy/README.zh.md @@ -14,7 +14,7 @@ 会话标题与其他所有领域一样搭乘这对通用投影机制——历史尾页的 `projections` 块外加 `title` 键下的 `session/projection` 帧（专设的 `session/title` 帧已下线）。标题不会加入 `session.list`；冷会话在其中仍只有元数据，直到打开或恢复操作附加其日志。`session.rename` 接受用户显式标题（冷会话先恢复），委托给 `ctx.sessionTitle.rename`——被接受的 `session/title` 事件将标题钉住、不再被自动生成覆盖——并返回规范化后的标题及其事件 seq，让 client 在推送帧到达前就结算自己的 `title` 投影格；规范化后为空的标题返回 `title-invalid`。 -会话模型路由属于会话领域契约。`session.models` 返回选中的提供方／模型／推理（reasoning）目标，以及按提供方分组的建议性模型、精确路由推理元数据和逐提供方查询失败记录。`session.selectModel` 校验由适配器持有的可选推理强度，并替换将在下一提示词组装边界使用的完整目标。目录成员关系不构成校验：适配器可以解析未列出的模型，而不可用路由或不受支持的推理强度会返回 `model-unavailable`。 +会话模型路由属于会话领域契约。`session.models` 返回选中的提供方／模型／推理（reasoning）目标，以及按提供方分组的建议性模型、精确路由推理元数据和逐提供方查询失败记录。`session.selectModel` 校验由适配器持有的可选推理强度，并替换将在下一提示词组装边界使用的完整目标。模型选择与包含图片的提示词准入串行执行；当图片正等待发布或仍存在于当前派生历史中时，会拒绝选择纯文本目标；被压缩（compaction）移除的图片不再阻止选择。目录成员关系不构成校验：适配器可以解析未列出的模型，而不可用路由或不受支持的推理强度会返回 `model-unavailable`。 Workspace 列表与 Session 列表是相互独立的重连基线。`workspace.create` 会创建唯一名称或接纳现有目录，`workspace.delete` 只移除 Workspace 注册记录，`session.create` 接受可选的预分配 Session id，`host/workspace-changed`、`host/workspace-removed` 与 `host/session-added` 则以任意到达顺序携带已提交的增量。删除注册记录会保留目录和会话日志；相关 Session 仍留在 `session.list` 中，并进入 Ungrouped。`SessionSummary.blank` 与 `host/session-added` 帧携带派生的零事件位：客户端隐藏空白会话并按 workspace 复用它们，在首个 `host/session-status(running:true)` 时翻转 blank，并以 `session.list` 作为重连权威；冷会话摘要永远不是空白：惰性持久化让从未追加过事件的会话根本不出现在 `list()` 中。 diff --git a/packages/host/apiproxy/src/api-proxy.ts b/packages/host/apiproxy/src/api-proxy.ts index c34093edbc..a7f56df1f7 100644 --- a/packages/host/apiproxy/src/api-proxy.ts +++ b/packages/host/apiproxy/src/api-proxy.ts @@ -92,30 +92,29 @@ async function durablePromptContent(ctx: Context, content: readonly PromptConten throw new AttachmentError('Prompt exceeds the configured aggregate image-byte limit.', 'IMAGES_TOO_LARGE') } for (const image of images) { - ctx.attachments.validateImage({ + await ctx.attachments.validateImage({ data: image.data, mediaType: image.part.mediaType, ...image.part.name === undefined ? {} : { name: image.part.name }, }) } - return Promise.all(prepared.map(async (item): Promise => { - if (!('data' in item)) return { type: 'text', text: item.text } + const blocks: ContentBlock[] = [] + for (const item of prepared) { + if (!('data' in item)) { + blocks.push({ type: 'text', text: item.text }) + continue + } const attachment = await ctx.attachments.saveImage({ data: item.data, mediaType: item.part.mediaType, ...item.part.name === undefined ? {} : { name: item.part.name }, }) - return { type: 'image', attachment } - })) + blocks.push({ type: 'image', attachment }) + } + return blocks } -/** - * The ONE recursive block walk shared by attachment authorization and the - * model-selection gate (nested tool-result content included). Both consumers - * must agree on what counts as replayed image content — a route added to one - * walker but not the other would silently skip authorization or stranding - * protection — so there is exactly one walker, parameterized by match. - */ +/** Search durable event content for an image reference, including nested tool results. */ function imageBlockIn(content: unknown, match: (ref: ImageAttachmentRef) => boolean): ImageAttachmentRef | undefined { if (!Array.isArray(content)) return undefined for (const value of content) { @@ -148,18 +147,15 @@ function imageInEvent(event: SessionEvent, match: (ref: ImageAttachmentRef) => b return undefined } -/** True when any block (nested tool-result content included) is an image block. */ -function contentHasImage(content: unknown): boolean { - return imageBlockIn(content, () => true) !== undefined +/** True when typed model content contains an image, including nested tool results. */ +function contentHasImage(content: readonly ContentBlock[]): boolean { + return content.some(block => block.type === 'image' + || (block.type === 'tool-result' && contentHasImage(block.content))) } -/** - * True when the session log already carries image content on any route a - * model request replays (message content, wrapped messages, streamed blocks). - * The log is immutable, so a true here is permanent for the session's life. - */ -function sessionHasImage(events: readonly SessionEvent[]): boolean { - return events.some(event => imageInEvent(event, () => true) !== undefined) +/** True when the current model-visible surface contains an image. */ +function messagesHaveImage(messages: readonly { content: readonly ContentBlock[] }[]): boolean { + return messages.some(message => contentHasImage(message.content)) } function referencedImage(events: readonly SessionEvent[], attachmentId: string): ImageAttachmentRef | undefined { @@ -564,6 +560,14 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro const pendingQuestions = new Map() const pendingApprovals = new Map() const muxQueues = new Set>>() + const imageAdmissionChains = new WeakMap>() + + /** Serialize model selection with image prompt admission for one agent. */ + function serializeImageAdmission(agent: Agent, operation: () => Promise): Promise { + const result = (imageAdmissionChains.get(agent) ?? Promise.resolve()).then(operation) + imageAdmissionChains.set(agent, result.then(() => undefined, () => undefined)) + return result + } /** * Install or return the session-local target that prompt assembly snapshots. @@ -619,18 +623,26 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro * Per-session inbox occurrence mirror serving the mux-open queue snapshot * (the same refresh-recovery baseline as pending questions). Each terminal * inbox event retires one matching occurrence, so repeated sends of the same - * identified message remain visible until every occurrence is claimed. + * identified message remain visible until every occurrence is published or + * discarded. Dequeue is not publication: the log append follows it. */ - const queuedMirror = new Map() + const queuedMirror = new Map() ctx.effect(() => { - const retire = (agent: Agent, id: MessageId, placement?: InboxPlacement): void => { - const entries = queuedMirror.get(agent.id) + const retire = (sessionId: SessionId, id: MessageId, placement?: InboxPlacement): void => { + const entries = queuedMirror.get(sessionId) if (entries === undefined) return const index = entries.findIndex(entry => entry.message.id === id && (placement === undefined || entry.steering === (placement === 'steering'))) if (index !== -1) entries.splice(index, 1) - if (entries.length === 0) queuedMirror.delete(agent.id) + if (entries.length === 0) queuedMirror.delete(sessionId) + } + const retireClaimed = (sessionId: SessionId): void => { + const entries = queuedMirror.get(sessionId) + if (entries === undefined) return + const pending = entries.filter(entry => !entry.claimed) + if (pending.length === 0) queuedMirror.delete(sessionId) + else queuedMirror.set(sessionId, pending) } const disposers = [ ctx.on('agent/inbox/enqueue', (agent: Agent, message: UserMessage, placement) => { @@ -640,7 +652,7 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro queuedMirror.set(agent.id, entries) } const steering = placement === 'steering' - entries.push({ message, steering }) + entries.push({ message, steering, claimed: false }) broadcast({ type: 'session/queued', sessionId: agent.id, @@ -649,10 +661,26 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro }) }), ctx.on('agent/inbox/dequeue', (agent: Agent, message: UserMessage, placement) => { - retire(agent, message.id, placement) + // A later claim proves any earlier claimed item either published (and + // was retired by session/event) or its admission ended without one. + retireClaimed(agent.id) + const entry = queuedMirror.get(agent.id)?.find(candidate => + candidate.message.id === message.id + && candidate.steering === (placement === 'steering')) + if (entry !== undefined) entry.claimed = true + }), + ctx.on('session/event', (session: Session, event: SessionEvent) => { + if (event.type === 'user/message') { + retire(session.id, event.data.id, 'queued') + } else if (event.type === 'steering/message') { + retire(session.id, (event.data as { message: UserMessage }).message.id, 'steering') + } }), ctx.on('agent/inbox/discard', (agent: Agent, messages: UserMessage[]) => { - for (const message of messages) retire(agent, message.id) + for (const message of messages) retire(agent.id, message.id) + }), + ctx.on('agent/status', (agent: Agent, status: AgentStatus) => { + if (status === 'idle') retireClaimed(agent.id) }), ctx.on('session/disposed', (session: Session) => { queuedMirror.delete(session.id) @@ -1133,48 +1161,47 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro const { sessionId, provider, model, reasoningEffort } = request.payload const found = await agentFor(sessionId) if ('error' in found) return err(request, found.error) - try { - const resolved = await ctx.llm.resolveCallConfig({ - provider, - model, - ...reasoningEffort === undefined - ? {} - : { reasoningEffort: ReasoningEffortId(reasoningEffort) }, - }) - // An image-bearing log replays into every later request, and both - // wire routes reject image content on text-only models — accepting - // this selection would strand the session (every turn fails, no - // in-product recovery). Refuse at the selection boundary instead. - // The pending inbox counts too: a queued image prompt enters the log - // only when claimed, which would happen AFTER this switch landed. - const queuedImage = (queuedMirror.get(sessionId) ?? []) - .some(entry => contentHasImage(entry.message.content)) - if (queuedImage || sessionHasImage(found.agent.session.events)) { - const info = await ctx.llm.resolveModelInfo(resolved.provider, resolved.model) - if (info.inputModalities !== undefined && !info.inputModalities.includes('image')) { - return err(request, { - code: 'model-unavailable', - message: `Model "${resolved.model}" does not accept image input, but this session's history already contains images; select an image-capable model.`, - details: { provider, model }, - }) + return serializeImageAdmission(found.agent, async () => { + try { + const resolved = await ctx.llm.resolveCallConfig({ + provider, + model, + ...reasoningEffort === undefined + ? {} + : { reasoningEffort: ReasoningEffortId(reasoningEffort) }, + }) + // A current image-bearing surface replays into the next request, + // while a dequeued prompt remains pending until its message event + // publishes. Refuse a text-only route at this shared boundary. + const queuedImage = (queuedMirror.get(sessionId) ?? []) + .some(entry => contentHasImage(entry.message.content)) + if (queuedImage || messagesHaveImage(found.agent.session.deriveMessages())) { + const info = await ctx.llm.resolveModelInfo(resolved.provider, resolved.model) + if (info.inputModalities !== undefined && !info.inputModalities.includes('image')) { + return err(request, { + code: 'model-unavailable', + message: `Model "${resolved.model}" does not accept image input, but this session's history already contains images; select an image-capable model.`, + details: { provider, model }, + }) + } } + const selected: AgentLlmTarget = { + provider: resolved.provider, + model: resolved.model, + ...resolved.reasoningEffort === undefined + ? {} + : { reasoningEffort: resolved.reasoningEffort }, + } + targetFor(found.agent).current = selected + return ok(request, { selected: { ...selected } }) + } catch (error: unknown) { + return err(request, { + code: 'model-unavailable', + message: error instanceof Error ? error.message : String(error), + details: { provider, model }, + }) } - const selected: AgentLlmTarget = { - provider: resolved.provider, - model: resolved.model, - ...resolved.reasoningEffort === undefined - ? {} - : { reasoningEffort: resolved.reasoningEffort }, - } - targetFor(found.agent).current = selected - return ok(request, { selected: { ...selected } }) - } catch (error: unknown) { - return err(request, { - code: 'model-unavailable', - message: error instanceof Error ? error.message : String(error), - details: { provider, model }, - }) - } + }) }, async rename(request) { @@ -1214,36 +1241,40 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro const agent = found.agent // The rpcId rides MessageSource into user/message (merge declaration in api/sessions.ts; provisional correlation). const source: MessageSource = { kind: 'user', rpcId: request.rpcId } - try { - if (content.some(part => part.type === 'image')) { - const target = targetFor(agent).current - const provider = target.provider - const model = target.model - const modelInfo = await ctx.llm.resolveModelInfo(provider, model) - if (modelInfo.inputModalities !== undefined && !modelInfo.inputModalities.includes('image')) { + const hasImage = content.some(part => part.type === 'image') + const admit = async (): Promise> => { + try { + if (hasImage) { + const target = targetFor(agent).current + const provider = target.provider + const model = target.model + const modelInfo = await ctx.llm.resolveModelInfo(provider, model) + if (modelInfo.inputModalities !== undefined && !modelInfo.inputModalities.includes('image')) { + return err(request, { + code: 'attachment-error', + message: `Model "${model}" does not support image input.`, + details: { reason: 'MODEL_DOES_NOT_SUPPORT_IMAGES' }, + }) + } + } + const durable = await durablePromptContent(ctx, content) + const message: UserMessage = createUserMessage({ content: durable, source }) + if (mode === 'steer') agent.steer(message) + else agent.followup(message) + } catch (error: unknown) { + if (error instanceof AttachmentError) { return err(request, { code: 'attachment-error', - message: `Model "${model}" does not support image input.`, - details: { reason: 'MODEL_DOES_NOT_SUPPORT_IMAGES' }, + message: error.message, + details: { reason: error.code }, }) } + // A synchronous throw from steer/followup means disposed or invalid input; surface as agent-busy with the reason attached. + return err(request, { code: 'agent-busy', message: 'prompt rejected', details: { reason: String(error) } }) } - const durable = await durablePromptContent(ctx, content) - const message: UserMessage = createUserMessage({ content: durable, source }) - if (mode === 'steer') agent.steer(message) - else agent.followup(message) - } catch (error: unknown) { - if (error instanceof AttachmentError) { - return err(request, { - code: 'attachment-error', - message: error.message, - details: { reason: error.code }, - }) - } - // A synchronous throw from steer/followup means disposed or invalid input; surface as agent-busy with the reason attached. - return err(request, { code: 'agent-busy', message: 'prompt rejected', details: { reason: String(error) } }) + return ok(request, { accepted: true as const }) } - return ok(request, { accepted: true as const }) + return hasImage ? serializeImageAdmission(agent, admit) : admit() }, async attachment(request) { diff --git a/packages/host/apiproxy/tests/api-proxy-commands.spec.ts b/packages/host/apiproxy/tests/api-proxy-commands.spec.ts index ad70e6b26a..c409eff423 100644 --- a/packages/host/apiproxy/tests/api-proxy-commands.spec.ts +++ b/packages/host/apiproxy/tests/api-proxy-commands.spec.ts @@ -302,7 +302,7 @@ describe('session/queued frames', () => { expect(replayFrames.filter(f => f.type === 'session/queued')).toEqual(liveFrames) }) - it('retires mirror entries on their terminal dequeue', async () => { + it('retains each dequeued entry until its durable message publishes', async () => { const ctx = await harness() const api = createApiProxy(ctx, DEFAULTS) const agent = stubAgent(ctx) @@ -311,15 +311,50 @@ describe('session/queued frames', () => { ctx.emit('agent/inbox/enqueue', agent, queued, 'queued') ctx.emit('agent/inbox/enqueue', agent, steering, 'steering') ctx.emit('agent/inbox/dequeue', agent, queued, 'queued') - ctx.emit('agent/inbox/dequeue', agent, steering, 'steering') + const pendingAbort = new AbortController() + const pending = await collect( + api.events.mux({ rpcId: RpcId('t-mux-dequeued'), payload: {} }, pendingAbort.signal), 3, pendingAbort) + expect(pending.filter(f => f.type === 'session/queued')).toEqual([ + { type: 'session/queued', sessionId: agent.id, message: queued, steering: false }, + { type: 'session/queued', sessionId: agent.id, message: steering, steering: true }, + ]) + + agent.session.append('user/message', queued, { surfaceOp: 'append' }) + ctx.emit('agent/inbox/dequeue', agent, steering, 'steering') + agent.session.append('steering/message', { turn: 1, message: steering }, { surfaceOp: 'append' }) const abort = new AbortController() const frames = await collect( api.events.mux({ rpcId: RpcId('t-mux-after'), payload: {} }, abort.signal), 1, abort) expect(frames.filter(f => f.type === 'session/queued')).toHaveLength(0) }) - it('retires the matching placement when one message identity is queued and steering', async () => { + it('retires claimed entries whose admission ends without publication', async () => { + const ctx = await harness() + const api = createApiProxy(ctx, DEFAULTS) + const agent = stubAgent(ctx) + const rejected = inboxMessage('m-rejected', 'rejected') + const successor = inboxMessage('m-successor', 'successor') + ctx.emit('agent/inbox/enqueue', agent, rejected, 'queued') + ctx.emit('agent/inbox/dequeue', agent, rejected, 'queued') + ctx.emit('agent/inbox/enqueue', agent, successor, 'queued') + ctx.emit('agent/inbox/dequeue', agent, successor, 'queued') + + const pendingAbort = new AbortController() + const pending = await collect( + api.events.mux({ rpcId: RpcId('t-mux-rejected'), payload: {} }, pendingAbort.signal), 2, pendingAbort) + expect(pending.filter(f => f.type === 'session/queued')).toEqual([ + { type: 'session/queued', sessionId: agent.id, message: successor, steering: false }, + ]) + + ctx.emit('agent/status', agent, 'idle') + const idleAbort = new AbortController() + const idle = await collect( + api.events.mux({ rpcId: RpcId('t-mux-rejected-idle'), payload: {} }, idleAbort.signal), 1, idleAbort) + expect(idle.filter(f => f.type === 'session/queued')).toHaveLength(0) + }) + + it('retires the matching published placement when one message identity is queued and steering', async () => { const ctx = await harness() const api = createApiProxy(ctx, DEFAULTS) const agent = stubAgent(ctx) @@ -328,6 +363,7 @@ describe('session/queued frames', () => { ctx.emit('agent/inbox/enqueue', agent, repeated, 'steering') ctx.emit('agent/inbox/dequeue', agent, inboxMessage('unknown', 'not queued'), 'queued') ctx.emit('agent/inbox/dequeue', agent, repeated, 'steering') + agent.session.append('steering/message', { turn: 1, message: repeated }, { surfaceOp: 'append' }) const abort = new AbortController() const frames = await collect( diff --git a/packages/host/apiproxy/tests/api-proxy-models.spec.ts b/packages/host/apiproxy/tests/api-proxy-models.spec.ts index 10b3e96e47..450d9cbcf9 100644 --- a/packages/host/apiproxy/tests/api-proxy-models.spec.ts +++ b/packages/host/apiproxy/tests/api-proxy-models.spec.ts @@ -117,10 +117,26 @@ function expectValue(response: { result: { ok: true; value: T } | { ok: false return response.result.value } +function registerTextOnly(ctx: Context): void { + ctx.llm.registerAdapter(['text-only'], new class extends CatalogAdapter { + override resolveModel(provider: string, model: string): Promise { + return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text'] }) + } + }('Text Only', [])) +} + describe('Web session model selection', () => { it('accepts ordered multi-image prompts and rejects configured batch-limit excess before persistence', async () => { const { ctx, agent, sessionId } = await harness() - const validateImage = vi.fn((_input: { data: Uint8Array }): void => {}) + let secondValidationStarted!: () => void + let releaseSecondValidation!: () => void + const secondStarted = new Promise((resolve) => { secondValidationStarted = resolve }) + const secondReleased = new Promise((resolve) => { releaseSecondValidation = resolve }) + const validateImage = vi.fn(async (input: { data: Uint8Array }): Promise => { + if (input.data[0] !== 2) return + secondValidationStarted() + await secondReleased + }) const saveImage = vi.fn((input: { data: Uint8Array; mediaType: 'image/png'; name?: string }) => { return Promise.resolve({ attachmentId: `att-${String(input.data[0])}`, @@ -141,11 +157,15 @@ describe('Web session model selection', () => { const api = createApiProxy(ctx, { provider: 'deepseek', model: 'deepseek-chat', cwd: '/tmp', workspaceRoot: '/tmp' }) const first = { type: 'image' as const, mediaType: 'image/png' as const, data: 'AQ==', name: 'first.png' } const second = { type: 'image' as const, mediaType: 'image/png' as const, data: 'Ag==', name: 'second.png' } - const accepted = await api.sessions.prompt(request({ + const accepting = api.sessions.prompt(request({ sessionId, mode: 'queue' as const, content: [first, { type: 'text' as const, text: 'compare' }, second], })) + await secondStarted + expect(saveImage).not.toHaveBeenCalled() + releaseSecondValidation() + const accepted = await accepting expect(accepted.result).toMatchObject({ ok: true, value: { accepted: true } }) expect(validateImage.mock.calls.map(([input]) => [...input.data])).toEqual([[1], [2]]) expect(saveImage.mock.calls.map(([input]) => [...input.data])).toEqual([[1], [2]]) @@ -294,13 +314,9 @@ describe('Web session model selection', () => { await ctx.fiber.dispose() }) - it('refuses a text-only selection once the session log carries an image', async () => { + it('refuses a text-only selection while current derived history carries an image', async () => { const { ctx, sessionId, agent } = await harness() - ctx.llm.registerAdapter(['text-only'], new class extends CatalogAdapter { - override resolveModel(provider: string, model: string): Promise { - return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text'] }) - } - }('Text Only', [])) + registerTextOnly(ctx) ctx.llm.registerAdapter(['vision'], new class extends CatalogAdapter { override resolveModel(provider: string, model: string): Promise { return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text', 'image'] }) @@ -318,7 +334,7 @@ describe('Web session model selection', () => { content: [{ type: 'image', attachment: { attachmentId: 'att-1', mediaType: 'image/png', bytes: 8, width: 1, height: 1 } }], } as never, { surfaceOp: 'append' }) - // The log is immutable: a text-only route would fail every later turn. + // The image remains on the current request surface, so a text-only route would fail the next turn. const stranded = await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain', })) @@ -337,31 +353,87 @@ describe('Web session model selection', () => { await ctx.fiber.dispose() }) - it('refuses a text-only selection while an image prompt is still queued (not yet logged)', async () => { + it('keeps a dequeued image pending until publication, then follows the compacted surface', async () => { const { ctx, sessionId, agent } = await harness() - ctx.llm.registerAdapter(['text-only'], new class extends CatalogAdapter { - override resolveModel(provider: string, model: string): Promise { - return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text'] }) - } - }('Text Only', [])) + registerTextOnly(ctx) const api = createApiProxy(ctx, { provider: 'deepseek', model: 'deepseek-chat', cwd: '/tmp', workspaceRoot: '/tmp' }) - // The queued message enters the session log only when claimed — after a - // model switch would already have landed. The pending-inbox mirror must - // therefore gate the switch too. - ctx.emit('agent/inbox/enqueue', agent, { + const queued = { id: 'q-1', role: 'user', source: { kind: 'user' }, content: [{ type: 'image', attachment: { attachmentId: 'att-q', mediaType: 'image/png', bytes: 8, width: 1, height: 1 } }], - } as never, 'queued') - const stranded = await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain' })) - expect(stranded.result.ok).toBe(false) - // Claiming the message drains the mirror; the log now owns the decision. - ctx.emit('agent/inbox/dequeue', agent, { id: 'q-1' } as never, 'queued') + } as never + ctx.emit('agent/inbox/enqueue', agent, queued, 'queued') + expect((await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain' }))).result.ok).toBe(false) + + // Dequeue precedes the authoritative append, so it cannot open a switch window. + ctx.emit('agent/inbox/dequeue', agent, queued, 'queued') + expect((await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain' }))).result.ok).toBe(false) + + const imageEvent = agent.session.append('user/message', queued, { surfaceOp: 'append' }) + expect((await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain' }))).result.ok).toBe(false) + + // Publication retires the mirror; once compaction shadows the image, the + // current model-visible surface no longer requires an image-capable route. + agent.session.append('user/message', { + id: 'summary', role: 'user', source: { kind: 'plugin', plugin: 'compact' }, + content: [{ type: 'text', text: 'image summarized' }], + } as never, { + surfaceOp: { op: 'replace', start: imageEvent.seq, end: imageEvent.seq }, + sourceEventSeqs: [imageEvent.seq], + }) expect(expectValue(await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain', }))).selected).toEqual({ provider: 'text-only', model: 'plain' }) await ctx.fiber.dispose() }) + it('serializes an image save with a concurrent model selection', async () => { + const { ctx, sessionId, agent } = await harness() + registerTextOnly(ctx) + let saveStarted!: () => void + let releaseSave!: () => void + const started = new Promise((resolve) => { saveStarted = resolve }) + const released = new Promise((resolve) => { releaseSave = resolve }) + const ref = { attachmentId: 'att-race', mediaType: 'image/png' as const, bytes: 1, width: 1, height: 1 } + ctx.provide('attachments', { + imageLimits: { + maxImageBytes: 1, + maxImagesPerMessage: 1, + maxMessageImageBytes: 1, + maxImagePixels: 1, + mediaTypes: ['image/png'], + }, + validateImage: () => Promise.resolve(), + saveImage: async () => { + saveStarted() + await released + return ref + }, + } as never) + Object.assign(agent, { + followup(message: UserMessage) { + ctx.emit('agent/inbox/enqueue', agent, message, 'queued') + }, + }) + const api = createApiProxy(ctx, { provider: 'deepseek', model: 'deepseek-chat', cwd: '/tmp', workspaceRoot: '/tmp' }) + + const prompt = api.sessions.prompt(request({ + sessionId, + mode: 'queue' as const, + content: [{ type: 'image' as const, mediaType: 'image/png' as const, data: 'AA==' }], + })) + await started + const selection = api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain' })) + expect(await Promise.race([ + selection.then(() => 'settled' as const), + new Promise<'pending'>((resolve) => { setTimeout(() => { resolve('pending') }, 0) }), + ])).toBe('pending') + + releaseSave() + expect((await prompt).result.ok).toBe(true) + expect((await selection).result.ok).toBe(false) + await ctx.fiber.dispose() + }) + it('authorizes an attachment read referenced only from wrapped message content', async () => { const { ctx, sessionId, agent } = await harness() const ref = { attachmentId: 'att-w', mediaType: 'image/png' as const, bytes: 4, width: 1, height: 1 } @@ -369,9 +441,8 @@ describe('Web session model selection', () => { readImage: () => Promise.resolve({ ref, data: new Uint8Array([1, 2, 3, 4]) }), } as never) const api = createApiProxy(ctx, { provider: 'deepseek', model: 'deepseek-chat', cwd: '/tmp', workspaceRoot: '/tmp' }) - // The only reference lives inside an assistant/message wrapper — the same - // walk that gates model selection must authorize the read, or a real host - // denies galleries the fixture (with its own authorization mirror) serves. + // The only reference lives inside an assistant/message wrapper; the + // authorization walk must follow that durable event shape. agent.session.append('assistant/message', { turn: 1, step: 0, message: { id: 'a-1', role: 'assistant', source: { kind: 'model', provider: 'p', model: 'm' }, content: [{ type: 'image', attachment: ref }] }, @@ -383,7 +454,7 @@ describe('Web session model selection', () => { await ctx.fiber.dispose() }) - it('detects images on every replayed route: wrapped messages, streamed blocks, nested tool results', async () => { + it('detects images in wrapped messages and nested tool results on the current surface', async () => { const image = { type: 'image', attachment: { attachmentId: 'att-x', mediaType: 'image/png', bytes: 8, width: 1, height: 1 } } const cases: { label: string; append: (agent: Agent) => void }[] = [ { @@ -394,14 +465,6 @@ describe('Web session model selection', () => { } as never, { surfaceOp: 'append' }) }, }, - { - label: 'streamed assistant block', - append: (agent) => { - agent.session.append('assistant/chunk', { - turn: 1, step: 0, chunk: { type: 'block-end', index: 0, block: image }, - } as never) - }, - }, { label: 'nested tool-result content', append: (agent) => { @@ -414,11 +477,7 @@ describe('Web session model selection', () => { ] for (const { label, append } of cases) { const { ctx, sessionId, agent } = await harness() - ctx.llm.registerAdapter(['text-only'], new class extends CatalogAdapter { - override resolveModel(provider: string, model: string): Promise { - return Promise.resolve({ provider, id: model, name: model, inputModalities: ['text'] }) - } - }('Text Only', [])) + registerTextOnly(ctx) const api = createApiProxy(ctx, { provider: 'deepseek', model: 'deepseek-chat', cwd: '/tmp', workspaceRoot: '/tmp' }) append(agent) const stranded = await api.sessions.selectModel(request({ sessionId, provider: 'text-only', model: 'plain' })) diff --git a/packages/llm/llm-pi-ai/README.i18n.yaml b/packages/llm/llm-pi-ai/README.i18n.yaml index 3c6c7729c4..394e12a772 100644 --- a/packages/llm/llm-pi-ai/README.i18n.yaml +++ b/packages/llm/llm-pi-ai/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md -README.md: 3cd64b8170ac0f6b6c4316f19bb816c65c223935 -README.zh.md: 478ccb91df996aaf67bd952e95596f4ea65564bc +README.md: 885a2dcbc6fea3c21421d83202941f8251bc3c06 +README.zh.md: d5ea4b80bdc2e0655994667cbd099af7d20aefe9 diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index 3cd64b8170..885a2dcbc6 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -45,7 +45,7 @@ Supported profile fields are `provider`, `apiKey`, `baseURL`, `headers`, `reason The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`. -Image requests resolve the optional `ctx.attachments` service when the request is dispatched, so Cordis plugin load order does not freeze attachment availability. A visual request still fails explicitly with `UNSUPPORTED_CONTENT` when the service or the selected model's image capability is absent. +Image requests resolve the optional `ctx.attachments` service when the request is dispatched, so Cordis plugin load order does not freeze attachment availability. Image detection and conversion recurse through nested `tool-result` content, so a nested image is neither flattened nor skipped. A visual request still fails explicitly with `UNSUPPORTED_CONTENT` when the service or the selected model's image capability is absent. ## Provider/model routing and replay diff --git a/packages/llm/llm-pi-ai/README.zh.md b/packages/llm/llm-pi-ai/README.zh.md index 478ccb91df..d5ea4b80bd 100644 --- a/packages/llm/llm-pi-ai/README.zh.md +++ b/packages/llm/llm-pi-ai/README.zh.md @@ -45,7 +45,7 @@ 适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。 -图片请求会在请求分发时解析可选的 `ctx.attachments` 服务，因此 Cordis 插件加载顺序不会固化附件可用性。当该服务或所选模型的图片能力不存在时，视觉请求仍会明确以 `UNSUPPORTED_CONTENT` 失败。 +图片请求会在请求分发时解析可选的 `ctx.attachments` 服务，因此 Cordis 插件加载顺序不会固化附件可用性。图片检测与转换会递归遍历嵌套的 `tool-result` 内容，因此嵌套图片既不会被展平，也不会被跳过。当该服务或所选模型的图片能力不存在时，视觉请求仍会明确以 `UNSUPPORTED_CONTENT` 失败。 ## 提供方／模型路由与回放 diff --git a/packages/llm/llm-pi-ai/src/adapter.ts b/packages/llm/llm-pi-ai/src/adapter.ts index 2a4b08e607..3b5bb9b9cb 100644 --- a/packages/llm/llm-pi-ai/src/adapter.ts +++ b/packages/llm/llm-pi-ai/src/adapter.ts @@ -33,7 +33,7 @@ import type { import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout' import { resolveProfiles } from './config.ts' import type { PiAiProviderProfile, ResolvedPiAiProviderProfile } from './config.ts' -import { toPiContext } from './context.ts' +import { contentHasImage, toPiContext } from './context.ts' import { toStreamChunks } from './stream.ts' /** Constructor options for {@link PiAiAdapter}. */ @@ -192,8 +192,7 @@ export class PiAiAdapter extends LlmAdapter { const containsImage = options.messages.some((message) => { // The discriminant is part of same-process message validity and is read before content. void message.role - return message.content.some(block => block.type === 'image' - || (block.type === 'tool-result' && block.content.some(piece => piece.type === 'image'))) + return contentHasImage(message.content) }) if (containsImage && !model.input.includes('image')) { throw new LlmError(`pi-ai model "${model.id}" does not support image input`, 'UNSUPPORTED_CONTENT') diff --git a/packages/llm/llm-pi-ai/src/context.ts b/packages/llm/llm-pi-ai/src/context.ts index 90d2176b1c..b1464d7289 100644 --- a/packages/llm/llm-pi-ai/src/context.ts +++ b/packages/llm/llm-pi-ai/src/context.ts @@ -18,6 +18,23 @@ function flattenText(message: Message): string { .join('') } +/** + * Return whether content contains an image, including nested tool results. + * @param blocks - content to inspect recursively. + * @returns whether any nested block is an image. + */ +export function contentHasImage(blocks: readonly ContentBlock[]): boolean { + return blocks.some(block => block.type === 'image' + || (block.type === 'tool-result' && contentHasImage(block.content))) +} + +/** Flatten text recursively inside one tool result. */ +function toolResultText(blocks: readonly ContentBlock[]): string { + return blocks.map(block => block.type === 'text' + ? block.text + : block.type === 'tool-result' ? toolResultText(block.content) : '').join('') +} + async function userContent( blocks: readonly ContentBlock[], attachments: AttachmentStore, @@ -38,6 +55,14 @@ async function userContent( break } case 'tool-result': + { + const nested = await userContent(block.content, attachments) + if (typeof nested === 'string') { + if (nested.length > 0) content.push({ type: 'text', text: nested }) + } else { + content.push(...nested) + } + } break default: // Other merge-extensible blocks are not user-input vocabulary for pi-ai. @@ -72,8 +97,7 @@ function textOnlyContext(options: GenerateOptions): PiContext { const toolNames = new Map() const messages: PiMessage[] = [] for (const message of options.messages) { - if (message.content.some(block => block.type === 'image' - || (block.type === 'tool-result' && block.content.some(piece => piece.type === 'image')))) { + if (contentHasImage(message.content)) { throw new LlmError('pi-ai image conversion requires the durable attachment service', 'UNSUPPORTED_CONTENT') } if (message.role === 'system') { @@ -96,7 +120,7 @@ function textOnlyContext(options: GenerateOptions): PiContext { toolName: toolNames.get(result.toolCallId) ?? 'unknown', content: [{ type: 'text', - text: result.content.filter(block => block.type === 'text').map(block => block.text).join('') || '(no output)', + text: toolResultText(result.content) || '(no output)', }], isError: result.isError ?? false, timestamp: 0, @@ -131,7 +155,7 @@ async function toPiContextWithImages(options: GenerateOptions, attachments: Atta for (const message of options.messages) { if (message.role === 'system') { - if (message.content.some(block => block.type === 'image')) { + if (contentHasImage(message.content)) { throw new LlmError('pi-ai cannot represent an image in an in-history system message', 'UNSUPPORTED_CONTENT') } // pi-ai has a single systemPrompt slot; in-history system messages are diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index 272135bef9..cde449d34a 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -256,8 +256,8 @@ describe('PiAiAdapter provider routing', () => { mediaTypes: ['image/png'], } - validateImage(_input: SaveImageAttachment): void { - throw new Error('not used') + validateImage(_input: SaveImageAttachment): Promise { + return Promise.reject(new Error('not used')) } saveImage(_input: SaveImageAttachment): Promise { @@ -613,8 +613,12 @@ describe('provider profile lifecycle', () => { messages: [createUserMessage({ content: [{ type: 'tool-result', - toolCallId: 'call-image' as never, - content: [{ type: 'image', attachment: IMAGE_REF }], + toolCallId: 'call-outer' as never, + content: [{ + type: 'tool-result', + toolCallId: 'call-inner' as never, + content: [{ type: 'image', attachment: IMAGE_REF }], + }], }], source: { kind: 'plugin', plugin: 'test' }, })], diff --git a/packages/llm/llm-pi-ai/tests/convert.spec.ts b/packages/llm/llm-pi-ai/tests/convert.spec.ts index f57cacd13a..f6d55e39da 100644 --- a/packages/llm/llm-pi-ai/tests/convert.spec.ts +++ b/packages/llm/llm-pi-ai/tests/convert.spec.ts @@ -97,6 +97,55 @@ describe('toPiContext', () => { }) }) + it('flattens nested tool-result images into the enclosing result', async () => { + const attachment = { + attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), + mediaType: 'image/png' as const, + bytes: 3, + width: 1, + height: 1, + } + const readImage = vi.fn().mockResolvedValue({ ref: attachment, data: Uint8Array.of(1, 2, 3) }) + const context = await toPiContext({ + provider: 'openai', + model: 'gpt-4.1', + messages: [createUserMessage({ + content: [{ + type: 'tool-result', + toolCallId: CallId('outer'), + content: [ + { type: 'tool-result', toolCallId: CallId('empty'), content: [] }, + { type: 'text', text: 'before' }, + { type: 'tool-result', toolCallId: CallId('text'), content: [{ type: 'text', text: 'middle' }] }, + { + type: 'tool-result', + toolCallId: CallId('inner'), + content: [ + { type: 'image', attachment }, + { type: 'text', text: 'after' }, + ], + }, + ], + }], + source: { kind: 'plugin', plugin: 'test' }, + })], + }, { readImage } as unknown as AttachmentStore) + + expect(context.messages).toEqual([{ + role: 'toolResult', + toolCallId: 'outer', + toolName: 'unknown', + content: [ + { type: 'text', text: 'before' }, + { type: 'text', text: 'middle' }, + { type: 'image', data: 'AQID', mimeType: 'image/png' }, + { type: 'text', text: 'after' }, + ], + isError: false, + timestamp: 0, + }]) + }) + it('rejects structured image history when no durable resolver is supplied', () => { expect(() => toPiContext({ provider: 'openai', model: 'gpt-4.1', @@ -205,7 +254,15 @@ describe('toPiContext', () => { source: { kind: 'plugin', plugin: 'test' }, }), createUserMessage({ - content: [{ type: 'tool-result', toolCallId: CallId('c1'), content: [{ type: 'text', text: 'Sunny' }] }], + content: [{ + type: 'tool-result', + toolCallId: CallId('c1'), + content: [ + { type: 'text', text: 'Sunny' }, + { type: 'tool-result', toolCallId: CallId('nested'), content: [{ type: 'text', text: '!' }] }, + { type: 'chart', data: 'ignored' } as unknown as ContentBlock, + ], + }], source: { kind: 'plugin', plugin: 'test' }, }), ], @@ -214,7 +271,7 @@ describe('toPiContext', () => { role: 'toolResult', toolCallId: 'c1', toolName: 'get_weather', - content: [{ type: 'text', text: 'Sunny' }], + content: [{ type: 'text', text: 'Sunny!' }], isError: false, timestamp: 0, }) diff --git a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts index 43bd0833d4..fbc39c2dac 100644 --- a/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts +++ b/packages/llm/llm-pi-ai/tests/provider-apis.e2e.ts @@ -74,8 +74,8 @@ async function harness(image?: StoredImageAttachment): Promise { mediaTypes: [fixture.ref.mediaType], } - validateImage(_input: SaveImageAttachment): void { - throw new Error('e2e attachment fixture is read-only') + validateImage(_input: SaveImageAttachment): Promise { + return Promise.reject(new Error('e2e attachment fixture is read-only')) } saveImage(_input: SaveImageAttachment): Promise { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d1515b247c..c135d13219 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -708,6 +708,9 @@ importers: schemastery: specifier: ^3.18.0 version: 3.18.0 + sharp: + specifier: ^0.35.3 + version: 0.35.3(@types/node@22.20.0) devDependencies: '@deepseek-ai/dsh-attachment': specifier: workspace:^ @@ -1092,6 +1095,9 @@ importers: '@deepseek-ai/dsh-attachment': specifier: workspace:^ version: link:../../attachment/attachment + '@deepseek-ai/dsh-brand': + specifier: workspace:^ + version: link:../../util/brand '@deepseek-ai/dsh-client-locale': specifier: workspace:^ version: link:../locale @@ -6457,6 +6463,9 @@ packages: '@emnapi/runtime@1.11.0': resolution: {integrity: sha512-55coeOFKHv1ywEcUXJtWU5f+Jr/W5tZDvZig8DLKSwUN1JpROQ4rk/SNOQiFWmaR/VKF4zuFyW1B8JduOSv6Pg==} + '@emnapi/runtime@1.11.3': + resolution: {integrity: sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==} + '@emnapi/wasi-threads@1.2.1': resolution: {integrity: sha512-uTII7OYF+/Mes/MrcIOYp5yOtSMLBWSIoLPpcgwipoiKbli6k322tcoFsxoIIxPDqW01SQGAgko4EzZi2BNv2w==} @@ -6996,6 +7005,168 @@ packages: '@iconify/utils@3.1.3': resolution: {integrity: sha512-LPKOXPn/zV+zis1oOfGWogaXVpqUybF3ZS6SCZIsz8vg0ivVp9+fVqyYB7xq0aiST/VhUQYGO1qo6uoYSiEJqw==} + '@img/colour@1.1.0': + resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} + engines: {node: '>=18'} + + '@img/sharp-darwin-arm64@0.35.3': + resolution: {integrity: sha512-RMnFX7YQsMoh7lWfcM4NEHHymBX/rLuKNPVM84XE9ONPcaSCDgE7CHIHpSgPcO2xcRthgBy1HfNO319mwhIAkg==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [darwin] + + '@img/sharp-darwin-x64@0.35.3': + resolution: {integrity: sha512-Xo+5uFBtLN0BKqieTxiFzFPQAUlBbbH5iBKyRX/z1JrbnYsHTfKJnUfL8+p2TPXr1pXqao4eeL4Rl144uDpK9w==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [darwin] + + '@img/sharp-freebsd-wasm32@0.35.3': + resolution: {integrity: sha512-lUxcqWIj2wMQ9BrwNjngcr1gWUr5xgaGThBRqPPalIC2n67Cqj1uPh8NnA/ZhAg8hUbKl+kVHKwgUIwe6ZYPrg==} + engines: {node: '>=20.9.0'} + os: [freebsd] + + '@img/sharp-libvips-darwin-arm64@1.3.2': + resolution: {integrity: sha512-9J6ypZFpQBj4YnePGoq/S38w6nz+vqg5WZLrLGY4YuSemdMq47GMLBPO42MzwdGwpg/agZ7xzZcFHa48xlywfg==} + cpu: [arm64] + os: [darwin] + + '@img/sharp-libvips-darwin-x64@1.3.2': + resolution: {integrity: sha512-m2pW1n6cns9VaubNwsZ+c3CRYjxNQWgJ5gPlnL1nbBcpkBvFm6SCFN5o0psFHI8w9n11NKhFkeEDns98tiqbEw==} + cpu: [x64] + os: [darwin] + + '@img/sharp-libvips-linux-arm64@1.3.2': + resolution: {integrity: sha512-dqVSFynCox4C/J8kT16V7SIFAns0IjgLwkvYT7p8LQVmJ5OS5b6tI9IGflxTeuBS//zXeFIUbwt5dwxyZ17cnA==} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@img/sharp-libvips-linux-arm@1.3.2': + resolution: {integrity: sha512-1eMLzy92I4J6rmi4mAT8yC3HxOtniyGELlzGbNMLLeqe052ahFQ0h6LFq+lh5DsDIdYViIDst08abvSbcEdLXQ==} + cpu: [arm] + os: [linux] + libc: [glibc] + + '@img/sharp-libvips-linux-ppc64@1.3.2': + resolution: {integrity: sha512-3z0NHDxD6n5I9gc05U1eW1AyRm+Gznzq3naMrthPNqE6oYykcogW0l/jfpJdjYnuNl8R7yI9pNbE1XiUeyq0Aw==} + cpu: [ppc64] + os: [linux] + libc: [glibc] + + '@img/sharp-libvips-linux-riscv64@1.3.2': + resolution: {integrity: sha512-bsb4rI+NldGOsXuej2r8OdSS8+zXDVaCWxyWrcv6kneTOlgAHtZABRzBBCwdsPiD90J4myNJuHpg6kA20ImW/w==} + cpu: [riscv64] + os: [linux] + libc: [glibc] + + '@img/sharp-libvips-linux-s390x@1.3.2': + resolution: {integrity: sha512-/ABshyj8gCpyIrNXnHn4LorDJ0HHm1VhXPBlxZ8zAtfVPAaSafXPGn+sUSIRiwaSBy0mmFjSjiXI5mkcwdChKQ==} + cpu: [s390x] + os: [linux] + libc: [glibc] + + '@img/sharp-libvips-linux-x64@1.3.2': + resolution: {integrity: sha512-ITPEtgffGJ0S6G9dRyw/366tJQqFRcHWPHhC+Stpg3Z8AEMrDrTr2lhdz4f/Y/HMbRh//7Z5mBzEpVdi62Oc3w==} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@img/sharp-libvips-linuxmusl-arm64@1.3.2': + resolution: {integrity: sha512-zE9EdiUzUmg5mDT5a1rk5fYJ6GWPloTwWBYDS14naqHsL+EaMpDj1AWnpLgh3u0YCORv2Tt50wrcrpYqkP97Kw==} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@img/sharp-libvips-linuxmusl-x64@1.3.2': + resolution: {integrity: sha512-m0lrLiUt+lBYnCFr8qV/65yMR4E/c7/wf78I5eKTdkEakFAlZ9QlzEM3QIhhAwVeUhLAHLcCq7a7Vszq/oFNZQ==} + cpu: [x64] + os: [linux] + libc: [musl] + + '@img/sharp-linux-arm64@0.35.3': + resolution: {integrity: sha512-QgKDspHPnrU+GQ55XPhGwyhC8acLVOOSyAvo1oVfFmrIXLkDNmGWzAfDZ4xK8oSA1qBQrALcHX0G5UZni/SuFQ==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@img/sharp-linux-arm@0.35.3': + resolution: {integrity: sha512-affVWCTLooy8TSxbDx2qkzuDeaWLNVBA+P//FNBirHsXpP2fuBhk5AuboYUnrDnzoXes8GFjpTx0SBFOCRg+FA==} + engines: {node: '>=20.9.0'} + cpu: [arm] + os: [linux] + libc: [glibc] + + '@img/sharp-linux-ppc64@0.35.3': + resolution: {integrity: sha512-sMd8rDxmpLOwv/7N44klFjOD5DUO7FLdjiXDI0hoxYaf7Ar262dQIEkosE98bps+5HPLtp/EvNqeqQtOycP/IA==} + engines: {node: '>=20.9.0'} + cpu: [ppc64] + os: [linux] + libc: [glibc] + + '@img/sharp-linux-riscv64@0.35.3': + resolution: {integrity: sha512-0Eob78yjlYPfL5vMNWAW55l3R9Y6BQS/gOfe0ZcP9mEz9ohhKSt4im1hayiknXgf8AWrFqMvJcKIdmLmEe7yeQ==} + engines: {node: '>=20.9.0'} + cpu: [riscv64] + os: [linux] + libc: [glibc] + + '@img/sharp-linux-s390x@0.35.3': + resolution: {integrity: sha512-KgAxQ0DxpNOq1rG2t5cgTgShJFGSuU7XO45cqC+1NVOuZnP6tlgZRuSYOfNupGkHID0o3cJOsw4DVeJpMovcGw==} + engines: {node: '>=20.9.0'} + cpu: [s390x] + os: [linux] + libc: [glibc] + + '@img/sharp-linux-x64@0.35.3': + resolution: {integrity: sha512-8pqvxubL2PGdhlPy6GLqzDYMUjyRmKAwKHYKixpdJYBUK7PJ0C029XdsnpFIdgRZG68fZiGdHVWcKPvtiPB4cA==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@img/sharp-linuxmusl-arm64@0.35.3': + resolution: {integrity: sha512-Vz0iQjzzcSX3HCbfwFfCSG/9SCIqyO0mH2sXyiHaAYfBk0cRsCWXRyQYX0ovCK/PAQBbTzQ0dsPQHh5MAFL59w==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@img/sharp-linuxmusl-x64@0.35.3': + resolution: {integrity: sha512-6O1NPKcDVj9QEdg7Hx549EX8U0rp6yXQERqru6yRN7fGBn32UvIRJUlWnk+8xDCiG76hXVBbX82NZ/ZKr0euIg==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [linux] + libc: [musl] + + '@img/sharp-wasm32@0.35.3': + resolution: {integrity: sha512-cZ0XkcYGpHZkqW6iCkqTcmUC0CD9DhD5d/qeZlZkfRBn6GnHniZXLUo5+9xw8Iv76YE6LQFN9YNBlKREcCG76w==} + engines: {node: '>=20.9.0'} + + '@img/sharp-webcontainers-wasm32@0.35.3': + resolution: {integrity: sha512-2rnq7bX3NzeR2T4YWgz8qiG4h3TSdMe+vN1iQXpJleSJ3SM5zQ8Fy2SyyXAWlbxpEZ2Y+Z4u1BePgJEYbSy80Q==} + engines: {node: '>=20.9.0'} + cpu: [wasm32] + + '@img/sharp-win32-arm64@0.35.3': + resolution: {integrity: sha512-4bPwFdMbeC4JQ8L8LOyWp6nsHcboP5fxkp6iPOXz2Vg49R42TuMs2whkJ5OAP4/Ul035qOzy0AecOF9VOscn4w==} + engines: {node: '>=20.9.0'} + cpu: [arm64] + os: [win32] + + '@img/sharp-win32-ia32@0.35.3': + resolution: {integrity: sha512-r53mXsBN6lFUDiST764SvgwUdHAqM4rPAiDzAmf4fLoB6X/rkfyTrLCg6+g17wJJiCmB3JYgHuUldCWUIRFSXw==} + engines: {node: ^20.9.0} + cpu: [ia32] + os: [win32] + + '@img/sharp-win32-x64@0.35.3': + resolution: {integrity: sha512-D4y1vNeZrIIJCN+uHaWVtH86B+aCrdMYYjicy9pXHvbGZeGYLLSd3wdVuC37FxVXlU1ARsk84eKWfWMXGYEqvA==} + engines: {node: '>=20.9.0'} + cpu: [x64] + os: [win32] + '@isaacs/cliui@8.0.2': resolution: {integrity: sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA==} engines: {node: '>=12'} @@ -10420,6 +10591,11 @@ packages: engines: {node: '>=10'} hasBin: true + semver@7.8.5: + resolution: {integrity: sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==} + engines: {node: '>=10'} + hasBin: true + send@1.2.1: resolution: {integrity: sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==} engines: {node: '>= 18'} @@ -10434,6 +10610,15 @@ packages: setprototypeof@1.2.0: resolution: {integrity: sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==} + sharp@0.35.3: + resolution: {integrity: sha512-ej0zVHuZGHCiABXcNxeYhpRnPNPAcvbG8RMdBAhDAxLKkCRVSpK3Iyu7qbqw3JMzoj0REeM6f3tJLtVwl0023Q==} + engines: {node: '>=20.9.0'} + peerDependencies: + '@types/node': '*' + peerDependenciesMeta: + '@types/node': + optional: true + shebang-command@2.0.0: resolution: {integrity: sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==} engines: {node: '>=8'} @@ -11767,6 +11952,11 @@ snapshots: tslib: 2.8.1 optional: true + '@emnapi/runtime@1.11.3': + dependencies: + tslib: 2.8.1 + optional: true + '@emnapi/wasi-threads@1.2.1': dependencies: tslib: 2.8.1 @@ -12079,6 +12269,112 @@ snapshots: '@iconify/types': 2.0.0 import-meta-resolve: 4.2.0 + '@img/colour@1.1.0': {} + + '@img/sharp-darwin-arm64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-darwin-arm64': 1.3.2 + optional: true + + '@img/sharp-darwin-x64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-darwin-x64': 1.3.2 + optional: true + + '@img/sharp-freebsd-wasm32@0.35.3': + dependencies: + '@img/sharp-wasm32': 0.35.3 + optional: true + + '@img/sharp-libvips-darwin-arm64@1.3.2': + optional: true + + '@img/sharp-libvips-darwin-x64@1.3.2': + optional: true + + '@img/sharp-libvips-linux-arm64@1.3.2': + optional: true + + '@img/sharp-libvips-linux-arm@1.3.2': + optional: true + + '@img/sharp-libvips-linux-ppc64@1.3.2': + optional: true + + '@img/sharp-libvips-linux-riscv64@1.3.2': + optional: true + + '@img/sharp-libvips-linux-s390x@1.3.2': + optional: true + + '@img/sharp-libvips-linux-x64@1.3.2': + optional: true + + '@img/sharp-libvips-linuxmusl-arm64@1.3.2': + optional: true + + '@img/sharp-libvips-linuxmusl-x64@1.3.2': + optional: true + + '@img/sharp-linux-arm64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linux-arm64': 1.3.2 + optional: true + + '@img/sharp-linux-arm@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linux-arm': 1.3.2 + optional: true + + '@img/sharp-linux-ppc64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linux-ppc64': 1.3.2 + optional: true + + '@img/sharp-linux-riscv64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linux-riscv64': 1.3.2 + optional: true + + '@img/sharp-linux-s390x@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linux-s390x': 1.3.2 + optional: true + + '@img/sharp-linux-x64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linux-x64': 1.3.2 + optional: true + + '@img/sharp-linuxmusl-arm64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linuxmusl-arm64': 1.3.2 + optional: true + + '@img/sharp-linuxmusl-x64@0.35.3': + optionalDependencies: + '@img/sharp-libvips-linuxmusl-x64': 1.3.2 + optional: true + + '@img/sharp-wasm32@0.35.3': + dependencies: + '@emnapi/runtime': 1.11.3 + optional: true + + '@img/sharp-webcontainers-wasm32@0.35.3': + dependencies: + '@img/sharp-wasm32': 0.35.3 + optional: true + + '@img/sharp-win32-arm64@0.35.3': + optional: true + + '@img/sharp-win32-ia32@0.35.3': + optional: true + + '@img/sharp-win32-x64@0.35.3': + optional: true + '@isaacs/cliui@8.0.2': dependencies: string-width: 5.1.2 @@ -15822,6 +16118,8 @@ snapshots: semver@7.8.4: {} + semver@7.8.5: {} + send@1.2.1: dependencies: debug: 4.4.3 @@ -15851,6 +16149,39 @@ snapshots: setprototypeof@1.2.0: {} + sharp@0.35.3(@types/node@22.20.0): + dependencies: + '@img/colour': 1.1.0 + detect-libc: 2.1.2 + semver: 7.8.5 + optionalDependencies: + '@img/sharp-darwin-arm64': 0.35.3 + '@img/sharp-darwin-x64': 0.35.3 + '@img/sharp-freebsd-wasm32': 0.35.3 + '@img/sharp-libvips-darwin-arm64': 1.3.2 + '@img/sharp-libvips-darwin-x64': 1.3.2 + '@img/sharp-libvips-linux-arm': 1.3.2 + '@img/sharp-libvips-linux-arm64': 1.3.2 + '@img/sharp-libvips-linux-ppc64': 1.3.2 + '@img/sharp-libvips-linux-riscv64': 1.3.2 + '@img/sharp-libvips-linux-s390x': 1.3.2 + '@img/sharp-libvips-linux-x64': 1.3.2 + '@img/sharp-libvips-linuxmusl-arm64': 1.3.2 + '@img/sharp-libvips-linuxmusl-x64': 1.3.2 + '@img/sharp-linux-arm': 0.35.3 + '@img/sharp-linux-arm64': 0.35.3 + '@img/sharp-linux-ppc64': 0.35.3 + '@img/sharp-linux-riscv64': 0.35.3 + '@img/sharp-linux-s390x': 0.35.3 + '@img/sharp-linux-x64': 0.35.3 + '@img/sharp-linuxmusl-arm64': 0.35.3 + '@img/sharp-linuxmusl-x64': 0.35.3 + '@img/sharp-webcontainers-wasm32': 0.35.3 + '@img/sharp-win32-arm64': 0.35.3 + '@img/sharp-win32-ia32': 0.35.3 + '@img/sharp-win32-x64': 0.35.3 + '@types/node': 22.20.0 + shebang-command@2.0.0: dependencies: shebang-regex: 3.0.0 diff --git a/scripts/test-invariants.ts b/scripts/test-invariants.ts index c843f94a5a..2ee623c606 100644 --- a/scripts/test-invariants.ts +++ b/scripts/test-invariants.ts @@ -96,8 +96,8 @@ class TestAttachmentStore extends AttachmentStore { mediaTypes: ['image/png'], } - validateImage(_input: SaveImageAttachment): void { - throw new Error('test invariant attachment store does not validate images') + validateImage(_input: SaveImageAttachment): Promise { + return Promise.reject(new Error('test invariant attachment store does not validate images')) } saveImage(_input: SaveImageAttachment): Promise {