fix: preserve multi-image prompt batches
This commit is contained in:
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: ab1bd449f065346bc327984eacc85f71d4fd5a28
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: d51f9135352ba09b2cd2a083ade6b62f4a276216
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 134e3cf6137abec41398d7f4ded9b838e197a5fc
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: f0c16d10c57a499b605fffead9f35eb2f6149004
|
||||
+2
-2
@@ -112,7 +112,7 @@ type PromptInputPart =
|
||||
}
|
||||
```
|
||||
|
||||
Base64 crosses JSON-RPC once and is discarded after persistence. Version one accepts at most one image per prompt. The host validates canonical base64, individual bytes, magic-byte MIME, intrinsic dimensions, and intrinsic pixel count while durably saving that image. Only after the save succeeds does it call the agent with normalized text and a durable image block. A failure appends no user event and exposes no attachment path or raw bytes.
|
||||
Base64 crosses JSON-RPC once and is discarded after persistence. The host validates canonical base64, image count, aggregate bytes, individual bytes, magic-byte MIME, intrinsic dimensions, and decoded-pixel count: it validates the complete batch through the seam's storage-free `validateImage` before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Only after every image succeeds does it call the agent with normalized text and durable image blocks in the submitted order. A failure appends no user event and exposes no attachment path or raw bytes.
|
||||
|
||||
`session.attachment` is a read-only, session-scoped endpoint. The host serves bytes only when a durable event in that session references the requested attachment identifier. The client deduplicates loads by session and attachment identifier while that session is rendered, revokes resolved URLs on rendered-session disposal, and invalidates late loads so an unmounted session cannot repopulate the cache.
|
||||
|
||||
@@ -138,7 +138,7 @@ Composer thumbnails and each `MessageImage` own ephemeral original-preview state
|
||||
|
||||
### Limits and trust boundaries
|
||||
|
||||
Version one accepts at most one PNG, JPEG, WebP, or GIF per prompt. SVG and remote URLs are excluded. Default deployment limits are 5 MiB and 40 million intrinsic pixels per image; they are validated backend configuration and projected to the client for fast-path guidance, while host validation remains authoritative. The client connection carrier independently caps buffered API request bodies from the single-image byte limit plus base64 and envelope expansion; a body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
Version one accepts up to 10 ordered PNG, JPEG, WebP, or GIF images per prompt by default. SVG and remote URLs are excluded. Default deployment limits are 5 MiB and 40 million intrinsic pixels per image plus 20 MiB aggregate image bytes per prompt; all are validated backend configuration and projected to the client for fast-path guidance, while host validation remains authoritative. The client connection carrier independently caps buffered API request bodies from the aggregate image-byte limit plus base64 and envelope expansion; a body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
|
||||
Malformed base64, unsupported or mismatched media, truncated headers, excess bytes, excess image count, excess pixels, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser.
|
||||
|
||||
|
||||
+2
-2
@@ -112,7 +112,7 @@ type PromptInputPart =
|
||||
}
|
||||
```
|
||||
|
||||
Base64 只跨越一次 JSON-RPC,并在持久化后丢弃。第一版每条提示词最多接受一张图片。宿主在持久保存该图片时,会校验规范 base64、单张图片字节数、与魔数字节一致的 MIME、固有尺寸和固有像素数。只有保存成功后,宿主才会用规范化文本和一个持久图片块调用 agent。失败时不追加用户事件,也不公开任何附件路径或原始字节。
|
||||
Base64 只跨越一次 JSON-RPC,并在持久化后丢弃。宿主会校验规范 base64、图片数量、总字节数、单张图片字节数、与魔数字节一致的 MIME、固有尺寸和解码像素数:它会在保存任何成员之前,通过服务边界上不触碰存储的 `validateImage` 校验完整批次,因此一张畸形图片不会把批次中的有效成员留成无引用对象。只有每张图片都成功后,宿主才会用规范化文本和按提交顺序排列的持久图片块调用 agent。失败时不追加用户事件,也不公开任何附件路径或原始字节。
|
||||
|
||||
`session.attachment` 是只读且限定于会话作用域的端点。只有该会话中的持久事件引用了所请求的附件标识符,宿主才提供字节。会话处于渲染状态时,客户端会按会话和附件标识符对加载操作去重;已渲染会话释放时会撤销已解析的 URL,并使延迟完成的加载失效,以免已卸载的会话重新写入缓存。
|
||||
|
||||
@@ -138,7 +138,7 @@ Pi-AI 适配器是首条视觉输入路径:它在请求时解析 `ctx.attachme
|
||||
|
||||
### 限制与信任边界
|
||||
|
||||
第一版每条提示词最多接受一张 PNG、JPEG、WebP 或 GIF 图片。不接受 SVG 和远程 URL。默认部署限制为每张图片 5 MiB 和 4,000 万个固有像素;这些限制属于经过校验的后端配置,并会投影给客户端以提供快速路径指引,而宿主校验仍是权威结果。客户端连接载体根据单张图片字节上限加上 base64 和请求封装的膨胀量,独立限制 API 请求体的缓冲大小;未声明长度的请求体在越过上限的瞬间即被拒绝,而不是先读完再拒。
|
||||
第一版默认每条提示词最多接受 10 张有序的 PNG、JPEG、WebP 或 GIF 图片。不接受 SVG 和远程 URL。默认部署限制为每张图片 5 MiB 和 4,000 万个固有像素,外加每条提示词 20 MiB 图片总字节数;这些限制都属于经过校验的后端配置,并会投影给客户端以提供快速路径指引,而宿主校验仍是权威结果。客户端连接载体根据图片总字节数上限加上 base64 和请求封装的膨胀量,独立限制 API 请求体的缓冲大小;未声明长度的请求体在越过上限的瞬间即被拒绝,而不是先读完再拒。
|
||||
|
||||
格式错误的 base64、不支持或不匹配的媒体、截断的文件头、超出字节限制、超出图片数量、超出像素限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段,控制字符会被移除,并且任何本地路径都不会写入日志或返回浏览器。
|
||||
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
# Agent Note: Narrow Web image input version one
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-29-narrow-web-image-input-v1.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The first durable Web image-input slice also introduced speculative surfaces for multiple-image transactions, arbitrary CLI provider mounting, output-modality discovery, alternative text, and provider-neutral visual token pricing. None was required to paste or drop one image, persist it before its message event, replay it through a visual adapter, or render it from authorized history. Keeping those surfaces would turn unchosen future behavior into public contracts and make the initial capability harder to review and maintain.
|
||||
|
||||
## Decision
|
||||
|
||||
Version one accepts at most one image in a submitted prompt. The client gives immediate feedback, the host enforces the invariant, and the attachment seam validates and commits that one object. Deployment configuration retains per-image byte and pixel limits; request buffering derives from the per-image byte limit. There is no batch prevalidation, aggregate byte limit, image-count setting, transaction layer, or rollback protocol.
|
||||
|
||||
The CLI only patches the selected provider and model. The boot composition must already register the route, as it does for the shipped DeepSeek, OpenAI, and Anthropic routes; the CLI does not inspect the yml provider roster or dynamically mount an adapter.
|
||||
|
||||
Exact-model metadata carries only the input modalities that current admission decisions consume. `ImageBlock` carries the durable attachment reference; its optional display name supplies accessible UI text, so the core block has no separate alternative-text field. Provider-neutral token estimation does not apply one provider's visual pricing formula to other routes.
|
||||
|
||||
The attachment seam exposes its limits plus `saveImage` and `readImage`. The host depends on that seam rather than implementation re-exports. Browser draft and historical-image operations remain concrete conversation-plugin internals; the public `IConversation` face contains only the input registry and the scoped send, cancel, and history verbs used across package boundaries.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep multiple images and add transaction or rollback machinery.** Without garbage collection, a partially persisted batch needs an ownership or reclamation design. One image satisfies the initial user path without creating that lifecycle.
|
||||
|
||||
**Keep future-facing fields and methods as placeholders.** Output modalities, block alternative text, batch validation, and active-model handshake data had no current decision consumer. Adding them later with their first consumer preserves freedom to choose the correct contract.
|
||||
|
||||
**Estimate every image with one tile formula.** Visual pricing varies by provider, model, detail mode, and preprocessing. A hard-coded provider-neutral estimate would look authoritative while being wrong; provider usage is the authoritative accounting source.
|
||||
|
||||
**Mount any CLI-selected provider dynamically.** Configuration already owns plugin composition and credentials. Making selection also mutate composition duplicates that responsibility and requires parsing the config tree outside the loader.
|
||||
|
||||
## Consequences
|
||||
|
||||
The initial feature has fewer public fields, lifecycle operations, configuration knobs, and route-assembly branches. A prompt needing multiple images is rejected and must wait for an explicit multi-image persistence design. A provider absent from the composition cannot be selected solely with CLI flags. Pre-request token pressure may undercount visual input until a provider-aware estimator is designed, while reported usage remains exact.
|
||||
|
||||
Reintroducing any removed surface requires a concrete consumer and its failure, lifecycle, replay, and testing contract rather than compatibility with this pre-release shape.
|
||||
@@ -1,35 +0,0 @@
|
||||
# Agent Note: 收窄 Web 图片输入第一版
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-29-narrow-web-image-input-v1.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
首个持久化 Web 图片输入切片还引入了多张图片事务、由 CLI(命令行界面)挂载任意提供方、输出模态发现、替代文本以及与提供方无关的视觉 token 定价等推测性表面。要实现粘贴或拖放一张图片、在其消息事件之前将其持久化、通过视觉适配器回放它,或从经授权的历史记录中渲染它,并不需要上述任何表面。保留这些表面会把尚未选择的未来行为变成公共契约,并使初始能力更难评审和维护。
|
||||
|
||||
## 决策
|
||||
|
||||
第一版在每条提交的提示词中最多接受一张图片。客户端立即提供反馈,宿主强制执行该不变量,附件服务边界则校验并提交这一个对象。部署配置保留单张图片的字节和像素限制;请求缓冲上限由单张图片的字节限制派生。不设批量预校验、总字节限制、图片数量设置、事务层或回滚协议。
|
||||
|
||||
CLI 只修改选定的提供方和模型。启动组合必须已经注册该路由,随项目交付的 DeepSeek、OpenAI 和 Anthropic 路由正是如此;CLI 不会检查 yml 提供方清单,也不会动态挂载适配器。
|
||||
|
||||
确切模型元数据只携带当前准入决策会消费的输入模态。`ImageBlock` 携带持久附件引用;其可选显示名称提供无障碍 UI 文本,因此核心块没有单独的替代文本字段。与提供方无关的 token 估算不会把某一提供方的视觉定价公式应用于其他路由。
|
||||
|
||||
附件服务边界公开其限制以及 `saveImage` 和 `readImage`。宿主依赖该服务边界,而不是实现层重新导出的内容。浏览器草稿和历史图片操作仍是具体会话插件的内部实现;公开的 `IConversation` 表面只包含输入注册表,以及跨包边界使用的按作用域发送、取消和历史记录操作。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**保留多张图片,并添加事务或回滚机制。** 没有垃圾回收时,部分持久化的批次需要一套所有权或回收设计。一张图片即可满足首个用户路径,而不引入该生命周期。
|
||||
|
||||
**将面向未来的字段和方法保留为占位符。** 输出模态、块替代文本、批量校验和活跃模型握手数据目前都没有决策消费方。等到第一个消费方出现时再加入这些内容,可以保留选择正确契约的自由。
|
||||
|
||||
**使用一种图块公式估算每张图片。** 视觉定价因提供方、模型、细节模式和预处理而异。一项硬编码且与提供方无关的估算会看似权威,实际却是错误的;提供方用量才是权威核算来源。
|
||||
|
||||
**动态挂载 CLI 选择的任意提供方。** 配置已经负责插件组合和凭据。让选择操作同时改变组合会造成职责重复,并要求在加载器之外解析配置树。
|
||||
|
||||
## 后果
|
||||
|
||||
初始功能具有更少的公开字段、生命周期操作、配置项和路由组装分支。需要多张图片的提示词会被拒绝,必须等待明确的多张图片持久化设计。未加入组合的提供方无法仅凭 CLI 标志选择。在设计出提供方感知型估算器之前,请求前的 token 压力计算可能少计视觉输入,而上报的用量仍保持精确。
|
||||
|
||||
重新引入任何已移除表面时,都必须有具体消费方,并为其定义失败、生命周期、回放和测试契约,而不是为了兼容这一预发布形态。
|
||||
+3
-3
@@ -1,6 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-29-narrow-web-image-input-v1.md
|
||||
2026-07-29-narrow-web-image-input-v1.md: 5a42c7acea778f8d1ff00e30bde0f849de12eb6b
|
||||
2026-07-29-narrow-web-image-input-v1.zh.md: 0459074d0145c0d43008d32196ac6577eca958f3
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-29-simplify-web-image-input-v1.md
|
||||
2026-07-29-simplify-web-image-input-v1.md: 1262f1f42ef0b338c0aa7d29e479d4a599cc17a4
|
||||
2026-07-29-simplify-web-image-input-v1.zh.md: 60d9c47be1ebd9f259b44aaa6658e3582dfef2d4
|
||||
@@ -0,0 +1,37 @@
|
||||
# Agent Note: Simplify Web image input version one
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-29-simplify-web-image-input-v1.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The first durable Web image-input slice introduced required ordered multi-image intake alongside speculative surfaces for arbitrary CLI provider mounting, output-modality discovery, alternative text, provider-neutral visual token pricing, and browser lifecycle APIs with no cross-package consumer. Keeping the speculative surfaces would turn unchosen future behavior into public contracts and make the initial capability harder to review and maintain.
|
||||
|
||||
## Decision
|
||||
|
||||
Version one accepts ordered image batches bounded by configurable per-message count and aggregate-byte limits plus per-image byte and pixel limits. The client gives immediate feedback, while the host authoritatively decodes the complete batch, checks its bounds, validates every image without storage writes, and only then saves every image while preserving submitted order in the resulting durable blocks. Request buffering derives from the aggregate image-byte limit. This preserves validation atomicity without adding a batch transaction or rollback protocol.
|
||||
|
||||
The CLI only patches the selected provider and model. The boot composition must already register the route, as it does for the shipped DeepSeek, OpenAI, and Anthropic routes; the CLI does not inspect the yml provider roster or dynamically mount an adapter.
|
||||
|
||||
Exact-model metadata carries only the input modalities that current admission decisions consume. `ImageBlock` carries the durable attachment reference; its optional display name supplies accessible UI text, so the core block has no separate alternative-text field. Provider-neutral token estimation does not apply one provider's visual pricing formula to other routes.
|
||||
|
||||
The attachment seam exposes its limits plus storage-free `validateImage`, `saveImage`, and `readImage`. The host depends on that seam rather than implementation re-exports. Browser draft and historical-image operations remain concrete conversation-plugin internals; the public `IConversation` face contains only the input registry and the scoped send, cancel, and history verbs used across package boundaries.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Accept only one image.** Comparing or combining several images is a current product requirement. Count and aggregate-byte bounds keep that path finite without reducing it to a single image.
|
||||
|
||||
**Add a storage transaction or rollback protocol.** Storage-free validation prevents malformed later members from leaving earlier valid members unreferenced. Stronger all-or-nothing storage across independent content-addressed objects would require ownership or reclamation semantics that the current product path does not need.
|
||||
|
||||
**Keep future-facing fields and methods as placeholders.** Output modalities, block alternative text, and active-model handshake data had no current decision consumer. Adding them later with their first consumer preserves freedom to choose the correct contract.
|
||||
|
||||
**Estimate every image with one tile formula.** Visual pricing varies by provider, model, detail mode, and preprocessing. A hard-coded provider-neutral estimate would look authoritative while being wrong; provider usage is the authoritative accounting source.
|
||||
|
||||
**Mount any CLI-selected provider dynamically.** Configuration already owns plugin composition and credentials. Making selection also mutate composition duplicates that responsibility and requires parsing the config tree outside the loader.
|
||||
|
||||
## Consequences
|
||||
|
||||
The feature retains the two batch limits and one storage-free validation method required by multi-image prompts, while removing unrelated public fields, lifecycle operations, and route-assembly branches. A provider absent from the composition cannot be selected solely with CLI flags. Pre-request token pressure may undercount visual input until a provider-aware estimator is designed, while reported usage remains exact.
|
||||
|
||||
Reintroducing any removed surface requires a concrete consumer and its failure, lifecycle, replay, and testing contract rather than compatibility with this pre-release shape.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Agent Note: 简化 Web 图片输入第一版
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-29-simplify-web-image-input-v1.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
首个持久化 Web 图片输入切片在引入按顺序接收多张图片的必需能力时,也引入了由 CLI(命令行界面)挂载任意提供方、输出模态发现、替代文本、提供方无关的视觉 token 定价,以及没有跨包(package)消费方的浏览器生命周期 API 等推测性表面。保留这些推测性表面会把尚未选择的未来行为变成公共契约,并使初始能力更难评审和维护。
|
||||
|
||||
## 决策
|
||||
|
||||
第一版接受有序图片批次,并以可配置的每条消息图片数量与图片总字节数上限,以及单张图片字节数与像素上限约束批次。客户端立即提供反馈,而宿主会以权威方式解码完整批次、检查各项上限、在不写入存储的情况下校验每张图片,之后才保存每张图片,并在生成的持久图片块中保留提交顺序。请求缓冲上限由图片总字节数上限派生。这样无需添加批次事务或回滚协议,就能保持校验的原子性。
|
||||
|
||||
CLI 只修改选定的提供方和模型。启动组合必须已经注册该路由,随项目交付的 DeepSeek、OpenAI 和 Anthropic 路由正是如此;CLI 不会检查 yml 提供方清单,也不会动态挂载适配器。
|
||||
|
||||
确切模型元数据只携带当前准入决策会消费的输入模态。`ImageBlock` 携带持久附件引用;其可选显示名称提供无障碍 UI 文本,因此核心块没有单独的替代文本字段。与提供方无关的 token 估算不会把某一提供方的视觉定价公式应用于其他路由。
|
||||
|
||||
附件服务边界公开其限制、不触碰存储的 `validateImage`,以及 `saveImage` 和 `readImage`。宿主依赖该服务边界,而不是实现层重新导出的内容。浏览器草稿和历史图片操作仍是具体会话插件的内部实现;公开的 `IConversation` 表面只包含输入注册表,以及跨包边界使用的按作用域发送、取消和历史记录操作。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**只接受一张图片。** 比较或组合多张图片是当前产品要求。图片数量和总字节数上限使这条路径保持有界,而无需将其缩减为单张图片。
|
||||
|
||||
**添加存储事务或回滚协议。** 不触碰存储的校验能防止后面的畸形成员使前面有效的成员成为无引用对象。若要在独立的内容寻址对象之间实现更强的全有或全无存储保证,就需要当前产品路径并不需要的所有权或回收语义。
|
||||
|
||||
**将面向未来的字段和方法保留为占位符。** 输出模态、块替代文本和活跃模型握手数据目前都没有决策消费方。等到第一个消费方出现时再加入这些内容,可以保留选择正确契约的自由。
|
||||
|
||||
**使用一种图块公式估算每张图片。** 视觉定价因提供方、模型、细节模式和预处理而异。一项硬编码且与提供方无关的估算会看似权威,实际却是错误的;提供方用量才是权威核算来源。
|
||||
|
||||
**动态挂载 CLI 选择的任意提供方。** 配置已经负责插件组合和凭据。让选择操作同时改变组合会造成职责重复,并要求在加载器之外解析配置树。
|
||||
|
||||
## 后果
|
||||
|
||||
该功能保留了多图片提示词所需的两个批次上限和一个不触碰存储的校验方法,同时移除了无关的公开字段、生命周期操作和路由组装分支。未加入组合的提供方无法仅凭 CLI 标志选择。在设计出提供方感知型估算器之前,请求前的 token 压力计算可能少计视觉输入,而上报的用量仍保持精确。
|
||||
|
||||
重新引入任何已移除表面时,都必须有具体消费方,并为其定义失败、生命周期、回放和测试契约,而不是为了兼容这一预发布形态。
|
||||
@@ -5,8 +5,7 @@
|
||||
// user message and an assistant message, and pins the product surfaces: the
|
||||
// history ImageGallery loading real fixture bytes through the authorized
|
||||
// sessions.attachment route, the double-click ImageLightbox, and the composer
|
||||
// intake chain (paste → thumbnail rail → one-image limit → image-only send
|
||||
// enablement → remove).
|
||||
// intake chain (paste → ordered thumbnail rail → image-only send enablement → remove).
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
import { act, cleanup, fireEvent, screen, waitFor, within } from '@testing-library/react'
|
||||
@@ -162,7 +161,7 @@ it('renders the history image pair through the authorized attachment route and o
|
||||
})
|
||||
})
|
||||
|
||||
it('accepts a pasted image into the composer rail and removes it', async () => {
|
||||
it('accepts pasted images into the composer rail in order and removes them', async () => {
|
||||
boot('?fixture=empty')
|
||||
|
||||
await screen.findByPlaceholderText('Choose a workspace to start', {}, { timeout: 10_000 })
|
||||
@@ -211,12 +210,14 @@ it('accepts a pasted image into the composer rail and removes it', async () => {
|
||||
getData: () => '',
|
||||
},
|
||||
})
|
||||
expect(await screen.findByText('每条消息最多添加 1 张图片')).toBeTruthy()
|
||||
expect(rail.querySelectorAll('img')).toHaveLength(1)
|
||||
await waitFor(() => {
|
||||
expect([...rail.querySelectorAll('img')].map(img => img.getAttribute('alt')))
|
||||
.toEqual(['pasted.png', 'second.png'])
|
||||
})
|
||||
|
||||
const remove = rail.querySelector('button[aria-label^="移除图片"]')
|
||||
if (remove === null) throw new Error('remove button missing')
|
||||
fireEvent.click(remove)
|
||||
const remove = [...rail.querySelectorAll('button[aria-label^="移除图片"]')]
|
||||
if (remove.length !== 2) throw new Error('remove buttons missing')
|
||||
for (const button of remove) fireEvent.click(button)
|
||||
await waitFor(() => {
|
||||
expect(document.querySelector('[role="group"][aria-label="待发送图片"]')).toBeNull()
|
||||
})
|
||||
|
||||
@@ -196,12 +196,16 @@ export interface Config {
|
||||
dshHome?: string
|
||||
/** Maximum encoded bytes accepted for one image. */
|
||||
maxImageBytes?: number
|
||||
/** Maximum image count accepted in one submitted message. */
|
||||
maxImagesPerMessage?: number
|
||||
/** Maximum aggregate encoded image bytes accepted in one submitted message. */
|
||||
maxMessageImageBytes?: number
|
||||
/** Maximum intrinsic width multiplied by height accepted for one image. */
|
||||
maxImagePixels?: number
|
||||
}
|
||||
```
|
||||
|
||||
Source: [`packages/attachment/attachment-local/src/index.ts:20`](../packages/attachment/attachment-local/src/index.ts)
|
||||
Source: [`packages/attachment/attachment-local/src/index.ts:24`](../packages/attachment/attachment-local/src/index.ts)
|
||||
|
||||
## `@deepseek-ai/dsh-bash-local`
|
||||
|
||||
|
||||
@@ -260,6 +260,13 @@ Source: [`packages/ui/user-approval/src/index.ts:217`](../../packages/ui/user-ap
|
||||
Immutable binary attachment service. Implementations validate bytes before publishing a reference.
|
||||
|
||||
```ts cordis-catalog
|
||||
/**
|
||||
* Validate one image without persisting it.
|
||||
* Batch callers validate every member before saving any member.
|
||||
* @param input - encoded bytes, declared media type, and optional display name.
|
||||
*/
|
||||
abstract validateImage(input: SaveImageAttachment): void
|
||||
|
||||
/**
|
||||
* Validate and durably commit one image before its owning session event is appended.
|
||||
* @param input - encoded bytes, declared media type, and optional display name.
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/core-data-structures/attachment.md
|
||||
attachment.md: a06142b718dfcdc24ab037987a771869d68b31fa
|
||||
attachment.zh.md: d41a3bf9cd49c7abb684c2476a1ade6e2d373732
|
||||
attachment.md: 566f0198ce6e797e270b63e3cfd255a0025d7135
|
||||
attachment.zh.md: 95d755407b3b2ea7ab5b109f2e78edea16ccdd9c
|
||||
@@ -39,6 +39,8 @@ interface ImageAttachmentRef {
|
||||
/** Deployment-resolved limits shared by upload consumers and UI preflight. */
|
||||
interface ImageAttachmentLimits {
|
||||
maxImageBytes: number
|
||||
maxImagesPerMessage: number
|
||||
maxMessageImageBytes: number
|
||||
maxImagePixels: number
|
||||
mediaTypes: readonly ImageMediaType[]
|
||||
}
|
||||
@@ -67,4 +69,4 @@ interface StoredImageAttachment {
|
||||
}
|
||||
```
|
||||
|
||||
`saveImage()` validates bytes and atomically commits one object before returning its reference. `readImage()` accepts a reference from an authorized session path and returns bytes only after integrity verification. The service is deliberately retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to any one session's deletion.
|
||||
`saveImage()` validates bytes and atomically commits one object before returning its reference. `validateImage()` runs the same admission checks without persisting anything; batch callers validate every member through it before saving any member, so admission rejection leaves no partial objects behind. `readImage()` accepts a reference from an authorized session path and returns bytes only after integrity verification. The service is deliberately retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to any one session's deletion.
|
||||
@@ -39,6 +39,8 @@ interface ImageAttachmentRef {
|
||||
/** Deployment-resolved limits shared by upload consumers and UI preflight. */
|
||||
interface ImageAttachmentLimits {
|
||||
maxImageBytes: number
|
||||
maxImagesPerMessage: number
|
||||
maxMessageImageBytes: number
|
||||
maxImagePixels: number
|
||||
mediaTypes: readonly ImageMediaType[]
|
||||
}
|
||||
@@ -67,4 +69,4 @@ interface StoredImageAttachment {
|
||||
}
|
||||
```
|
||||
|
||||
`saveImage()` 校验字节并以原子方式提交一个对象,之后才返回其引用。`readImage()` 接受来自已授权会话路径的引用,只在完整性校验通过后返回字节。该服务刻意不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,而不是与任何一个会话的删除绑定。
|
||||
`saveImage()` 校验字节并以原子方式提交一个对象,之后才返回其引用。`validateImage()` 执行相同的准入检查,但不持久化任何内容;批量调用方会在保存任何成员前通过它校验所有成员,因此准入拒绝不会留下部分对象。`readImage()` 接受来自已授权会话路径的引用,只在完整性校验通过后返回字节。该服务刻意不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,而不是与任何一个会话的删除绑定。
|
||||
@@ -6,13 +6,17 @@ import z from 'schemastery'
|
||||
import { AttachmentStore } from '@deepseek-ai/dsh-attachment'
|
||||
import type { ImageAttachmentLimits, ImageAttachmentRef, SaveImageAttachment, StoredImageAttachment } from '@deepseek-ai/dsh-attachment'
|
||||
import { resolveDshHome } from '@deepseek-ai/dsh-paths'
|
||||
import { readImageFile, saveImageFile } from './store.ts'
|
||||
import { readImageFile, saveImageFile, validateImageFile } from './store.ts'
|
||||
|
||||
export { detectImage } from './image.ts'
|
||||
export { readImageFile, saveImageFile } from './store.ts'
|
||||
export { readImageFile, saveImageFile, validateImageFile } from './store.ts'
|
||||
|
||||
/** Default maximum encoded bytes for one image. */
|
||||
export const DEFAULT_MAX_IMAGE_BYTES = 5 * 1024 * 1024
|
||||
/** Default maximum images in one prompt. */
|
||||
export const DEFAULT_MAX_IMAGES_PER_MESSAGE = 10
|
||||
/** Default maximum aggregate image bytes in one prompt. */
|
||||
export const DEFAULT_MAX_MESSAGE_IMAGE_BYTES = 20 * 1024 * 1024
|
||||
/** Default maximum intrinsic pixels for one image. */
|
||||
export const DEFAULT_MAX_IMAGE_PIXELS = 40_000_000
|
||||
|
||||
@@ -22,6 +26,10 @@ export interface Config {
|
||||
dshHome?: string
|
||||
/** Maximum encoded bytes accepted for one image. */
|
||||
maxImageBytes?: number
|
||||
/** Maximum image count accepted in one submitted message. */
|
||||
maxImagesPerMessage?: number
|
||||
/** Maximum aggregate encoded image bytes accepted in one submitted message. */
|
||||
maxMessageImageBytes?: number
|
||||
/** Maximum intrinsic width multiplied by height accepted for one image. */
|
||||
maxImagePixels?: number
|
||||
}
|
||||
@@ -31,6 +39,8 @@ export class LocalAttachmentStore extends AttachmentStore {
|
||||
static Config: z<Config> = z.object({
|
||||
dshHome: z.string(),
|
||||
maxImageBytes: z.number().step(1).min(1).default(DEFAULT_MAX_IMAGE_BYTES),
|
||||
maxImagesPerMessage: z.number().step(1).min(1).default(DEFAULT_MAX_IMAGES_PER_MESSAGE),
|
||||
maxMessageImageBytes: z.number().step(1).min(1).default(DEFAULT_MAX_MESSAGE_IMAGE_BYTES),
|
||||
maxImagePixels: z.number().step(1).min(1).default(DEFAULT_MAX_IMAGE_PIXELS),
|
||||
})
|
||||
|
||||
@@ -43,11 +53,17 @@ export class LocalAttachmentStore extends AttachmentStore {
|
||||
this.root = resolve(join(resolveDshHome(config.dshHome), 'attachments', 'v1'))
|
||||
this.imageLimits = Object.freeze({
|
||||
maxImageBytes: config.maxImageBytes ?? DEFAULT_MAX_IMAGE_BYTES,
|
||||
maxImagesPerMessage: config.maxImagesPerMessage ?? DEFAULT_MAX_IMAGES_PER_MESSAGE,
|
||||
maxMessageImageBytes: config.maxMessageImageBytes ?? DEFAULT_MAX_MESSAGE_IMAGE_BYTES,
|
||||
maxImagePixels: config.maxImagePixels ?? DEFAULT_MAX_IMAGE_PIXELS,
|
||||
mediaTypes: Object.freeze(['image/png', 'image/jpeg', 'image/webp', 'image/gif'] as const),
|
||||
})
|
||||
}
|
||||
|
||||
validateImage(input: SaveImageAttachment): void {
|
||||
validateImageFile(input, this.imageLimits)
|
||||
}
|
||||
|
||||
async saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef> {
|
||||
return saveImageFile(this.root, input, this.imageLimits)
|
||||
}
|
||||
|
||||
@@ -52,6 +52,15 @@ function validateAdmission(metadata: Omit<ImageAttachmentRef, 'attachmentId' | '
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the full admission policy for one image without touching storage.
|
||||
* @param input - encoded bytes and declared metadata.
|
||||
* @param limits - resolved storage policy.
|
||||
*/
|
||||
export function validateImageFile(input: SaveImageAttachment, limits: ImageAttachmentLimits): void {
|
||||
validateAdmission(inspectMetadata(input.data, input.mediaType), limits)
|
||||
}
|
||||
|
||||
/**
|
||||
* Make a directory's entries durable (fsync on a read-only directory handle).
|
||||
* A synced file alone does not survive a crash when its directory entry never
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Context } from 'cordis'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
@@ -6,6 +7,8 @@ import { describe, expect, it } from 'vitest'
|
||||
import LocalAttachmentStore, {
|
||||
DEFAULT_MAX_IMAGE_BYTES,
|
||||
DEFAULT_MAX_IMAGE_PIXELS,
|
||||
DEFAULT_MAX_IMAGES_PER_MESSAGE,
|
||||
DEFAULT_MAX_MESSAGE_IMAGE_BYTES,
|
||||
} from '../src/index.ts'
|
||||
|
||||
describe('local attachment service', () => {
|
||||
@@ -13,6 +16,8 @@ describe('local attachment service', () => {
|
||||
const service = new LocalAttachmentStore(new Context(), {})
|
||||
expect(service.imageLimits).toEqual({
|
||||
maxImageBytes: DEFAULT_MAX_IMAGE_BYTES,
|
||||
maxImagesPerMessage: DEFAULT_MAX_IMAGES_PER_MESSAGE,
|
||||
maxMessageImageBytes: DEFAULT_MAX_MESSAGE_IMAGE_BYTES,
|
||||
maxImagePixels: DEFAULT_MAX_IMAGE_PIXELS,
|
||||
mediaTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'],
|
||||
})
|
||||
@@ -32,4 +37,21 @@ describe('local attachment service', () => {
|
||||
await rm(dshHome, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('validates without persisting: a rejected image leaves no storage root behind', async () => {
|
||||
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-validate-'))
|
||||
try {
|
||||
const service = new LocalAttachmentStore(new Context(), { dshHome })
|
||||
expect(() => { service.validateImage({ data: Uint8Array.of(1, 2, 3), mediaType: 'image/png' }) })
|
||||
.toThrow(/Unsupported or malformed image data/)
|
||||
const valid = Uint8Array.from(Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
))
|
||||
expect(() => { service.validateImage({ data: valid, mediaType: 'image/png' }) }).not.toThrow()
|
||||
expect(existsSync(service.root)).toBe(false)
|
||||
} finally {
|
||||
await rm(dshHome, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -28,6 +28,8 @@ const PNG = Uint8Array.from(Buffer.from(
|
||||
|
||||
const LIMITS: ImageAttachmentLimits = {
|
||||
maxImageBytes: 1024,
|
||||
maxImagesPerMessage: 2,
|
||||
maxMessageImageBytes: 2048,
|
||||
maxImagePixels: 16,
|
||||
mediaTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'],
|
||||
}
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md
|
||||
README.md: b25ef7bcf89b3b85600ed02ab12eb0cdd1117244
|
||||
README.zh.md: f45933c2a011978b31a306a1de80d7f031838c8e
|
||||
README.md: 4f450316294e554396adb9a8454051a08d9befd3
|
||||
README.zh.md: fe51b0003cdf1659c7c56106b97c6f3139ebe890
|
||||
@@ -4,7 +4,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
The durable attachment seam. `ctx.attachments` validates and atomically commits immutable image bytes, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, or base64 in session events.
|
||||
|
||||
Unsent composer images remain browser-owned temporary drafts. `saveImage` validates and commits one image at message submission or while committing structured provider output, before any model-visible session event is published. `readImage` verifies the content-addressed object against its logged metadata.
|
||||
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the same admission policy without persisting; batch writers validate every member first so a malformed member cannot strand earlier members as unreferenced objects. `saveImage` commits each accepted image before any model-visible session event is published, and `readImage` verifies the content-addressed object against its logged metadata.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
持久附件服务边界。`ctx.attachments` 校验并以原子方式提交不可变图片字节,随后返回可序列化的 `ImageAttachmentRef`;消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL 或 base64。
|
||||
|
||||
未发送的输入区图片仍是由浏览器持有的临时草稿。`saveImage` 在提交消息或提交结构化提供方输出时校验并提交一张图片,且发生在发布任何模型可见的会话事件之前。`readImage` 根据已记录的元数据校验内容寻址对象。
|
||||
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行相同的准入策略,但不执行持久化;批量写入方会先校验每个成员,避免某个格式错误的成员使较早的成员成为无引用对象。`saveImage` 会在发布任何模型可见的会话事件前提交每张已接受的图片,`readImage` 则根据已记录的元数据校验内容寻址对象。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -33,6 +33,13 @@ export abstract class AttachmentStore extends Service {
|
||||
/** Deployment-resolved image policy used by authoritative and fast-path validation. */
|
||||
abstract readonly imageLimits: ImageAttachmentLimits
|
||||
|
||||
/**
|
||||
* Validate one image without persisting it.
|
||||
* Batch callers validate every member before saving any member.
|
||||
* @param input - encoded bytes, declared media type, and optional display name.
|
||||
*/
|
||||
abstract validateImage(input: SaveImageAttachment): void
|
||||
|
||||
/**
|
||||
* Validate and durably commit one image before its owning session event is appended.
|
||||
* @param input - encoded bytes, declared media type, and optional display name.
|
||||
|
||||
@@ -36,6 +36,8 @@ export interface ImageAttachmentRef {
|
||||
/** Deployment-resolved limits shared by upload consumers and UI preflight. */
|
||||
export interface ImageAttachmentLimits {
|
||||
maxImageBytes: number
|
||||
maxImagesPerMessage: number
|
||||
maxMessageImageBytes: number
|
||||
maxImagePixels: number
|
||||
mediaTypes: readonly ImageMediaType[]
|
||||
}
|
||||
|
||||
@@ -1262,6 +1262,8 @@ export function createFixtureApi(options: FixtureOptions = {}): ApiProxy {
|
||||
model: 'fx-vision',
|
||||
imageLimits: {
|
||||
maxImageBytes: 5 * 1024 * 1024,
|
||||
maxImagesPerMessage: 10,
|
||||
maxMessageImageBytes: 20 * 1024 * 1024,
|
||||
maxImagePixels: 40_000_000,
|
||||
mediaTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'],
|
||||
},
|
||||
|
||||
@@ -52,7 +52,7 @@ export function apply(ctx: Context, config?: ConnectionConfig): void {
|
||||
for (const entry of trustedHosts) assertTrustedAuthority(entry)
|
||||
const apiHandler = toFetchHandler(ctx.apiProxy)
|
||||
const maxRequestBodyBytes = Math.ceil(
|
||||
ctx.attachments.imageLimits.maxImageBytes * 4 / 3,
|
||||
ctx.attachments.imageLimits.maxMessageImageBytes * 4 / 3,
|
||||
) + REQUEST_ENVELOPE_HEADROOM_BYTES
|
||||
const route: WebRoute = {
|
||||
kind: 'prefix',
|
||||
|
||||
@@ -23,7 +23,7 @@ function fakeHttpServer(routes: WebRoute[]): Pick<HttpServerService, 'register'
|
||||
|
||||
/** Structural attachments fake: the plugin only reads imageLimits for the body cap. */
|
||||
function fakeAttachments(): AttachmentStore {
|
||||
return { imageLimits: { maxImageBytes: 5 * 1024 * 1024 } } as AttachmentStore
|
||||
return { imageLimits: { maxMessageImageBytes: 20 * 1024 * 1024 } } as AttachmentStore
|
||||
}
|
||||
|
||||
/** Bodyless GET carrying the given headers (enough for the trust fence + bridge). */
|
||||
|
||||
@@ -283,11 +283,14 @@ export class ConversationService extends Service implements IConversation {
|
||||
): void {
|
||||
if (files.length === 0 && current.length === 0) return
|
||||
// Model capability is checked only by the host against the session's
|
||||
// current target; the client owns deployment limits and the one-image UI.
|
||||
// current target; the client owns deployment upload limits.
|
||||
const description = this.requireSessions().hostDescription()
|
||||
const limits = description?.imageLimits
|
||||
const all = [...current.map(attachment => attachment.file), ...files]
|
||||
if (all.length > 1) throw new Error('每条消息最多添加 1 张图片')
|
||||
if (limits !== undefined && all.length > limits.maxImagesPerMessage) {
|
||||
throw new Error(`每条消息最多添加 ${limits.maxImagesPerMessage} 张图片`)
|
||||
}
|
||||
let totalBytes = 0
|
||||
for (const file of all) {
|
||||
const mediaType = imageMediaType(file.type)
|
||||
if (limits !== undefined && !limits.mediaTypes.includes(mediaType)) {
|
||||
@@ -296,6 +299,10 @@ export class ConversationService extends Service implements IConversation {
|
||||
if (limits !== undefined && file.size > limits.maxImageBytes) {
|
||||
throw new Error(`图片 ${file.name || '未命名图片'} 超过单张大小限制`)
|
||||
}
|
||||
totalBytes += file.size
|
||||
}
|
||||
if (limits !== undefined && totalBytes > limits.maxMessageImageBytes) {
|
||||
throw new Error('图片总大小超过单条消息限制')
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -29,25 +29,6 @@ async function bench() {
|
||||
}
|
||||
|
||||
describe('ConversationService', () => {
|
||||
it('keeps the browser draft to one image', async () => {
|
||||
const b = await bench()
|
||||
const created = vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:draft-one')
|
||||
const revoked = vi.spyOn(URL, 'revokeObjectURL').mockReturnValue(undefined)
|
||||
try {
|
||||
const [first] = b.root.createDraftImages([new File([Uint8Array.of(1)], 'first.png', { type: 'image/png' })])
|
||||
if (first === undefined) throw new Error('draft attachment missing')
|
||||
expect(() => b.root.createDraftImages(
|
||||
[new File([Uint8Array.of(2)], 'second.png', { type: 'image/png' })],
|
||||
[first],
|
||||
)).toThrow('每条消息最多添加 1 张图片')
|
||||
expect(created).toHaveBeenCalledOnce()
|
||||
} finally {
|
||||
created.mockRestore()
|
||||
revoked.mockRestore()
|
||||
}
|
||||
await b.runtime.dispose()
|
||||
})
|
||||
|
||||
it('routes operations through the public Session binding', async () => {
|
||||
const b = await bench()
|
||||
await b.scoped.send('hello', 'steer')
|
||||
@@ -68,6 +49,47 @@ describe('ConversationService', () => {
|
||||
await b.runtime.dispose()
|
||||
})
|
||||
|
||||
it('accepts ordered batches and preflights their advertised count and aggregate limits', async () => {
|
||||
const b = await bench()
|
||||
const described = vi.spyOn(b.runtime.sessions, 'hostDescription').mockReturnValue({
|
||||
version: 'test',
|
||||
cwd: '/tmp',
|
||||
imageLimits: {
|
||||
maxImageBytes: 3,
|
||||
maxImagesPerMessage: 2,
|
||||
maxMessageImageBytes: 3,
|
||||
maxImagePixels: 4,
|
||||
mediaTypes: ['image/png'],
|
||||
},
|
||||
attachedSessions: 1,
|
||||
})
|
||||
const created = vi.spyOn(URL, 'createObjectURL')
|
||||
.mockReturnValueOnce('blob:first')
|
||||
.mockReturnValueOnce('blob:second')
|
||||
const revoked = vi.spyOn(URL, 'revokeObjectURL').mockReturnValue(undefined)
|
||||
try {
|
||||
const attachments = b.root.createDraftImages([
|
||||
new File([Uint8Array.of(1)], 'first.png', { type: 'image/png' }),
|
||||
new File([Uint8Array.of(2)], 'second.png', { type: 'image/png' }),
|
||||
])
|
||||
expect(attachments.map(attachment => attachment.file.name)).toEqual(['first.png', 'second.png'])
|
||||
expect(() => b.root.createDraftImages([
|
||||
new File([Uint8Array.of(3)], 'third.png', { type: 'image/png' }),
|
||||
], attachments)).toThrow('每条消息最多添加 2 张图片')
|
||||
const first = attachments[0]
|
||||
if (first === undefined) throw new Error('first draft attachment missing')
|
||||
expect(() => b.root.createDraftImages([
|
||||
new File([Uint8Array.of(3, 4, 5)], 'large.png', { type: 'image/png' }),
|
||||
], [first])).toThrow('图片总大小超过单条消息限制')
|
||||
expect(created).toHaveBeenCalledTimes(2)
|
||||
} finally {
|
||||
await b.runtime.dispose()
|
||||
described.mockRestore()
|
||||
created.mockRestore()
|
||||
revoked.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('releases draft images when the session scope is disposed', async () => {
|
||||
const b = await bench()
|
||||
const created = vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:draft-1')
|
||||
|
||||
@@ -160,6 +160,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
key: 'attachments',
|
||||
summary: 'Immutable binary attachment service.',
|
||||
methods: [
|
||||
{
|
||||
signature: 'abstract validateImage(input: SaveImageAttachment): void',
|
||||
jsDoc: '/**\n * Validate one image without persisting it.\n * Batch callers validate every member before saving any member.\n * @param input - encoded bytes, declared media type, and optional display name.\n */',
|
||||
},
|
||||
{
|
||||
signature: 'abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>',
|
||||
jsDoc: '/**\n * Validate and durably commit one image before its owning session event is appended.\n * @param input - encoded bytes, declared media type, and optional display name.\n * @returns a durable content-addressed reference.\n */',
|
||||
|
||||
@@ -79,23 +79,34 @@ async function durablePromptContent(ctx: Context, content: readonly PromptConten
|
||||
if (content.every(part => part.type === 'text')) {
|
||||
return content.map(part => ({ type: 'text', text: part.text }))
|
||||
}
|
||||
if (content.filter(part => part.type === 'image').length > 1) {
|
||||
throw new AttachmentError('A prompt may contain at most one image.', 'TOO_MANY_IMAGES')
|
||||
const limits = ctx.attachments.imageLimits
|
||||
if (content.filter(part => part.type === 'image').length > limits.maxImagesPerMessage) {
|
||||
throw new AttachmentError('Prompt exceeds the configured image-count limit.', 'TOO_MANY_IMAGES')
|
||||
}
|
||||
const durable: ContentBlock[] = []
|
||||
for (const part of content) {
|
||||
if (part.type === 'text') {
|
||||
durable.push({ type: 'text', text: part.text })
|
||||
continue
|
||||
}
|
||||
const attachment = await ctx.attachments.saveImage({
|
||||
data: decodeBase64(part.data),
|
||||
mediaType: part.mediaType,
|
||||
...part.name === undefined ? {} : { name: part.name },
|
||||
const prepared = content.map(part => part.type === 'text'
|
||||
? part
|
||||
: { part, data: decodeBase64(part.data) })
|
||||
const images = prepared.filter((part): part is Extract<typeof part, { data: Uint8Array }> => 'data' in part)
|
||||
const totalBytes = images.reduce((sum, image) => sum + image.data.byteLength, 0)
|
||||
if (totalBytes > limits.maxMessageImageBytes) {
|
||||
throw new AttachmentError('Prompt exceeds the configured aggregate image-byte limit.', 'IMAGES_TOO_LARGE')
|
||||
}
|
||||
for (const image of images) {
|
||||
ctx.attachments.validateImage({
|
||||
data: image.data,
|
||||
mediaType: image.part.mediaType,
|
||||
...image.part.name === undefined ? {} : { name: image.part.name },
|
||||
})
|
||||
durable.push({ type: 'image', attachment })
|
||||
}
|
||||
return durable
|
||||
return Promise.all(prepared.map(async (item): Promise<ContentBlock> => {
|
||||
if (!('data' in item)) return { type: 'text', text: item.text }
|
||||
const attachment = await ctx.attachments.saveImage({
|
||||
data: item.data,
|
||||
mediaType: item.part.mediaType,
|
||||
...item.part.name === undefined ? {} : { name: item.part.name },
|
||||
})
|
||||
return { type: 'image', attachment }
|
||||
}))
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -19,6 +19,8 @@ export const hostDescribeValueSchema = z.object({
|
||||
model: z.string().optional(),
|
||||
imageLimits: z.object({
|
||||
maxImageBytes: z.number().int().positive(),
|
||||
maxImagesPerMessage: z.number().int().positive(),
|
||||
maxMessageImageBytes: z.number().int().positive(),
|
||||
maxImagePixels: z.number().int().positive(),
|
||||
mediaTypes: z.array(imageMediaTypeSchema),
|
||||
}).optional(),
|
||||
|
||||
@@ -4,14 +4,14 @@
|
||||
* models, and the prompt-assembly boundary for a running selection change.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import AgentRegistry, { agentEvents } from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import LlmService, { LlmAdapter, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
GenerateOptions, LlmCallConfig, LlmModelInfo, LlmModelReasoningInfo, LlmProviderInfo,
|
||||
LlmResolvedModelInfo, StreamChunk,
|
||||
LlmResolvedModelInfo, StreamChunk, UserMessage,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import type { SessionId } from '@deepseek-ai/dsh-session'
|
||||
@@ -118,23 +118,66 @@ function expectValue<T>(response: { result: { ok: true; value: T } | { ok: false
|
||||
}
|
||||
|
||||
describe('Web session model selection', () => {
|
||||
it('rejects a second prompt image before attachment persistence', async () => {
|
||||
const { ctx, sessionId } = await harness()
|
||||
it('accepts ordered multi-image prompts and rejects configured batch-limit excess before persistence', async () => {
|
||||
const { ctx, agent, sessionId } = await harness()
|
||||
const validateImage = vi.fn((_input: { data: Uint8Array }): void => {})
|
||||
const saveImage = vi.fn((input: { data: Uint8Array; mediaType: 'image/png'; name?: string }) => {
|
||||
return Promise.resolve({
|
||||
attachmentId: `att-${String(input.data[0])}`,
|
||||
mediaType: input.mediaType,
|
||||
bytes: input.data.byteLength,
|
||||
width: 1,
|
||||
height: 1,
|
||||
...(input.name === undefined ? {} : { name: input.name }),
|
||||
})
|
||||
})
|
||||
const followup = vi.fn((_message: UserMessage): void => {})
|
||||
Object.assign(agent, { followup })
|
||||
ctx.provide('attachments', {
|
||||
imageLimits: { maxImageBytes: 4, maxImagesPerMessage: 2, maxMessageImageBytes: 4, maxImagePixels: 4, mediaTypes: ['image/png'] },
|
||||
validateImage,
|
||||
saveImage,
|
||||
} as never)
|
||||
const api = createApiProxy(ctx, { provider: 'deepseek', model: 'deepseek-chat', cwd: '/tmp', workspaceRoot: '/tmp' })
|
||||
const image = { type: 'image' as const, mediaType: 'image/png' as const, data: 'AA==' }
|
||||
const response = await api.sessions.prompt(request({
|
||||
const first = { type: 'image' as const, mediaType: 'image/png' as const, data: 'AQ==', name: 'first.png' }
|
||||
const second = { type: 'image' as const, mediaType: 'image/png' as const, data: 'Ag==', name: 'second.png' }
|
||||
const accepted = await api.sessions.prompt(request({
|
||||
sessionId,
|
||||
mode: 'queue' as const,
|
||||
content: [image, image],
|
||||
content: [first, { type: 'text' as const, text: 'compare' }, second],
|
||||
}))
|
||||
expect(response.result).toEqual({
|
||||
ok: false,
|
||||
error: {
|
||||
code: 'attachment-error',
|
||||
message: 'A prompt may contain at most one image.',
|
||||
details: { reason: 'TOO_MANY_IMAGES' },
|
||||
},
|
||||
expect(accepted.result).toMatchObject({ ok: true, value: { accepted: true } })
|
||||
expect(validateImage.mock.calls.map(([input]) => [...input.data])).toEqual([[1], [2]])
|
||||
expect(saveImage.mock.calls.map(([input]) => [...input.data])).toEqual([[1], [2]])
|
||||
expect(followup.mock.calls[0]?.[0].content).toEqual([
|
||||
{ type: 'image', attachment: { attachmentId: 'att-1', mediaType: 'image/png', bytes: 1, width: 1, height: 1, name: 'first.png' } },
|
||||
{ type: 'text', text: 'compare' },
|
||||
{ type: 'image', attachment: { attachmentId: 'att-2', mediaType: 'image/png', bytes: 1, width: 1, height: 1, name: 'second.png' } },
|
||||
])
|
||||
|
||||
const tooMany = await api.sessions.prompt(request({
|
||||
sessionId,
|
||||
mode: 'queue' as const,
|
||||
content: [first, second, first],
|
||||
}))
|
||||
expect(tooMany.result).toMatchObject({
|
||||
ok: false, error: { code: 'attachment-error', details: { reason: 'TOO_MANY_IMAGES' } },
|
||||
})
|
||||
const tooLarge = await api.sessions.prompt(request({
|
||||
sessionId,
|
||||
mode: 'queue' as const,
|
||||
content: [
|
||||
{ ...first, data: 'AQID' },
|
||||
{ ...second, data: 'BAUG' },
|
||||
],
|
||||
}))
|
||||
expect(tooLarge.result).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: 'attachment-error', details: { reason: 'IMAGES_TOO_LARGE' } },
|
||||
})
|
||||
expect(validateImage).toHaveBeenCalledTimes(2)
|
||||
expect(saveImage).toHaveBeenCalledTimes(2)
|
||||
expect(followup).toHaveBeenCalledTimes(1)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
|
||||
@@ -250,10 +250,16 @@ describe('PiAiAdapter provider routing', () => {
|
||||
class LateAttachmentStore extends AttachmentStore {
|
||||
readonly imageLimits: ImageAttachmentLimits = {
|
||||
maxImageBytes: 1,
|
||||
maxImagesPerMessage: 1,
|
||||
maxMessageImageBytes: 1,
|
||||
maxImagePixels: 1,
|
||||
mediaTypes: ['image/png'],
|
||||
}
|
||||
|
||||
validateImage(_input: SaveImageAttachment): void {
|
||||
throw new Error('not used')
|
||||
}
|
||||
|
||||
saveImage(_input: SaveImageAttachment): Promise<ImageAttachmentRef> {
|
||||
return Promise.reject(new Error('not used'))
|
||||
}
|
||||
|
||||
@@ -68,10 +68,16 @@ async function harness(image?: StoredImageAttachment): Promise<Context> {
|
||||
class E2eAttachmentStore extends AttachmentStore {
|
||||
readonly imageLimits: ImageAttachmentLimits = {
|
||||
maxImageBytes: fixture.data.byteLength,
|
||||
maxImagesPerMessage: 1,
|
||||
maxMessageImageBytes: fixture.data.byteLength,
|
||||
maxImagePixels: fixture.ref.width * fixture.ref.height,
|
||||
mediaTypes: [fixture.ref.mediaType],
|
||||
}
|
||||
|
||||
validateImage(_input: SaveImageAttachment): void {
|
||||
throw new Error('e2e attachment fixture is read-only')
|
||||
}
|
||||
|
||||
saveImage(_input: SaveImageAttachment): Promise<ImageAttachmentRef> {
|
||||
return Promise.reject(new Error('e2e attachment fixture is read-only'))
|
||||
}
|
||||
|
||||
@@ -90,10 +90,16 @@ const ATTACHMENT_COMPANION = '../packages/attachment/attachment-local/src/invari
|
||||
class TestAttachmentStore extends AttachmentStore {
|
||||
readonly imageLimits: ImageAttachmentLimits = {
|
||||
maxImageBytes: 1,
|
||||
maxImagesPerMessage: 1,
|
||||
maxMessageImageBytes: 1,
|
||||
maxImagePixels: 1,
|
||||
mediaTypes: ['image/png'],
|
||||
}
|
||||
|
||||
validateImage(_input: SaveImageAttachment): void {
|
||||
throw new Error('test invariant attachment store does not validate images')
|
||||
}
|
||||
|
||||
saveImage(_input: SaveImageAttachment): Promise<ImageAttachmentRef> {
|
||||
return Promise.reject(new Error('test invariant attachment store does not save images'))
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user