Merge master into worktree/custom-model-image-input-0185de
This commit is contained in:
Yichen Jiang
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: f1fbcbd29b188505e647265c4c974886c066e936
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: a55467cacc412b6f313dd932a0e4868965c039ea
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 8639a1ab638c85fc01a29083a1b81eacdc2152d4
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 1090edd06a62d70a736c85eb1c7d9e6edba187c8
|
||||
+1
-1
@@ -140,7 +140,7 @@ Composer thumbnails and each `MessageImage` own ephemeral original-preview state
|
||||
|
||||
### Limits and trust boundaries
|
||||
|
||||
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 5 MiB per image, 10 images and 20 MiB aggregate image bytes per message, and 40 million intrinsic pixels per image. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap (32 MiB by default) for every API request and fails load if it cannot hold the attachment service's aggregate image limit after base64 and envelope expansion; lowering image policy therefore never silently lowers the carrier limit for valid text or other RPCs. A body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 5 MiB per image, 20 images and 100 MiB aggregate image bytes per message, and 40 million intrinsic pixels per image. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap (160 MiB by default) for every API request and fails load if it cannot hold the attachment service's aggregate image limit after base64 and envelope expansion; lowering image policy therefore never silently lowers the carrier limit for valid text or other RPCs. A body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
|
||||
Malformed base64, unsupported or mismatched media, truncated image payloads, excess bytes, excess image count, excess pixels, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser.
|
||||
|
||||
|
||||
+1
-1
@@ -140,7 +140,7 @@ Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachme
|
||||
|
||||
### 限制与信任边界
|
||||
|
||||
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 5 MiB、每条消息 10 张图片和 20 MiB 图片总字节数，以及每张图片 4,000 万个固有像素。这些随部署变化的限制属于经过校验的后端配置，并由宿主在持久化前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限（默认 32 MiB）；如果该上限无法容纳附件服务的图片总量限制经 base64 和请求封装膨胀后的大小，加载就会失败。因此，降低图片策略绝不会静默降低有效文本或其他 RPC 的载体上限。未声明长度的请求体在越过上限的瞬间即被拒绝，而不是先读完再拒。
|
||||
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 5 MiB、每条消息 20 张图片和 100 MiB 图片总字节数，以及每张图片 4,000 万个固有像素。这些随部署变化的限制属于经过校验的后端配置，并由宿主在持久化前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限（默认 160 MiB）；如果该上限无法容纳附件服务的图片总量限制经 base64 和请求封装膨胀后的大小，加载就会失败。因此，降低图片策略绝不会静默降低有效文本或其他 RPC 的载体上限。未声明长度的请求体在越过上限的瞬间即被拒绝，而不是先读完再拒。
|
||||
|
||||
格式错误的 base64、不支持或不匹配的媒体、截断的图片数据、超出字节限制、超出图片数量、超出像素限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段，控制字符会被移除，并且任何本地路径都不会写入日志或返回浏览器。
|
||||
|
||||
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-12-web-image-intake-and-limits-alignment.md
|
||||
2026-08-12-web-image-intake-and-limits-alignment.md: 2f8b99bb4850d9875dcba0a03ae8ad9f340d1506
|
||||
2026-08-12-web-image-intake-and-limits-alignment.zh.md: 62d5ebd54275ae8de0e0b9ba701ba34042dfcde7
|
||||
@@ -0,0 +1,35 @@
|
||||
# Agent Note: Whole-page image drop, projected intake limits, and thumbnail tiling
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-12-web-image-intake-and-limits-alignment.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The second alignment step for issue #2248, after the [attachment display note](2026-08-11-web-attachment-display-alignment.md) (whose rail/toast/atoms decisions stand; this note supersedes its history-gallery geometry and the lightbox backdrop specifics). Remaining gaps against DeepSeek Chat: images could only be dropped on the composer card — a drop over the transcript navigated the browser away to the file; the lightbox close glyph was a bare `×` text character (buttons inherit no font family and the glyph's ink sits above the line box center, so it rendered visibly off-center) over a `color-mix(label-primary 74%)` backdrop that inverts to a bright white wash in dark mode; a message's images stacked vertically as up-to-240px blocks because the gallery container itself was pinned to 240px; and nothing client-side enforced or displayed the image limits — a user could stack 50 images and learn about `maxImagesPerMessage` only from a raw `attachment-error (TOO_MANY_IMAGES)` toast after submit, watching the rail empty and refill around the failure.
|
||||
|
||||
## Decision
|
||||
|
||||
**Whole-page drop.** InputBar binds `dragenter`/`dragover`/`dragleave`/`drop` on the document (enter/leave depth counting, viewport-edge and `dragend` resets, `Files`-type gating so text drags keep their native textarea path) and renders the new `DropOverlay` atom in `ui-attachment`: a body-portaled, pointer-inert full-viewport layer (DeepSeek Chat's DragMask visuals — white/70% + 10px blur, dark `rgba(39,39,48,0.7)`, illustration, title, limits line) whose `disabled` variant announces a locked/busy composer. Pointer-inertness is load-bearing: drag events keep targeting the page below, so the depth count never sees the overlay itself. Document-level listener state is safe because the composer-bar slot is `kind: 'single'`.
|
||||
|
||||
**Lightbox.** The close control is `ui-primitives`' `IconCloseOutline16` (the Modal precedent — an SVG centered in its viewBox needs no font metrics). The backdrop is the shared dialog mask (`--dsw-alias-bg-mask-1` + `--dsw-mask-blur`, black-based in both themes) painted on a separate sibling layer, because `backdrop-filter` on the container would blur the previewed image itself.
|
||||
|
||||
**History thumbnails (DeepSeek Chat rules).** A message's lone image renders at 240px on its long edge with the displayed ratio clamped to [0.25, 4], cropped by `cover` with the anchor at the top of very tall images and the left of very wide ones, never upscaled; several images render as fixed 64px square tiles in one wrapping row (10px gap, user messages right-aligned). Consecutive assistant `image` blocks merge into one gallery so they tile instead of each opening a one-image row.
|
||||
|
||||
**Limits aligned and projected.** Defaults are 20 images / 5 MiB per image / 100 MiB aggregate (`attachment-local`), with the HTTP carrier cap raised to one shared `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB` (http-bridge, previously two independent 32 MiB literals) to satisfy the load-time capacity assertion (aggregate × 4/3 + headroom ≈ 134.3 MiB). Consumer products cluster at 10–20 attachments (ChatGPT 10, Gemini 10, Claude 20; DeepSeek Chat's 50 is the outlier), and a vision-model image costs roughly 1300–4800 tokens, so 50 images can fill a 200k context in one message. The 5 MiB per-image default admits images across Anthropic routes that impose either a 5 MiB or 10 MiB maximum; deployments using only routes with the larger limit can override it. A 512 MiB aggregate cannot pass this transport because base64-in-JSON would need a single JSON string past V8's ~512 MiB string ceiling. The limits reach clients as the `imageLimits` session projection — a constant-per-boot unit (`apply` returns the same state reference, so baselines alone carry it and no change frames exist) registered by **apiproxy**, not the attachment Service Definition: `dsh-llm` depends on `dsh-attachment` (`ImageBlock` → `ImageAttachmentRef`), so the seam package referencing `dsh-session-projection` (whose graph reaches `dsh-llm` through `dsh-session`) closes a project-reference cycle, and the per-message count/aggregate rules the value describes are the proxy's own admission checks anyway. The `SessionProjectionMap` merge rides the proxy's sessions wire-contract file, which every client program already includes through the carrier's type re-exports.
|
||||
|
||||
**Intake pre-check and error copy.** Both intake gestures converge on one `intakeImages` wrapper in InputBar that checks count, per-image bytes, and aggregate bytes against the projection before `addImages`: a violating batch is refused whole (DeepSeek Chat semantics) with an immediate banner naming the limit — no submit-time rollback theater. The host checks stay as the backstop for callers that bypass the composer. Banner copy follows one principle the user set: reasons a user can act on (model without vision, count, size, resolution, format — now a positive list of supported formats instead of echoing the rejected MIME type) get product sentences naming the way out; reasons they cannot act on (corrupt base64, lost references, read failures) fold into one send-failed sentence that keeps the reason code, because the product currently faces developers and a reportable code beats a dead end. Non-attachment error codes keep the raw message + code presentation.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Registering the projection unit in the attachment Service Definition's constructor.** The natural seam owner, and the first implementation — rejected by the dependency graph (the cycle above) and by a test-harness interaction: the base constructor calling `ctx.inject` made directly-constructed stores in specs trigger the global invariant host, which then double-mounted an `attachments` service into the same root.
|
||||
|
||||
**`--dsw-alias-bg-mask-photo` (0.88 black, theme-stable, unused) for the lightbox.** The design system's photo-viewer token and dsweb's likely lightbox wash; the user chose consistency with the settings dialog mask (`bg-mask-1` + blur) — both fix the dark-mode inversion.
|
||||
|
||||
**Pre-checking inside `apply.ts`'s `addImages` inject.** The seam-purist placement, rejected for plumbing cost: the projection store has no non-React face exposed to the inject factory, while InputBar already consumes projections idiomatically and is the single caller of both gestures.
|
||||
|
||||
**A `host.describe` field instead of a projection.** Session-independent and cheaper, but delivered through an injected prop chain rather than `useProjection`, and the projection's key-absence semantics ("no attachment service composed → no pre-check") fall out for free.
|
||||
|
||||
## Consequences
|
||||
|
||||
A drop anywhere on the window now lands in the rail, over-limit intake fails at the moment of the gesture with copy naming the limit, and history images tile like DeepSeek Chat's. The carrier's default request-body budget is ~5× larger and remains a per-request resident-memory bound (the bridge buffers bodies whole; recorded in the connection README's limitations). The fixture transport mirrors the projection with hardcoded default numbers — a deployment that overrides the limits diverges from fixture-mode copy, acceptable for a keyless demo lane. Gallery arrow navigation, lightbox zoom/download, and non-image file cards remain deferred (#2248).
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
# Agent Note：整页图片拖放、上限投影预检与缩略图平铺
|
||||
|
||||
状态：implemented
|
||||
|
||||
[English](2026-08-12-web-image-intake-and-limits-alignment.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
issue #2248 的第二步对齐，接在[附件展示 note](2026-08-11-web-attachment-display-alignment.md) 之后（其附件栏、toast 与原子组件包的决策继续有效；本 note 取代其中历史画廊几何与灯箱 backdrop 的具体规格）。与 DeepSeek Chat 相比剩下的差距：图片只能拖到 composer 卡片上——拖到聊天记录区会让浏览器直接导航到文件；灯箱关闭钮是裸 `×` 文本字符（button 不继承字体，且该字形的墨迹在行框中心之上，因此明显偏斜），backdrop 用 `color-mix(label-primary 74%)`，dark 下反转成刺眼的白色蒙层；一条消息的多张图各自以最大 240px 的块竖着堆叠，因为画廊容器本身被钉在 240px；客户端完全不执行也不展示图片限额——用户可以攒 50 张图，直到提交后收到原始的 `attachment-error (TOO_MANY_IMAGES)` toast，眼看附件栏清空又回滚。
|
||||
|
||||
## 决策
|
||||
|
||||
**整页拖放。** InputBar 在 document 上绑定 `dragenter`/`dragover`/`dragleave`/`drop`（enter/leave 深度计数、视口边缘与 `dragend` 复位、按 `Files` 类型门控使文本拖拽保留原生 textarea 路径），并渲染 `ui-attachment` 新增的 `DropOverlay` 原子组件：经 body portal、不接收指针事件的全视口层（DeepSeek Chat DragMask 的视觉——白色 70% 加 10px 模糊，dark 为 `rgba(39,39,48,0.7)`，插画、标题、上限行），`disabled` 变体宣告锁定或忙碌的 composer。指针惰性是承重的：拖拽事件继续命中下方页面，深度计数永远看不到遮罩自己。document 级监听状态是安全的，因为 composer-bar slot 为 `kind: 'single'`。
|
||||
|
||||
**灯箱。** 关闭钮换成 `ui-primitives` 的 `IconCloseOutline16`（Modal 的先例——在 viewBox 内居中的 SVG 不依赖字体度量）。backdrop 用共享的对话框遮罩（`--dsw-alias-bg-mask-1` 加 `--dsw-mask-blur`，两个主题都是黑基色），画在独立的兄弟图层上，因为 `backdrop-filter` 画在容器上会把预览图自己也模糊掉。
|
||||
|
||||
**历史缩略图（DeepSeek Chat 规则）。** 一条消息仅有的一张图长边 240px、展示比例钳制在 [0.25, 4]，`cover` 裁切，特别高的图锚定顶部、特别宽的锚定左侧，从不放大；多张图渲染为固定 64px 方块，单个可换行的横排（10px 间距，用户消息右对齐）。assistant 连续的 `image` 块合并进同一个画廊，平铺而不是各占一行。
|
||||
|
||||
**上限对齐并投影。** 默认值为每条消息 20 张、单图 5 MiB、总量 100 MiB（`attachment-local`），HTTP 载体上限提为唯一共享的 `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB`（http-bridge，原先是两个独立的 32 MiB 字面量），以满足加载时的容量断言（总量 × 4/3 加余量 ≈ 134.3 MiB）。消费级产品集中在 10 到 20 个附件（ChatGPT 10、Gemini 10、Claude 20；DeepSeek Chat 的 50 是例外），且视觉模型一张图约 1300 到 4800 token，因此 50 张图可在一条消息中填满 200k 上下文。默认单图上限采用 5 MiB，可适用于分别采用 5 MiB 或 10 MiB 上限的 Anthropic 路由；仅使用较大上限路由的部署可以覆盖该值。512 MiB 总量无法通过当前传输，因为 base64 进 JSON 需要一个超过 V8 约 512 MiB 字符串上限的单个 JSON 字符串。限额以 `imageLimits` 会话投影到达客户端。它是每次启动恒定的单元（`apply` 返回同一状态引用，因此只靠基线携带、不存在变更帧），由 **apiproxy** 而非 attachment Service Definition 注册：`dsh-llm` 依赖 `dsh-attachment`（`ImageBlock` → `ImageAttachmentRef`），seam 包引用 `dsh-session-projection`（其图谱经 `dsh-session` 到达 `dsh-llm`）会闭合 project-reference 环，而该值描述的每消息数量与总量规则本来就是 proxy 自己的准入检查。`SessionProjectionMap` 合并放在 proxy 的 sessions 协议文件里，每个客户端程序都经载体的类型再导出包含它。
|
||||
|
||||
**加入预检与错误文案。** 两种加入手势汇合到 InputBar 的一个 `intakeImages` 包装：在 `addImages` 之前按投影检查数量、单图字节与总字节，违规的一批整体拒收（DeepSeek Chat 语义）并立刻弹出点名上限的横幅——不再有提交时的回滚戏码。宿主检查保留，兜底绕过 composer 的调用方。横幅文案遵循用户定下的一条原则：用户能解决的原因（模型不支持视觉、数量、大小、分辨率、格式——格式改为正面列出支持列表而不是回显被拒的 MIME 类型）用点明出路的产品句子；用户无法解决的原因（base64 损坏、引用丢失、读取失败）折叠为一条保留原因码的发送失败句子，因为产品当前面向开发者，可上报的码好过死胡同。非附件错误码保留原文加错误码的展示。
|
||||
|
||||
## 备选方案
|
||||
|
||||
**在 attachment Service Definition 构造函数里注册投影单元。** 天然的 seam 归属，也是第一版实现——被依赖图（上述环）和一个测试基建交互否决：基类构造函数调用 `ctx.inject` 使得 spec 中直接构造的 store 触发全局 invariant 宿主，后者往同一 root 重复挂载 `attachments` 服务。
|
||||
|
||||
**灯箱用 `--dsw-alias-bg-mask-photo`（0.88 黑、主题恒定、无人使用）。** 设计系统的照片查看器 token，也可能是 dsweb 灯箱实际的蒙层；用户选择与 settings 对话框遮罩一致（`bg-mask-1` 加模糊）——两者都能修复 dark 反转。
|
||||
|
||||
**在 `apply.ts` 的 `addImages` inject 里预检。** seam 纯度上的位置，因管线成本否决：投影仓没有暴露给 inject 工厂的非 React 面，而 InputBar 已经以惯用方式消费投影，且是两种手势的唯一调用方。
|
||||
|
||||
**用 `host.describe` 字段代替投影。** 与会话无关且更便宜，但要经注入 prop 链而非 `useProjection` 送达，而投影的键缺席语义（"未组合 attachment 服务 → 不预检"）是白拿的。
|
||||
|
||||
## 后果
|
||||
|
||||
拖到窗口任何位置都能进附件栏，超限加入在手势发生的那一刻就以点名上限的文案失败，历史图片像 DeepSeek Chat 一样平铺。载体的默认请求体预算扩大约 5 倍，并且仍是单请求驻留内存上界（桥把请求体整体缓冲；已记录在 connection README 的限制节）。fixture 传输用硬编码的默认数字镜像该投影——改配置的部署会与 fixture 模式的文案分叉，对 keyless 演示通道可接受。画廊左右切换、灯箱缩放与下载、非图片文件卡片仍然推迟（#2248）。
|
||||
Reference in New Issue
Block a user