diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml index efdb14ea94..dc9b924f54 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md -2026-06-17-filesystem-capability-seam.md: fee0161e5e8397ac1d1c0e2850efad840c65d971 -2026-06-17-filesystem-capability-seam.zh.md: 46e3ad22c530319d0d1b6ba23a8aba8ffc2fdb8c +2026-06-17-filesystem-capability-seam.md: 2fb360e5ad3c972f4a8de50c602a79af158dfc22 +2026-06-17-filesystem-capability-seam.zh.md: 1f70113b1583d890047ee4a17161f76b6390d8e8 diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md index fee0161e5e..2fb360e5ad 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md @@ -62,8 +62,9 @@ The root `tool-fs` plugin registers the full filesystem tool suite (`read`, `wri The interface covers these semantic operations: - Resolve a model/plugin-supplied path into a backend-defined target. +- Convert a resolved target to the canonical process path or `file:` URI for the same execution world, and test containment without parsing its opaque key. - Stat target metadata without reading file contents. -- Read a bounded UTF-8 text page from a target. +- Read complete or streamed UTF-8 text, including one stable-handle byte-bounded whole-file operation. - Create or replace a UTF-8 text file. - Edit an existing UTF-8 text file by literal replacement. @@ -83,9 +84,11 @@ Resolved targets must expose at least three concepts: - An opaque `targetKey`, used for stale guards and file-state lookup. The local backend might use a realpath-like key; a remote backend might use a workspace URI or file id. Consumers must not parse or assume this is a local absolute path. - A `displayPath`, used for model/UI-facing output. It may be a local absolute path, workspace-relative path, or remote URI depending on the backend. +`targetKey` remains opaque even when another capability shares the provider's execution world. Such consumers ask the provider for `processPath(target)`, `fileUrl(target)`, or `contains(parent, child)`; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns why these facts sit on the filesystem seam. + Read and mutation results must include an opaque file `version`. The local backend derives its token from bigint stat metadata (`dev`, `ino`, `size`, `mtimeNs`, and `ctimeNs`) so same-size rewrites and inode replacement invalidate consumers reliably; a remote backend can use a revision id or hash-like token. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token. -The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files. Both own regular-file checks, bounded line/output handling is NOT theirs — line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`), which reads through `ctx.fs` and renders the model-facing window. The provider owns UTF-8 decoding and binary/NUL rejection; it does not know about line windows or views. +The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files, and `readTextBounded` holds one backend-owned stable handle while rejecting a complete file above its byte ceiling. Line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`). The provider owns regular-file checks, UTF-8 decoding, binary/NUL rejection, and the bounded read's replacement/growth race; it does not know about line windows or views. Observed-state recording is not on `ctx.fs`: after a successful read the executor emits `fs/observed`, and the `dsh-fs-policy` plugin records `{ version }` for the deriving owner. There is no `full`/`partial` view — a read at any window records the version, and freshness (not view completeness) authorizes a later write/edit. diff --git a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md index 46e3ad22c5..1f70113b15 100644 --- a/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.zh.md @@ -16,7 +16,7 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` 如果没有 `ctx.fs` 接口，将本地文件系统访问替换为沙箱或远程后端时，即使面向模型的契约应当保持稳定，工具 schema、演示和提示词引导也会被迫变动。这还使权限/沙箱边界更难推理：一个 `cwd` 选项看起来像沙箱，但除非有显式的后端或 `tools/execute` 策略强制隔离，否则它只是一个基础路径。 -我们需要文件系统工具在成为公开包接口之前，以与 bash 相同的能力 seam 形态落地。 +我们需要文件系统工具在成为公开包（package）接口之前，以与 bash 相同的能力 seam 形态落地。 ## 决策 @@ -51,7 +51,7 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` `@deepseek-ai/dsh-fs-local` 依赖 `@deepseek-ai/dsh-fs` 和 `cordis`。它继承 `FileSystem`，将自身注册为 `ctx.fs`，拥有本地后端配置（如基目录），并包含所有直接的 `node:fs` / `node:path` 访问。它不持有观测状态存储——新鲜度是后端铸造、策略插件记录的版本令牌。 -`@deepseek-ai/dsh-tool-fs` 依赖 `@deepseek-ai/dsh-fs`、`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-system-prompt` 和 `cordis`。它注册面向模型的工具和提示词段落。它禁止导入 `node:fs`、`node:path` 或 `@deepseek-ai/dsh-fs-local`；文件系统执行始终通过 `ctx.fs`。如果实现需要具体的 agent（智能体）或会话辅助类型，这些依赖属于 `tool-fs`；它们禁止回漏到 `dsh-fs` 中。 +`@deepseek-ai/dsh-tool-fs` 依赖 `@deepseek-ai/dsh-fs`、`@deepseek-ai/dsh-tools`、`@deepseek-ai/dsh-system-prompt` 和 `cordis`。它注册面向模型的工具和提示词段落。它禁止导入 `node:fs`、`node:path` 或 `@deepseek-ai/dsh-fs-local`；文件系统执行始终通过 `ctx.fs`。如果实现需要具体的 agent 或会话辅助类型，这些依赖属于 `tool-fs`；它们禁止回漏到 `dsh-fs` 中。 根 `tool-fs` 插件通过组合各工具的注册辅助函数来注册完整的文件系统工具套件（`read`、`write` 和 `edit`）。它注入 `fs`，从不导入实现包。 @@ -62,8 +62,9 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` 该接口涵盖以下语义操作： - 将模型/插件提供的路径解析为后端定义的目标。 +- 将解析后的目标转换为同一执行环境的规范进程路径或 `file:` URI，并在不解析其不透明键的情况下检查包含关系。 - 获取目标元数据而不读取文件内容。 -- 从目标读取有界的 UTF-8 文本页。 +- 读取完整或流式 UTF-8 文本，其中包括一项持有稳定句柄、以字节为上限的全文件读取操作。 - 创建或替换一个 UTF-8 文本文件。 - 通过字面替换编辑一个已有的 UTF-8 文本文件。 @@ -83,13 +84,15 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` - 不透明的 `targetKey`，用于陈旧守护和文件状态查找。本地后端可能使用类似 realpath 的键；远程后端可能使用工作区 URI 或文件 id。消费方禁止解析或假设它是本地绝对路径。 - `displayPath`，用于面向模型/UI 的输出。根据后端不同，它可能是本地绝对路径、工作区相对路径或远程 URI。 -读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据（`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`）派生令牌，因此同大小重写和 inode 替换都会可靠地使消费方持有的版本失效；远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查；消费方可以展示相关元数据但禁止解释版本令牌。 +即使另一项能力共享提供方的执行环境，`targetKey` 仍保持不透明。这类消费方通过提供方的 `processPath(target)`、`fileUrl(target)` 或 `contains(parent, child)` 获取所需事实；[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)说明这些事实为何属于文件系统 seam。 -提供方返回已解码的文本：`readText` 返回整个常规文本文件，`streamText` 为大文件流式输出相同的文本语义。两者负责常规文件检查；有界的行/输出处理不是它们的职责——行窗口化、带行号渲染和总行数统计位于执行器（`dsh-tool-fs`）中，执行器通过 `ctx.fs` 读取并渲染面向模型的窗口。提供方负责 UTF-8 解码和二进制/NUL 拒绝；它不知道行窗口或视图。 +读取和变更结果必须包含不透明的文件 `version`。本地后端从 bigint stat 元数据（`dev`、`ino`、`size`、`mtimeNs` 和 `ctimeNs`）派生令牌，因此同大小重写和 inode 替换都会可靠地使消费方失效；远程后端可以使用 revision id 或类似 hash 的令牌。`dsh-fs-policy` 插件记录版本用于陈旧检查；消费方可以展示相关元数据但禁止解释版本令牌。 + +提供方返回已解码的文本：`readText` 返回整个常规文本文件，`streamText` 为大文件流式传输相同的文本语义，`readTextBounded` 则持有一个归后端所有的稳定句柄，并在完整文件超过字节上限时拒绝。行窗口化、带行号渲染和总行数统计位于执行器（`dsh-tool-fs`）中。提供方负责普通文件检查、UTF-8 解码、二进制/NUL 拒绝，以及有界读取期间的路径替换／增长竞态；它不知道行窗口或视图。 观测状态记录不在 `ctx.fs` 上：成功读取后，执行器发出 `fs/observed`，`dsh-fs-policy` 插件为推导出的 owner 记录 `{ version }`。没有 `full`/`partial` 视图——任何窗口的读取都记录版本，新鲜度（而非视图完整性）授权后续的写入/编辑。 -全文件写入创建或替换 UTF-8 文本文件。后端在支持且有文档说明时可以创建父目录。已有的非常规目标被拒绝。`writeText` 接受一个可选期望：`createIfAbsent` 创建缺失的目标并拒绝已存在的（报 `FS_NOT_OBSERVED`，这是策略处理未观测 owner 时采用的分支）；`replaceIfVersion` 仅在目标处于观测版本时替换，否则报 `FS_STALE_VERSION`；省略期望则为无条件的裸提供方创建或覆盖。策略插件根据 owner 的观测状态选择提供哪个期望。 +全文件写入创建或替换 UTF-8 文本文件。后端在支持且有文档说明时可以创建父目录。已有的非常规目标被拒绝。`writeText` 接受一个可选期望：`createIfAbsent` 创建缺失的目标并拒绝已存在的（报 `FS_NOT_OBSERVED`，这是策略为未观测 owner 使用的路径）；`replaceIfVersion` 仅在目标处于观测版本时替换，否则报 `FS_STALE_VERSION`；省略期望则为无条件的裸提供方创建或覆盖。策略插件根据 owner 的观测状态选择提供哪个期望。 字面编辑是提供方原语（`editText`），而非在 `tool-fs` 中由读取加写入组合而成。字面匹配、重复匹配拒绝、CRLF 保留、二进制拒绝、可选的陈旧版本检查和原子读-改-写必须一起留在后端的变更临界区内。`editText` 接受相同的可选版本期望；陈旧检查在字面匹配之前运行，因此基于旧读取的编辑会报 `FS_STALE_VERSION`。远程后端可以将编辑实现为原生的 compare-and-edit 操作；消费方不强制本地风格的组合。 @@ -124,11 +127,11 @@ harness 已有一个具体的 `bash` 能力 seam（`dsh-bash` / `dsh-bash-local` ## 测试 -测试遵循包边界，而不仅是用户可见的工具：`dsh-fs` 中的服务 seam；`dsh-fs-local` 中通过 `ctx.fs` 接口测试的真实文件系统行为（解析、符号链接、流式输出、二进制/UTF-8 拒绝、无条件和版本守护的写入、字面编辑语义、行尾保留、结构化 `FsError` 错误码）；`dsh-tool-fs` 中基于真实本地提供方的消费方接口（只 mock 模型/时钟，从不 mock 协作者）；以及通过 `ctx.tools.execute()` 在有和没有 `dsh-fs-policy` 的情况下进行集成测试，通过从磁盘回读文件来验证世界状态，既不信任规范值，也不信任渲染内容。观测状态/owner 推导策略在 `dsh-fs-policy` 中测试，不在此处。 +测试遵循包边界，而不仅是用户可见的工具：`dsh-fs` 中的服务 seam；`dsh-fs-local` 中通过 `ctx.fs` 接口测试的真实文件系统行为（解析、符号链接、流式传输、二进制/UTF-8 拒绝、无条件和版本守护的写入、字面编辑语义、行尾保留、结构化 `FsError` 错误码）；`dsh-tool-fs` 中基于真实本地提供方的消费方接口（只 mock 模型/时钟，从不 mock 协作者）；以及通过 `ctx.tools.execute()` 在有和没有 `dsh-fs-policy` 的情况下进行集成测试，通过从磁盘回读文件来验证世界状态，既不信任规范值，也不信任渲染内容。观测状态/owner 推导策略在 `dsh-fs-policy` 中测试，不在此处。 本仓库曾踩过的防御性模式类别被直接固定： -- **原子写入临时文件安全。** 写入/编辑通过目标旁边一个私有随机 `0700` 目录中独占且仅所有者可访问（`'wx'`、`0o600`）的临时文件暂存，失败时清理，最后原子 rename——与 bash spill 文件规则一致，因为可预测的全局可读临时路径会招致符号链接竞争和信息泄露。测试断言权限，并断言已存在的临时路径不会被覆盖；此原语是 seam 的常设要求。 +- **原子写入临时文件安全。** 写入/编辑通过目标旁边一个私有随机 `0700` 目录中的独占 owner-only（`'wx'`、`0o600`）临时文件暂存，失败时清理，最后原子 rename——与 bash 溢出文件规则一致，因为可预测的 world-readable 临时路径招致符号链接竞争和信息泄露。测试断言权限，并断言已存在的临时路径不会被覆盖；此原语是 seam 的常设要求。 - **通过符号链接的 `targetKey` 同一性。** 两个输入路径解析到同一 realpath 时共享一个观测状态条目：通过路径 A 的 `read` 满足通过符号链接路径 B 的 `edit` 的读后编辑守护，通过一个路径的陈旧写入可通过另一个路径检测到。 - **并发/陈旧竞争。** 对同一目标的两个并发写入/编辑操作确定性地收敛——一个成功，另一个被 `FS_STALE_VERSION` 拒绝——成功的编辑刷新记录状态，使同一 owner 的下一次编辑可以继续。 - **HMR（热模块替换）安全与 dispose（资源释放）。** dispose 后端的 fiber 会撤回 `ctx.fs` 提供方；后续的提供方以无继承状态启动。 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml index 11073946ea..afb9251d33 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md -2026-07-15-lsp-capability-seam.md: d96b3a9c5139c1455a51f4fff793293d7b5a11c0 -2026-07-15-lsp-capability-seam.zh.md: 256b293f213acb06588f3ef6c655242b1c8fd5b9 +2026-07-15-lsp-capability-seam.md: a9dab45ebf034f6d5218b25a0868a7e86803719b +2026-07-15-lsp-capability-seam.zh.md: b411a2456ea2a3838e56422f58954473da51ae23 diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md index d96b3a9c51..a9dab45ebf 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md @@ -79,7 +79,7 @@ interface LspService { Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's resolved workspace root so consumers relativize file URIs in the same canonical namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch. -`dsh-lsp-local` owns host files, server configuration, JSON-RPC, process and transient-document state, and protocol translation; it depends on `dsh-lsp` and Node APIs, not `dsh-fs`. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. +`dsh-lsp-local` owns server configuration, JSON-RPC, process and transient-document state, and protocol translation. It reads through `ctx.fs` and launches through `ctx.subprocess`, depending on their interface packages rather than concrete providers; the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) owns that pairing. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider. ## Model-facing contract @@ -112,26 +112,26 @@ Provider disposal occurs outside tool execution, so `dsh-lsp-local` keeps `shutd ## Workspace, filesystem, and document synchronization -`dsh-lsp-local` canonicalizes and reads through Node APIs in the subprocess's host namespace. It rejects missing, non-regular, non-UTF-8, oversized, or canonical out-of-workspace sources and keeps one `O_NOFOLLOW | O_NONBLOCK` handle through validation and reading, so a FIFO with no writer cannot block before the regular-file check. It observes caller cancellation around each filesystem operation. It does not consume `ctx.fs` or emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. +`dsh-lsp-local` canonicalizes and reads through `ctx.fs` in the language server's execution world. It requires the workspace target to be a directory, rejects out-of-workspace sources through provider-owned containment, and uses `readTextBounded` so regular-file validation, UTF-8 decoding, the byte ceiling, and path replacement/growth safety stay one filesystem operation. It observes caller cancellation around each provider operation. It does not emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy. The `read` tool is unsuitable source because its output is windowed, numbered, transcript-visible, and observed. Reading in `tool-lsp` would also assign provider-specific synchronization to the consumer and preclude non-local providers. The local provider uses a compatibility-first transient-open sequence for every query. It accepts legacy `textDocumentSync` `Full` or `Incremental`, or options with `openClose: true`; omitted, `None`, or explicitly incompatible synchronization fails as unsupported before `didOpen`. -1. Canonicalize and validate the host path, then read the current source with Node filesystem APIs. +1. Resolve and contain the source through `ctx.fs`, then read its current bounded text through the same provider. 2. Send `textDocument/didOpen` with version `1`, full text, and the configured language id. Its write remains abortable; failure or cancellation invalidates the instance and awaits bounded process termination before the pool can reuse it. 3. Send the requested `textDocument/definition`, `textDocument/references`, `textDocument/implementation`, or `textDocument/hover` request. 4. If `didOpen` succeeded, attempt `textDocument/didClose` in `finally` after the request settles or aborts. A close-write failure does not replace the settled result or error, but invalidates the instance and awaits bounded process termination. Documents close after each call, so the first version needs no `didChange`, `didSave`, content cache, mutation listener, or document LRU. One abortable per-workspace provider queue serializes source-read/open/query/close lifecycles, so a waiting query reads current bytes only when its turn starts; the instance also keeps protocol lifecycles serialized. Distinct workspaces may run in parallel. The server's workspace index remains responsible for closed files reached from the source. -The canonical workspace `realpath` must be a directory and supplies process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity; symlink aliases therefore share an instance. Result locations may be external, but an external path cannot become a query source. Remote, virtual, or independently sandboxed filesystems require another provider. +The canonical workspace target must be a directory. Its target key supplies pool identity, its process path supplies cwd, and its provider-owned `file:` URI supplies `rootUri` and the sole `workspaceFolders` entry; aliases share an instance when the filesystem provider resolves them to one key. Result locations may be external, but an external path cannot become a query source. A filesystem that cannot share paths with the mounted subprocess provider is a composition error, not a reason for another LSP package. ## Local server lifecycle and protocol behavior -`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace realpath)`. At load it resolves the executable after credential scrubbing and environment overrides, failing before registration if unavailable; server process launch stays lazy (first query spawns it) and uses no shell. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter. +`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace target)`. At load it calls `ctx.subprocess.resolveExecutable()` with the configured environment, failing before registration if unavailable; first query launches through raw protocol pipes with no shell and a bounded collected stderr tail. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter. -Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits. +Initialization uses `processId: null` because the client and server may inhabit different process namespaces. It advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits. Navigation maps `Location` directly and `LocationLink` from `targetUri` plus `targetSelectionRange`. Positions must be nonnegative integers. Hover normalization accepts only valid `MarkupContent` and `MarkedString` shapes, preserves string values, renders language-tagged values as fenced code, and joins arrays with one blank line. The model-facing tool applies `maxResultChars` after rendering. @@ -143,7 +143,7 @@ Symbols are deferred because they need different schemas and overlap read/search Diagnostics need separate freshness, accumulation, and transcript rules. Mutations such as rename, code actions, and formatting require separate tools with preview, permission, and write-policy integration. -The local provider trusts its configured server and claims no sandbox confinement. Supporting untrusted binaries requires a later process/filesystem contract for workspace reads plus private cache and temporary writes; restricted, remote, or virtual workspaces require another provider. +The provider trusts its configured server. Its filesystem visibility and process confinement are exactly those of the mounted execution world; LSP adds no independent sandbox policy. ## Alternatives considered @@ -157,7 +157,7 @@ The local provider trusts its configured server and claims no sandbox confinemen **Wrap the signal in a per-seam execution-context object.** Web passes a bare `AbortSignal`; wrapping this single field would add unexplained asymmetry. `query()` gains a context object only when another field requires it. -**Read through `ctx.fs` or the `read` tool.** This could mix the document with a server index from another filesystem namespace; tool output is also windowed, numbered, and observed. The host-local provider reads unobserved full text beside its subprocess. +**Read through the model-facing `read` tool.** Rejected because tool output is windowed, numbered, transcript-visible, and observed. The provider reads bounded full text directly through the same `ctx.fs` execution world used by its subprocess. **Keep documents open.** Mirroring edits requires version ownership, all-path `didChange`, HMR recovery, eviction, and stale-state rules. Transient opens avoid that MVP state machine. @@ -180,7 +180,7 @@ The local provider trusts its configured server and claims no sandbox confinemen - Synchronization tests pin UTF-16 negotiation and conversion, supported and rejected `textDocumentSync` forms, blocked and failed open writes, balanced transient open/close, close-write failure, and malformed-response rejection. - Timeout tests pin one `TOOL_TIMEOUT` budget, unclassified upstream cancellation, no hidden seam deadline, and bounded awaited teardown. - Lifecycle tests pin startup single-flight, complete-lifecycle serialization with fresh queued source reads, cross-workspace parallelism, abortable queues, crash replacement without replay, failed-stdin teardown, and quiescent disposal. -- Host-filesystem tests pin session-cwd requirements, relative and absolute source containment through symlinks, document validation, file/non-file URI rendering, unformatted source, and no `fs/observed` event. +- Filesystem-host tests pin session-cwd requirements, provider-owned containment and URI rendering, bounded document reads, unformatted source, and no `fs/observed` event. - A keyless pinned TypeScript real-server e2e exercises all four operations; runnable configuration uses the same explicit provider mapping. - Snapshots cover model-visible schema, prompt, results, and omissions; a built-artifact smoke test covers framing and cleanup. - Package and architecture docs cover configuration, security boundaries, and search/read guidance; the new `packages/lsp/` group is added to the AGENTS.md repository-layout block, the packages/README.md group table, and architecture.md in the same change. @@ -195,4 +195,4 @@ Extension ownership is exclusive within one runtime. Two providers cannot both c UTF-16 cursor columns are exact for the protocol but difficult for a model to count around non-BMP characters. Invalid or off-symbol positions may produce empty results, so error text and prompt examples must explain the coordinate convention without encouraging broad LSP use. -Direct Node access aligns the query snapshot with the server index but bypasses `ctx.fs` and its policy. Canonical containment rejects source files outside the workspace; a trusted server may still read the workspace and use caches. The first implementation therefore requires trusted host-local deployment and provides no sandbox guarantee. +The paired filesystem/subprocess providers align the query snapshot with the server index but do not make a trusted language server safe. Canonical containment rejects query sources outside the workspace; the server itself receives the execution world's configured authority and may read other paths or use caches. diff --git a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md index 256b293f21..b411a2456e 100644 --- a/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.zh.md @@ -1,4 +1,4 @@ -# Agent Note: LSP 能力 seam 与面向模型的查询工具 +# Agent Note: LSP 能力服务边界与面向模型的查询工具 Status: implemented @@ -14,7 +14,7 @@ harness 已具备文本搜索与文件读取能力，但二者都无法识别程 ## 决策 -将 LSP 建成由三个包组成的能力 seam，其中包含一个只读模型工具和一个通用本地提供方实现： +将 LSP 建成由三个包（package）组成的能力服务边界，其中包含一个只读模型工具和一个通用本地提供方实现： 1. `packages/lsp/lsp` 下的 `@deepseek-ai/dsh-lsp` 负责 `ctx.lsp`、提供方注册与选择、标准化请求与结果、执行控制，以及结构化 LSP 错误。 2. `packages/lsp/lsp-local` 下的 `@deepseek-ai/dsh-lsp-local` 将配置的 stdio 语言服务器适配到该服务边界。一个插件实例接收具名服务器表，并为每组命令及扩展名到语言 id 的映射注册一个隔离的提供方。 @@ -79,7 +79,7 @@ interface LspService { 映射键规范化为带前导点的小写扩展名，并按 `filePath` 的最后一个扩展名选择；语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明：提供方在内部执行该约束，本地映射设置 `context.includeDeclaration: true`，调用方不能配置。封闭结果联合将导航统一为位置，将 `hover` 统一为内容或 `null`；导航结果携带提供方解析后的工作区根目录，使消费方依据同一规范化根目录相对化文件 URI。服务边界不公开协议类型、进程或文档控制，也不提供通用请求逃生口。 -`dsh-lsp-local` 负责主机文件、服务器配置、JSON-RPC、进程与临时文档状态和协议转换；它依赖 `dsh-lsp` 与 Node API，不依赖 `dsh-fs`。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置；如果后续映射无效或发生冲突，插件会撤销此前的注册，并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`，通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区，其取值方式与文件系统工具一致，也不导入提供方。 +`dsh-lsp-local` 负责服务器配置、JSON-RPC、进程与临时文档状态和协议转换。它通过 `ctx.fs` 读取，通过 `ctx.subprocess` 启动，只依赖二者的接口包而非具体提供方；[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)负责定义这种配对。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置；如果后续映射无效或发生冲突，插件会撤销此前的注册，并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`，通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区，其取值方式与文件系统工具一致，也不导入提供方。 ## 面向模型的契约 @@ -98,9 +98,9 @@ interface LspToolInput { 工具必须从会话 `header.cwd` 取得 `workspaceRoot`，没有后备值；缺失时在查询或启动前以 `LSP_WORKSPACE_REQUIRED` 失败。本地提供方基于根目录解析相对路径并直接接受绝对路径；两种路径都会进行规范化，如果目标位于规范工作区外，则在启动前拒绝。 -位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径，在工作区外转换为绝对路径；其他 URI 保持原样。`maxLocations` 默认值为 `100`，并报告省略的条目；`maxResultChars` 默认值为 `16_000`，并将每个完整渲染结果（包括截断元数据）限制在该字符数内。空位置与 `null` hover 是成功的无结果响应；服务器载荷缺失或格式错误时，以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 +位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径，在工作区外转换为绝对路径；其他 URI 保持原样。`maxLocations` 默认值为 `100`，并报告省略的条目；`maxResultChars` 默认值为 `16_000`，并限制每个完整渲染结果，其中包括截断元数据。空位置与 `null` hover 是成功的无结果响应；服务器载荷缺失或格式错误时，以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 -与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`，`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character，跟随位置聚焦输入行，标题保留完整光标；展示器仍为纯函数。 +与传输方式无关的展示器使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`，`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character，跟随位置聚焦输入行，标题保留完整光标；展示保持纯函数。 ## 超时归属 @@ -112,26 +112,26 @@ interface LspToolInput { ## 工作区、文件系统与文档同步 -`dsh-lsp-local` 通过 Node API 在子进程所在的主机命名空间中规范化并读取文件。它拒绝缺失、非普通、非 UTF-8、超大或规范路径越出工作区的源文件，并在校验与读取期间保持同一个 `O_NOFOLLOW | O_NONBLOCK` 句柄，因此没有写入方的 FIFO 不会在普通文件校验前造成阻塞。它在每项文件系统操作前后检查调用方是否取消。它不使用 `ctx.fs` 或发送 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足写前读取策略。 +`dsh-lsp-local` 在语言服务器的执行环境中通过 `ctx.fs` 规范化并读取文件。它要求工作区目标是目录，使用提供方自有的 containment 拒绝工作区外的源文件，并通过 `readTextBounded` 把普通文件校验、UTF-8 解码、字节上限和路径替换／增长安全性保留在同一项文件系统操作中。它在每项提供方操作前后检查调用方是否取消。它不发送 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足写前读取策略。 `read` 工具的输出带窗口与行号，进入 transcript（文本记录）且已被观察，不适合作为源文件。在 `tool-lsp` 内读取还会把提供方专用同步职责交给消费方，并排除非本地提供方。 本地提供方对每次查询都采用兼容优先的临时打开流程。它接受旧式 `textDocumentSync` 的 `Full` 或 `Incremental`，也接受设置了 `openClose: true` 的选项；同步能力缺失、为 `None` 或明确不兼容时，在 `didOpen` 前以不支持错误失败。 -1. 规范化并校验主机路径，再使用 Node 文件系统 API 读取当前源文件。 +1. 通过 `ctx.fs` 解析源文件并检查其位于工作区内，再通过同一提供方对当前文本进行有界读取。 2. 发送 `textDocument/didOpen`，其中包含版本 `1`、完整文本和配置的语言 id。该写入仍可取消；写入失败或遭取消会使实例失效，并等待有界进程终止完成，池才能复用它。 3. 发送所请求的 `textDocument/definition`、`textDocument/references`、`textDocument/implementation` 或 `textDocument/hover` 请求。 4. 如果 `didOpen` 成功，则在请求完成或取消后于 `finally` 中尝试发送 `textDocument/didClose`。关闭写入失败不会覆盖已经确定的结果或错误，但会使实例失效，并等待有界进程终止完成。 每次调用后都关闭文档，因此第一版不需要 `didChange`、`didSave`、内容缓存、变更监听器或文档 LRU。每个工作区的提供方队列可取消，并串行执行源文件读取、打开、查询和关闭的完整生命周期，因此等待中的查询只在轮到它时才读取当前字节；实例也会串行执行协议生命周期。不同工作区可以并行。服务器工作区索引仍负责从源文件跳转到的已关闭文件。 -规范工作区 `realpath` 必须是目录，并用于进程 cwd、`rootUri`、唯一的 `workspaceFolders` 条目和进程池 identity；符号链接别名因此共享实例。结果位置可以在工作区外，但外部路径不能成为查询源。远程、虚拟或独立沙箱化文件系统需要另一种提供方。 +规范工作区目标必须是目录。其目标键提供进程池 identity，进程路径提供 cwd，归提供方所有的 `file:` URI 则提供 `rootUri` 和唯一的 `workspaceFolders` 条目；文件系统提供方将别名解析为同一键时，它们共享实例。结果位置可以在工作区外，但外部路径不能成为查询源。无法与挂载的子进程提供方共享路径的文件系统属于组合错误，不是另建 LSP 包的理由。 ## 本地服务器生命周期与协议行为 -`dsh-lsp-local` 按 `(provider id, canonical workspace realpath)` 懒启动一个服务器，并通过 single-flight 合并启动。插件加载时，它在清除凭据并应用环境变量覆盖后解析可执行文件；命令不可用时在注册前失败。服务器进程的启动保持懒执行（首次查询时才拉起），且不经过 shell。`maxMessageBytes` 默认值为 `16_000_000`，`maxStderrBytes` 默认值为 `1_000_000`，`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放；后续查询可以替换进程。每次查询最多启动一个进程，因此 MVP 不设置跨请求重启计数器。 +`dsh-lsp-local` 按 `(provider id, canonical workspace target)` 懒启动一个服务器，并通过 single-flight 合并启动。插件加载时，它使用已配置的环境调用 `ctx.subprocess.resolveExecutable()`；命令不可用时在注册前失败。首次查询通过原始协议管道启动服务器，不经过 shell，并收集有界的 stderr 尾部。`maxMessageBytes` 默认值为 `16_000_000`，`maxStderrBytes` 默认值为 `1_000_000`，`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放；后续查询可以替换进程。每次查询最多启动一个进程，因此 MVP 不设置跨请求重启计数器。 -初始化声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`，以及 definition 与 implementation 的 `linkSupport: true`，但不支持动态注册。服务器返回的操作能力与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`；其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应，但客户端拒绝 `workspace/applyEdit`，绝不执行命令或编辑。 +初始化使用 `processId: null`，因为客户端与服务器可能位于不同的进程命名空间。它声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`，以及 definition 与 implementation 的 `linkSupport: true`，但不支持动态注册。服务器返回的操作与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`；其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应，但客户端拒绝 `workspace/applyEdit`，绝不执行命令或编辑。 导航结果直接映射 `Location`，并将 `LocationLink` 的 `targetUri` 与 `targetSelectionRange` 映射为统一位置。位置必须是非负整数。`hover` 归一化只接受有效的 `MarkupContent` 和 `MarkedString` 结构，保留字符串值，把带语言标签的值渲染为围栏代码块，并以一个空行连接数组。面向模型的工具在渲染后应用 `maxResultChars`。 @@ -143,7 +143,7 @@ interface LspToolInput { 诊断需要独立的新鲜度、累积与 transcript 规则。重命名、代码操作和格式化等变更能力需要单独工具，并集成预览、权限和写入策略。 -本地提供方信任配置的服务器，不声称具备沙箱隔离。支持不受信任的二进制文件需要后续补充允许读取工作区，并执行私有缓存写入与临时写入的进程／文件系统契约；受限、远程或虚拟工作区需要另一种提供方。 +提供方信任配置的服务器。其文件系统可见性与进程隔离完全取决于挂载的执行环境；LSP 不增加独立的沙箱策略。 ## 备选方案 @@ -155,9 +155,9 @@ interface LspToolInput { **公开 `resolve(request)` / `query(spec)`。** 没有需要填充默认值的字段时，resolve 只会暴露提供方选择，而公开 spec 可能活过提供方释放或替换。单一操作让选择与调用共用注册生命周期。 -**将信号包装为服务边界专用的执行上下文对象。** Web 传递裸 `AbortSignal`；仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时，`query()` 才引入上下文对象。 +**将信号包装为每服务边界的执行上下文对象。** Web 传递裸 `AbortSignal`；仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时，`query()` 才引入上下文对象。 -**通过 `ctx.fs` 或 `read` 工具读取。** 这可能把文档与另一文件系统命名空间中的服务器索引混合；工具输出还带窗口、行号且已被观察。host-local 提供方在子进程旁读取未观察的完整文本。 +**通过面向模型的 `read` 工具读取。**拒绝，因为工具输出带窗口与行号，会进入 transcript 且已被观察。提供方直接通过子进程所用的同一 `ctx.fs` 执行环境读取有界的完整文本。 **保持文档打开。** 镜像编辑需要版本归属、覆盖所有路径的 `didChange`、HMR 恢复、淘汰和陈旧状态规则。临时打开避免在 MVP 引入这套状态机。 @@ -180,14 +180,14 @@ interface LspToolInput { - 同步测试固定 UTF-16 协商与转换、受支持和被拒绝的 `textDocumentSync` 形式、打开写入阻塞与失败、配对的临时打开/关闭、关闭写入失败和错误响应拒绝。 - 超时测试固定一个 `TOOL_TIMEOUT` 预算、不对上游取消错误分类、服务边界无隐藏截止时间，以及受限且等待完成的清理。 - 生命周期测试固定启动 single-flight、完整生命周期串行化及排队查询读取最新源文件、跨工作区并行、可取消队列、崩溃后不重放的替换、stdin 失败后的进程拆除，以及释放后完全停稳。 -- 主机文件系统测试固定 session cwd 要求、符号链接下相对与绝对源路径的规范 containment、文档校验、file/non-file URI 渲染、无格式源文本和不发送 `fs/observed`。 +- 文件系统宿主测试固定 session cwd 要求、提供方自有的 containment 与 URI 渲染、有界文档读取、无格式源文本和不发送 `fs/observed`。 - 无密钥且固定版本的 TypeScript 真实服务器 e2e 覆盖四种操作；可运行配置使用同一项显式提供方映射。 - 快照覆盖模型可见 schema、提示词、结果和省略提示；构建产物冒烟测试覆盖分帧与清理。 - 包与架构文档覆盖配置、安全边界和搜索/读取指导；同一改动中，新的 `packages/lsp/` 包组要加入 AGENTS.md 的仓库布局块、packages/README.md 的分组表和 architecture.md。 ## 影响 -各语言服务器对方法支持、能力解释和索引就绪时机的处理不同；LSP 没有统一的「索引完成」信号。不具备兼容临时打开同步能力的服务器不受支持，即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果，因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线，不代表跨语言承诺。 +各语言服务器对方法支持、能力解释和索引就绪时机的处理不同；LSP 没有统一的“索引完成”信号。无法声明兼容临时打开同步能力的服务器不受支持，即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果，因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线，不代表跨语言承诺。 临时打开会重复解析并产生通知。实例内串行会增加并发 agent 的延迟，长期运行的工作区进程则持续占用内存直到释放。 @@ -195,4 +195,4 @@ interface LspToolInput { UTF-16 光标列与协议完全一致，但模型难以在包含非 BMP 字符的文本中准确计数。无效位置或不在符号上的位置可能返回空结果，因此错误文本和提示词示例必须说明坐标约定，同时避免鼓励模型广泛使用 LSP。 -直接访问 Node 文件系统会对齐查询快照与服务器索引，但绕过 `ctx.fs` 及其策略。规范路径 containment 会拒绝工作区外的源文件；受信任的服务器仍可读取工作区并使用缓存。因此，第一版要求受信任的 host-local 部署，不提供沙箱保证。 +配对的文件系统／子进程提供方会对齐查询快照与服务器索引，但不会因此使受信任的语言服务器变得安全。规范 containment 会拒绝工作区外的查询源；服务器本身获得执行环境所配置的权限，仍可读取其他路径或使用缓存。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml index daa727ccb0..469c3f9a24 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md -2026-07-26-subprocess-consumer-migration.md: 477dffc2271db08b8986b34645067b247ad7ace7 -2026-07-26-subprocess-consumer-migration.zh.md: 8e0e377fc3fe00ff452803fe7fe5f4115003f937 +2026-07-26-subprocess-consumer-migration.md: abc7c8504595641b2b821df5ddbd4645c3a15336 +2026-07-26-subprocess-consumer-migration.zh.md: 500dcfe36f63315156b1a3cc48670eaf1c204b20 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md index 477dffc227..abc7c85045 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.md @@ -6,7 +6,7 @@ English | [中文](2026-07-26-subprocess-consumer-migration.zh.md) ## Problem -The [subprocess seam](2026-07-26-subprocess-seam.md) shipped shaped for exactly one consumer family: batch-collected stdout/stderr, batch stdin, a single escalating `kill()`. That was deliberate scope control, and its own note records "migrate the other spawn sites" as rejected-for-now. Review on the introducing PR reversed that deferral: the stacked follow-up should reshape the interface toward Node's API and move the remaining process-running places onto the service. The remaining spawners each carried a private copy of some slice of the same mechanics — lsp-local had its own detached-tree signalling (POSIX group + Windows taskkill + liveness polling), subagent-subprocess had the dispose ladder and its own scrub, and mcp-client, pty-local, the SDK helper, and the TUI Git probe each carried another credential scrub — and none of it was swappable or centrally testable. +The [subprocess seam](2026-07-26-subprocess-seam.md) shipped shaped for exactly one consumer family: batch-collected stdout/stderr, batch stdin, a single escalating `kill()`. That was deliberate scope control, and its own note records "migrate the other spawn sites" as rejected-for-now. Review on the introducing PR reversed that deferral: the stacked follow-up should reshape the interface toward Node's API and move the remaining process-running places onto the service. The remaining spawners each carried a private copy of some slice of the same mechanics — lsp-local had its own detached-tree signalling (POSIX group + Windows taskkill + liveness polling), subagent-subprocess had the dispose ladder and its own scrub, mcp-client and pty-local and the SDK helper each had a third/fourth/fifth copy of the credential scrub — and none of it was swappable or centrally testable. ## Decision @@ -15,24 +15,24 @@ The seam's vocabulary is now Node-shaped, and every spawner that can ride the se - **Per-stream stdio dispositions** on `SubprocessSpawnSpec`: `'pipe'` (the raw `Readable`/`Writable`, for consumer-owned protocol framing), `'inherit'` (diagnostics to the parent's stream), and collect mode `{ maxBytes, spill? }` — the original bounded tail-keep shape, with the spill file now optional so a diagnostic tail (a language server's stderr) buffers without touching disk. stdin is `'ignore'`, `'pipe'`, or `{ data }` (write-and-close batch). - **`SubprocessOutcome` carries exit facts only** (Node's close-event vocabulary); collected output stays readable through `handle.collected` after settlement (spill fds seal at the settle boundary), so batch and streaming callers share one access path and nothing is copied into the outcome. - **Tree-scoped termination behind one verb**: `terminate()` owns the SIGTERM→grace→SIGKILL escalation (serves the spec's abort signal too, and is a no-op once the tree is gone) — the handle exposes no single-signal `kill(signal?)`, so a consumer cannot skip the grace window; `waitForExit()` polls tree liveness (POSIX group probe; direct-child boundary on Windows); Windows tree termination (`taskkill /T`, injectable) moved in from lsp-local, so tree semantics are platform-correct for every consumer. (The stdin-EOF-first dispose ladder initially absorbed from `subagent-subprocess` later moved back out to its one consumer — see the [ladder-ownership Agent Note](2026-07-27-dispose-ladder-to-consumer.md).) -- **One scrub definition**: `scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` live on the seam, and credential-shaped names contain `KEY`, `PASSWORD`, `SECRET`, or `TOKEN`, case-insensitively. Spawners that cannot route the spawn itself through the service — pty-local (node-pty owns the fork), mcp-client (the MCP SDK owns the transport spawn), and the TUI's synchronous Git branch probe — import the function, so environment policy is single-sourced even where process ownership is not. The SDK helper's `scrubEnvironment()` defaults through the function and applies the exported pattern when its caller supplies an explicit environment; the pattern remains public for this production consumer. +- **One scrub definition**: `scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` live on the seam. Ordinary and terminal local spawns apply it inside `dsh-subprocess-local`; mcp-client still imports it because the MCP SDK owns that transport spawn, and the SDK helper's `scrubEnvironment()` defaults through it as well. -Migrations landed with the reshape: **bash-local/bash-sandbox** (collect modes + batch stdin; the bash `kill()` maps to `terminate()` so `task_kill` keeps escalation semantics), **lsp-local** (piped protocol streams + a no-spill collected stderr tail; `LspConnection` takes the seam's spawn function; its private tree-op helpers deleted), **subagent-acp** (piped ndjson streams + inherited stderr; spawn failure surfaces through `done` rejection into the same startup race; disposal is the backend-owned `disposeAcpChild` ladder over the seam's verbs, with the plugin's configured graces). **`dsh-subagent-subprocess` is deleted** — the dispose ladder and scrub are the seam's; the unused isolated-config-dir helper died with it (no consumer existed). +Migrations landed with the reshape: **bash-local/bash-sandbox** (collect modes + batch stdin; the bash `kill()` maps to `terminate()` so `task_kill` keeps escalation semantics), **lsp-local** (piped protocol streams + a no-spill collected stderr tail; `LspConnection` takes the seam's spawn function; its private tree-op helpers deleted), **subagent-acp** (piped ndjson streams + inherited stderr; spawn failure surfaces through `done` rejection into the same startup race; disposal is the backend-owned `disposeAcpChild` ladder over the seam's verbs, with the plugin's configured graces), and **pty-local** through the later [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) (`node-pty` allocation and process inspection sit behind `spawnTerminal()`, while readiness and terminal policy remain in the consumer). **`dsh-subagent-subprocess` is deleted** — the dispose ladder and scrub are the seam's; the unused isolated-config-dir helper died with it (no consumer existed). Compositions mounting lsp-local or subagent-acp now load `dsh-subprocess-local` (the plugins inject `'subprocess'`); the acp/lsp test fixtures gained the row. ## Alternatives considered -**Keep the batch-only seam and let stream consumers stay bespoke.** The introducing note's position, rejected by review: it leaves three private copies of tree signalling and six of the scrub, and any future runner (containerized executor, remote process host) would have to pick which private copy to fork. The Node-shaped dispositions cover all three observed stream shapes without widening the outcome type or buffering piped streams. +**Keep the batch-only seam and let stream consumers stay bespoke.** The introducing note's position, rejected by review: it leaves three private copies of tree signalling and five of the scrub, and any future runner (containerized executor, remote process host) would have to pick which private copy to fork. The Node-shaped dispositions cover all three observed stream shapes without widening the outcome type or buffering piped streams. **A single `stdio: 'pipe' | 'inherit' | 'collect'` mode for all three streams at once.** Rejected: real consumers mix modes per stream (lsp: pipe/pipe/collect; acp: pipe/pipe/inherit; bash: data/collect/collect). Per-stream dispositions are exactly Node's shape and avoid a second spawn call for the mixed cases. -**Migrate pty-local and mcp-client spawns too.** Rejected on ownership grounds, not scope: node-pty's `fork()` allocates the terminal itself, and the MCP SDK's `StdioClientTransport` spawns internally — neither call site is ours to route. They adopt the shared scrub (the part that is policy), and their READMEs say why the spawn stays put. +**Keep pty-local and mcp-client spawns outside the service.** The MCP SDK still owns its transport spawn. PTY allocation is different: the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md) moves `node-pty` behind one deep terminal primitive, resolving the ownership objection without pretending an ordinary piped spawn can provide terminal semantics. -**Migrate the test-support launchers (acp-snapshot, loader-smoke), the SDK package-manager runner, and the TUI Git probe.** Rejected: the support packages are deliberately dependency-light test infrastructure that must not depend on product seams; the SDK wizard's `stdio: 'inherit'`-with-redirect semantics plus its out-of-composition lifecycle (no cordis context at all) make the service a poor fit; and the TUI probe is synchronous. The production callers share the scrub instead. +**Migrate the test-support launchers (acp-snapshot, loader-smoke) and the SDK package-manager runner.** Rejected: the support packages are deliberately dependency-light test infrastructure that must not depend on product seams, and the SDK wizard's `stdio: 'inherit'`-with-redirect semantics plus its out-of-composition lifecycle (no cordis context at all) make the service a poor fit; it shares the scrub instead. ## Consequences -Bought: one implementation of tree signalling, escalation, bounded collection, and the scrub, tested once in `dsh-subprocess-local`'s suites (including injected-platform Windows coverage that lsp-local's private copy never had); lsp-local and subagent-acp shed their process plumbing and their children now survive plugin reloads and die with composition teardown like bash's; a whole package (`dsh-subagent-subprocess`) is gone. The seam README's "one consumer family" limitation is retired. +Bought: one implementation of tree signalling, escalation, bounded collection, terminal process mechanics, and the scrub, tested once in `dsh-subprocess-local`'s suites (including injected-platform Windows coverage that lsp-local's private copy never had); lsp-local, pty-local, and subagent-acp shed provider-specific process plumbing and their children die with composition teardown like bash's; a whole package (`dsh-subagent-subprocess`) is gone. The seam README's "one consumer family" limitation is retired. -Cost: the seam is wider — three stdio modes and the terminate/waitForExit/dispose lifecycle surface instead of one mode and one verb — so a future backend implements more surface; the compositions for lsp-local/subagent-acp each carry the subprocess row now; and `SubprocessOutcome` no longer carries output, a breaking shape change inside the still-unreleased stack (the PR2 layer was updated in place rather than shimmed, per the pre-release stance). pty-local/mcp-client/SDK/TUI/test-support spawns remain outside the service by ownership or execution shape, with the scrub as the shared floor and its pattern intentionally exported for explicit-environment consumers. +Cost: the seam is wider — execution-world coordinates, executable lookup, three stdio modes, process-tree lifecycle, and one terminal primitive — so a future backend implements more surface; the compositions for lsp-local, pty-local, and subagent-acp each carry the subprocess row; and `SubprocessOutcome` no longer carries output, a breaking shape change inside the still-unreleased stack. MCP/SDK/test-support spawns remain outside the service by ownership, with the scrub as the shared floor. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md index 8e0e377fc3..500dcfe36f 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-consumer-migration.zh.md @@ -1,4 +1,4 @@ -# Agent Note: 子进程 seam 转向 Node 形状，所有具备条件的 spawn 调用点一并迁入 +# Agent Note: 进程 seam 转向 Node 形状，所有具备条件的 spawn 调用点一并迁入 Status: implemented @@ -6,7 +6,7 @@ Status: implemented ## 问题 -[子进程 seam](2026-07-26-subprocess-seam.md) 交付时恰好只为一个消费方家族塑形：批量收集的 stdout/stderr、批量 stdin、单一的升级式 `kill()`。那是有意的范围控制，其自身的 Agent Note 也把「迁移其余 spawn 调用点」记为暂缓否决项。引入该 seam 的 PR（Pull Request）上的评审推翻了这一暂缓决定：堆叠其上的后续变更应当把接口向 Node 的 API 方向重塑，并把其余运行进程之处迁到该服务上。其余各 spawn 调用点此前各自持有同一套机制中某个切片的私有副本——lsp-local 自带 detached 进程树信号发送（POSIX 进程组 + Windows taskkill + 存活轮询），subagent-subprocess 自带 dispose（资源释放）阶梯和自己的凭据清除，mcp-client、pty-local、SDK helper 与 TUI Git 探测则各自持有另一份凭据清除——而这一切既不可替换，也无法集中测试。 +[进程 seam](2026-07-26-subprocess-seam.md) 交付时恰好只为一个消费方家族塑形：批量收集的 stdout/stderr、批量 stdin、单一的升级式 `kill()`。那是有意的范围控制，其自身的 Agent Note 也把「迁移其余 spawn 调用点」记为暂缓否决项。引入该 seam 的 PR（Pull Request）上的评审推翻了这一暂缓决定：堆叠其上的后续变更应当把接口向 Node 的 API 方向重塑，并把其余运行进程之处迁到该服务上。其余各 spawn 调用点此前各自持有同一套机制中某个切片的私有副本——lsp-local 自带 detached 进程树信号发送（POSIX 进程组 + Windows taskkill + 存活轮询），subagent-subprocess 自带 dispose（资源释放）阶梯和自己的凭据清除，mcp-client、pty-local 与 SDK helper 则各自持有凭据清除的第三、第四、第五份副本——而这一切既不可替换，也无法集中测试。 ## 决策 @@ -15,24 +15,24 @@ Status: implemented - **按流划分的 stdio 处置方式（disposition）**，位于 `SubprocessSpawnSpec` 上：`'pipe'`（原始的 `Readable`/`Writable`，供消费方自有的协议分帧使用）、`'inherit'`（诊断输出直通父进程的流），以及收集模式（collect）`{ maxBytes, spill? }`——即最初的有界尾部保留形状，只是 spill 文件改为可选，使诊断尾部（例如语言服务器的 stderr）无需落盘即可缓冲。stdin 则为 `'ignore'`、`'pipe'` 或 `{ data }`（写完即关闭的批量形式）。 - **`SubprocessOutcome` 只承载退出事实**（Node close 事件的词汇）；收集到的输出在结算后仍可经 `handle.collected` 读取（spill 文件描述符在结算边界封存），因此批量与流式调用方共用一条访问路径，也没有任何内容被复制进这份结果。 - **以进程树为范围的终止，集中在一个动词后面**：`terminate()` 拥有 SIGTERM→宽限期→SIGKILL 升级（也承接 spec 的 abort 信号，进程树消亡后为空操作）——句柄不暴露单信号的 `kill(signal?)`，因此消费方无法跳过宽限窗口；`waitForExit()` 轮询进程树存活状态（POSIX 进程组探测；Windows 上以直接子进程为界）。Windows 进程树终止（`taskkill /T`，可注入）自 lsp-local 迁入，因此每个消费方拿到的进程树语义在各平台上都正确。（最初从 `subagent-subprocess` 吸收的以 stdin EOF 打头的 dispose 阶梯，后来又移回其唯一消费方——见[阶梯归属 Agent Note](2026-07-27-dispose-ladder-to-consumer.md)。） -- **凭据清除只有一份定义**：`scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` 定义在 seam 上，且凭据形状的名称不区分大小写地包含 `KEY`、`PASSWORD`、`SECRET` 或 `TOKEN`。无法把 spawn 本身路由到该服务的调用点——pty-local（node-pty 拥有 fork）、mcp-client（MCP SDK 拥有传输层的 spawn）与 TUI 的同步 Git 分支探测——改为导入该函数，因此即便进程所有权无法统一，环境策略仍是单一来源。SDK helper 的 `scrubEnvironment()` 默认委托给该函数，并在调用方显式传入环境时应用导出的正则；该正则因此生产消费方而保持公开。 +- **凭据清除只有一份定义**：`scrubbedParentEnv()`/`SENSITIVE_ENV_PATTERN` 定义在 seam 上。普通本地 spawn 与终端本地 spawn 都在 `dsh-subprocess-local` 内部应用该定义；mcp-client 仍需导入它，因为 MCP SDK 拥有该传输层 spawn，SDK helper 的 `scrubEnvironment()` 默认同样委托给它。 -各项迁移随这次重塑一并落地：**bash-local/bash-sandbox**（收集模式 + 批量 stdin；bash 的 `kill()` 映射到 `terminate()`，因此 `task_kill` 保有升级语义），**lsp-local**（管道化的协议流 + 无 spill 的 stderr 收集尾部；`LspConnection` 改为接收 seam 的 spawn 函数；其私有的进程树操作辅助函数已删除），**subagent-acp**（管道化的 ndjson 流 + inherit 的 stderr；spawn 失败经 `done` 的 reject 汇入同一个启动竞态；dispose 是后端自有的 `disposeAcpChild` 阶梯，经由 seam 的动词运行，携带插件所配置的宽限期）。**`dsh-subagent-subprocess` 已删除**——dispose 阶梯与凭据清除归 seam 所有；无人使用的隔离配置目录辅助函数随之消亡（其消费方本就不存在）。 +各项迁移随这次重塑一并落地：**bash-local/bash-sandbox**（收集模式 + 批量 stdin；bash 的 `kill()` 映射到 `terminate()`，因此 `task_kill` 保有升级语义），**lsp-local**（管道化的协议流 + 无 spill 的 stderr 收集尾部；`LspConnection` 改为接收 seam 的 spawn 函数；其私有的进程树操作辅助函数已删除），**subagent-acp**（管道化的 ndjson 流 + inherit 的 stderr；spawn 失败经 `done` 的 reject 汇入同一个启动竞态；dispose 是后端自有的 `disposeAcpChild` 阶梯，经由 seam 的动词运行，携带插件所配置的宽限期），以及后来通过[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)迁移的 **pty-local**（`node-pty` 分配与进程检查位于 `spawnTerminal()` 之后，就绪状态和终端策略仍归消费方所有）。**`dsh-subagent-subprocess` 已删除**——dispose 阶梯与凭据清除归 seam 所有；无人使用的隔离配置目录辅助函数随之消亡（其消费方本就不存在）。 挂载 lsp-local 或 subagent-acp 的组合如今都加载 `dsh-subprocess-local`（这两个插件注入 `'subprocess'`）；acp/lsp 测试 fixture（测试前置数据）补上了这一行组合配置。 ## 曾考虑的替代方案 -**保持只支持批量的 seam，让流式消费方继续各自为政。**这正是引入该 seam 的 Agent Note 当初的立场，评审将其否决：这样会留下三份进程树信号发送的私有副本和六份凭据清除的私有副本，而未来任何运行器（容器化执行器、远程进程宿主）都得挑选去 fork 哪一份私有副本。Node 形状的处置方式覆盖已观察到的全部三种流形状，既不拓宽结果类型，也不缓冲管道化的流。 +**保持只支持批量的 seam，让流式消费方继续各自为政。**这正是引入该 seam 的 Agent Note 当初的立场，评审将其否决：这样会留下三份进程树信号发送的私有副本和五份凭据清除的私有副本，而未来任何运行器（容器化执行器、远程进程宿主）都得挑选去 fork 哪一份私有副本。Node 形状的处置方式覆盖已观察到的全部三种流形状，既不拓宽结果类型，也不缓冲管道化的流。 **用单个 `stdio: 'pipe' | 'inherit' | 'collect'` 模式一次性统辖全部三条流。**否决：真实消费方按流混用模式（lsp：pipe/pipe/collect；acp：pipe/pipe/inherit；bash：data/collect/collect）。按流划分的处置方式恰好就是 Node 的形状，也免去了混用场景的第二个 spawn 调用。 -**把 pty-local 与 mcp-client 的 spawn 也一并迁移。**基于所有权而非范围否决：node-pty 的 `fork()` 自行分配终端，MCP SDK 的 `StdioClientTransport` 在内部完成 spawn——这两处调用点都不归我们路由。它们采纳共享的凭据清除（那正是属于策略的部分），并在各自的 README 中说明 spawn 为何留在原地。 +**让 pty-local 与 mcp-client 的 spawn 留在服务之外。**MCP SDK 仍拥有其传输层 spawn。PTY 分配则不同：[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)把 `node-pty` 放到一个深层终端原语之后，既解决所有权异议，也不假装普通管道化 spawn 能够提供终端语义。 -**迁移 test-support 启动器（acp-snapshot、loader-smoke）、SDK package-manager 运行器与 TUI Git 探测。**否决：support 各包是刻意保持轻依赖的测试基础设施，不得依赖产品 seam；SDK 向导那套附带重定向的 `stdio: 'inherit'` 语义，加上其完全脱离组合的生命周期（根本没有 cordis 上下文），使该服务并不合用；TUI 探测则是同步调用。这些生产调用点改为共享凭据清除。 +**迁移 test-support 启动器（acp-snapshot、loader-smoke）与 SDK package-manager 运行器。**否决：support 各包（package）是刻意保持轻依赖的测试基础设施，不得依赖产品 seam；而 SDK 向导那套附带重定向的 `stdio: 'inherit'` 语义，加上其完全脱离组合的生命周期（根本没有 cordis 上下文），使该服务并不合用；它改为共享凭据清除。 ## 后果 -换来的是：进程树信号发送、升级、有界收集与凭据清除各自只剩一份实现，且只在 `dsh-subprocess-local` 的测试套件中测试一次（其中包括 lsp-local 的私有副本从未有过的、以注入平台方式实现的 Windows 覆盖）；lsp-local 与 subagent-acp 卸下了自己的进程管道，其子进程如今像 bash 的一样，在插件重载后存活、随组合拆除而终止；一个完整的包（`dsh-subagent-subprocess`）就此消失。seam README 中「只有一个消费方家族」的限制说明也随之退役。 +换来的是：进程树信号发送、升级、有界收集、终端进程机制与凭据清除各自只剩一份实现，且只在 `dsh-subprocess-local` 的测试套件中测试一次（其中包括 lsp-local 的私有副本从未有过的、以注入平台方式实现的 Windows 覆盖）；lsp-local、pty-local 与 subagent-acp 卸下了提供方专用的进程管道，其子进程如今像 bash 的一样随组合拆除而终止；一个完整的包（`dsh-subagent-subprocess`）就此消失。seam README 中「只有一个消费方家族」的限制说明也随之退役。 -代价是：这道 seam 变宽了（stdio 模式从一种变为三种、生命周期接口面从一个动词扩展为 terminate/waitForExit/dispose 这一组），未来的后端因此要实现更宽的接口面；lsp-local/subagent-acp 的各组合如今都多出 subprocess 这一行组合配置；`SubprocessOutcome` 也不再承载输出，这是仍未发布的堆叠变更内部的一次破坏性形状变更（依照预发布立场，PR2 那一层被就地更新，而非加 shim）。pty-local/mcp-client/SDK/TUI/test-support 的 spawn 因所有权归属或执行形状留在该服务之外，以凭据清除作为共底线，且为显式环境的生产消费方有意保持该正则导出。 +代价是：这道 seam 变宽了，涵盖执行环境坐标、可执行文件查找、三种 stdio 模式、进程树生命周期和一个终端原语，未来的后端因此要实现更宽的表面；lsp-local、pty-local 与 subagent-acp 的各组合如今都多出 subprocess 这一行组合配置；`SubprocessOutcome` 也不再承载输出，这是仍未发布的堆叠变更内部的一次破坏性形状变更。MCP／SDK／test-support 的 spawn 因所有权归属留在该服务之外，以凭据清除作为共享底线。 diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml index 1a918fc62c..94711572d6 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md -2026-07-26-subprocess-seam.md: ad2f8522be51ba16b0df155aeb334a88f493890f -2026-07-26-subprocess-seam.zh.md: 575c02e346531824ef409ddc6155aa2b59ce4e63 +2026-07-26-subprocess-seam.md: d5dee7361e48f18f3147444b2db74da3dd1c1161 +2026-07-26-subprocess-seam.zh.md: 884c572dc65c0fbf240f7e4e3b0387038e21f5ad diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md index ad2f8522be..d5dee7361e 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md @@ -12,8 +12,8 @@ English | [中文](2026-07-26-subprocess-seam.zh.md) A new `subprocess/` capability family owns "run and manage a process"; the bash family keeps "run a bash command" and consumes it: -- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess` with one method, `spawn(spec): SubprocessHandle`, and the shared vocabulary: the fully-explicit `SubprocessSpawnSpec` (argv, cwd, per-stream stdio dispositions, grace — no defaults; deployment-varying knobs stay with the calling seam's config, per the `dsh-bash` request/spec template and the no-hidden-defaults rule), `SubprocessHandle` with non-consuming offset-based readers, `SubprocessOutcome` with deliberately no timeout/cancel classification, and the shared scrub plus `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` types. `argv` is never shell-interpreted. (The [consumer-migration Agent Note](2026-07-26-subprocess-consumer-migration.md) later widened the stdio and termination vocabulary Node-ward.) -- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`): detached groups, tail-keep truncation with private bounded spill files, credential scrub with the explicit-env merge after it, group kill escalation, and disposal that kills and joins every still-running managed process. It has no config; every limit arrives on the spec. The terminal `ENV_OVERRIDES` (`TERM=dumb` etc.) did NOT move — that is bash-tool presentation policy and stays in `dsh-bash-local`, merged through the ordinary env channel. +- **`@deepseek-ai/dsh-subprocess` (interface)** — the abstract `SubprocessService` owning `ctx.subprocess`: execution-world cwd and runtime storage, executable lookup, fully explicit ordinary spawns, and the terminal primitive added by the [portable execution-world decision](2026-07-28-portable-execution-world-consumers.md). Its vocabulary includes per-stream stdio dispositions, process and terminal handles, exit facts with deliberately no timeout/cancel classification, and the shared scrub plus `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` types. `argv` is never shell-interpreted. +- **`@deepseek-ai/dsh-subprocess-local` (implementation)** — `LocalSubprocessService` over the former `run.ts` plumbing (`spawn.ts`) plus `node-pty`: detached groups, bounded collection and private spill files, executable lookup, private runtime storage, foreground/session inspection, credential scrub with explicit env merged after it, tree cleanup, and disposal that terminates and joins every managed process. It has no config; every limit arrives on the spec. Bash and PTY presentation environment overrides stay in their consumers. - **`dsh-bash-local` (consumer)** — `inject: ['subprocess']`; maps each resolved `BashExecSpec` onto a `SubprocessSpawnSpec` (`['bash', '-c', command]`), keeps its config, `resolve()` defaulting, fused-deadline `timedOut`/`aborted` classification, the `[stderr]`-marked background read merge with its consuming cursor, and the `onProcessDone` subclass hook. `dsh-bash-sandbox` is unchanged apart from redeclaring the inherited inject; it still wraps at the command-string level and re-enters the inherited spawn path. - **`dsh-bash` (seam)** — re-exports the moved vocabulary from `dsh-subprocess`, so no bash consumer changes an import; `BashExecRequest`/`BashExecSpec`/`BashProcess` and the sandbox facts remain bash-owned. @@ -25,7 +25,7 @@ Background-process lifetime moved from the executor to the subprocess service: t **Leave the process plumbing inside `dsh-bash-local` (status quo).** Rejected for the same reason the [task registry split](2026-07-26-task-registry-seam.md) landed: the boundary is stable and already documented in-code (`run.ts`'s module doc said "this layer reacts to an abort signal; the executor owns deadlines and classifies causes"), and keeping it private makes every future non-shell runner either fork the mechanics or depend on a bash-named package for non-bash work. The user-visible driver for this stack was exactly this split. -**Migrate the repo's other spawn sites (lsp-local, pty-local, subagent-subprocess, sdk package-manager, test-support launchers) onto `ctx.subprocess` in the same change.** Rejected as scope creep with real design risk at this PR's scale: those sites have materially different stream and lifecycle needs — node-pty ownership (pty), LSP framing over long-lived stdio with tree-kill fallbacks (lsp), stdin-EOF-first disposal ladders and no output buffering (subagent transports) — and forcing them under a handle shaped for bounded batch output would either bloat the seam or misfit the consumers. The seam shipped proven against its one real consumer family, per the shape-interfaces-around-current-consumers rule. Review then asked for exactly that follow-up as a stacked PR; the [consumer-migration Agent Note](2026-07-26-subprocess-consumer-migration.md) records the Node-ward reshape and which sites moved (and which stayed, by ownership). +**Migrate the repo's other spawn sites in the introducing change.** Rejected as scope creep with real design risk at that scale. The later [consumer-migration](2026-07-26-subprocess-consumer-migration.md) and [portable execution-world](2026-07-28-portable-execution-world-consumers.md) decisions reshape the interface around the observed LSP, subagent, PTY, and Code Runtime consumers; SDK and test-support launchers remain outside by ownership. **Put `run_in_background`/task semantics into the process seam instead.** Rejected: that boundary already exists — `ctx.tasks` owns ids, ownership, and notices, and the bash tool adapts a `BashProcess` into task hooks. The process seam sits *below* the bash executor, not beside the task registry. @@ -33,6 +33,6 @@ Background-process lifetime moved from the executor to the subprocess service: t ## Consequences -Bought: "run and manage a process" is a swappable capability with the standard three-package shape (consumer count starts at two: `bash-local`, `bash-sandbox`); a containerized or remote process backend slots in without touching bash semantics; the shared `DSH_*`/output vocabulary has a non-shell home; and background processes survive executor reloads, matching the task registry's lifetime model. The spawn plumbing suite moved wholesale to `dsh-subprocess-local` (argv-based, plus argv-validation and service lifecycle/disposal suites); the executor suite now pins the bash-owned layers (classification, merge, spawn-failure note, service-owned lifetime) against the real service. +Bought: "run and manage a process" is a swappable capability used by Bash, LSP, PTY, Code Runtime, and ACP consumers; a containerized or remote process backend slots in without changing their domain semantics; the shared `DSH_*`/output vocabulary has a non-shell home; and background processes survive executor reloads, matching the task registry's lifetime model. Process and terminal plumbing is tested through `dsh-subprocess-local`; consumer suites pin only their owned behavior against the real service. Cost: one more package pair and one more composition row everywhere a bash executor loads — a boot that loads an executor without the subprocess service leaves `ctx.bash` pending on `ctx.subprocess` (standard missing-service behavior). The moved-vocabulary re-exports keep `dsh-bash` imports working but mean two packages now name the same types; the subprocess seam is the owner and the bash seam documents the re-export. The spawn-failure note became single-delivery through the read path where the old plumbing retained it in the stderr buffer for repeated `readFrom(0)` reads — acceptable because the bash background read path was already a consuming cursor, and the note reaches the one reader that exists. diff --git a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md index 575c02e346..884c572dc6 100644 --- a/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.zh.md @@ -1,4 +1,4 @@ -# Agent Note: 子进程服务是 bash 执行器之下的独立 seam（`dsh-subprocess` / `dsh-subprocess-local`） +# Agent Note: 进程管理器是 bash 执行器之下的独立 seam（`dsh-subprocess` / `dsh-subprocess-local`） Status: implemented @@ -6,33 +6,33 @@ Status: implemented ## 问题 -`dsh-bash-local` 原先把两项因不同原因而变化的能力捆绑在一起：*运行一条 bash 命令*（命令默认值补全、超时分类、对模型友好的终端环境、bash 工具所渲染的 stdout/stderr 合并）与*运行并管理一个子进程*（detached 进程组、附带 spill 文件的有界尾部保留输出、凭据清除与 `DSH_*` 合并次序、SIGTERM→宽限期→SIGKILL 升级、先终止再等待退出的 dispose（资源释放））。进程这一半（`run.ts`）约占整个包的一半，却没有属于自己的 seam：未来的非 shell 运行器（直接执行 argv 的执行器、worker supervisor）将不得不重新实现这套机制，或者探入 bash 内部；而共享的 `DSH_*`/`CollectedOutput` 词汇则存放在一个名字承诺 shell 语义的包里。这种捆绑还把后台进程的存续期系在执行器的 fiber 上：重载 bash 执行器会杀死每一个存活的后台进程。这一点不同于同级的[任务注册表](2026-07-26-task-registry-seam.md)：后者的注册存续期刻意长于生产方 fiber。 +`dsh-bash-local` 原先把两项因不同原因而变化的能力捆绑在一起：*运行一条 bash 命令*（命令默认值补全、超时分类、对模型友好的终端环境、bash 工具所渲染的 stdout/stderr 合并）与*运行并管理一个子进程*（detached 进程组、附带 spill 文件的有界尾部保留输出、凭据清除与 `DSH_*` 合并次序、SIGTERM→宽限期→SIGKILL 升级、先终止再等待退出的 dispose（资源释放））。进程这一半（`run.ts`）约占整个包（package）的一半，却没有属于自己的 seam：未来的非 shell 运行器（直接执行 argv 的执行器、worker supervisor）将不得不重新实现这套机制，或者探入 bash 内部；而共享的 `DSH_*`/`CollectedOutput` 词汇则存放在一个名字承诺 shell 语义的包里。这种捆绑还把后台进程的存续期系在执行器的 fiber 上：重载 bash 执行器会杀死每一个存活的后台进程。这一点不同于兄弟的[任务注册表](2026-07-26-task-registry-seam.md)：后者的注册存续期刻意长于生产方 fiber。 ## 决策 新的 `subprocess/` 能力家族拥有「运行并管理一个进程」；bash 家族保留「运行一条 bash 命令」，并成为前者的消费方： -- **`@deepseek-ai/dsh-subprocess`（接口）**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`（仅一个方法：`spawn(spec): SubprocessHandle`），以及共享词汇：完全显式的 `SubprocessSpawnSpec`（argv、cwd、按流划分的 stdio 处置方式（disposition）、宽限期，一律不设默认值；随部署变化的旋钮依照 `dsh-bash` 的 request/spec 模板与无隐藏默认值规则，留在调用方 seam 的配置里）、携带基于偏移量的非消费式读取器的 `SubprocessHandle`、刻意不含超时/取消分类的 `SubprocessOutcome`，以及共享的凭据清除与 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` 类型。`argv` 绝不经过 shell 解释。（[消费方迁移 Agent Note](2026-07-26-subprocess-consumer-migration.md) 其后将 stdio 与终止词汇拓宽为 Node 形状。） -- **`@deepseek-ai/dsh-subprocess-local`（实现）**——`LocalSubprocessService`，构建在原 `run.ts` 管道（现为 `spawn.ts`）之上：detached 进程组、带私有有界 spill 文件的尾部保留截断、清除之后合并显式 env 的凭据清除、进程组 kill 升级，以及会终止每个仍在运行的受管进程并等待其退出的 dispose。该实现没有任何配置；每项限制都随 spec 到达。终端相关的 `ENV_OVERRIDES`（`TERM=dumb` 等）并未迁移：那是 bash 工具的呈现策略，留在 `dsh-bash-local` 里，经普通 env 通道合并。 +- **`@deepseek-ai/dsh-subprocess`（接口）**——拥有 `ctx.subprocess` 的抽象 `SubprocessService`：执行环境 cwd 与运行时存储、可执行文件查找、完全显式的普通 spawn，以及[可移植执行环境决策](2026-07-28-portable-execution-world-consumers.md)新增的终端原语。其词汇包括按流划分的 stdio 处置方式（disposition）、进程与终端句柄、刻意不含超时／取消分类的退出事实，以及共享的凭据清除与 `DSH_ENV_PREFIX`/`DshEnvironment`/`CollectedOutput` 类型。`argv` 绝不经过 shell 解释。 +- **`@deepseek-ai/dsh-subprocess-local`（实现）**——`LocalSubprocessService` 构建在原 `run.ts` 管道（现为 `spawn.ts`）与 `node-pty` 之上：detached 进程组、有界收集与私有 spill 文件、可执行文件查找、私有运行时存储、前台／会话检查、清除之后合并显式 env 的凭据清除、进程树清理，以及终止每个受管进程并等待其退出的 dispose。该实现没有任何配置；每项限制都随 spec 到达。Bash 与 PTY 的呈现环境覆盖仍归各自消费方所有。 - **`dsh-bash-local`（消费方）**——`inject: ['subprocess']`；把每个解析后的 `BashExecSpec` 映射为一个 `SubprocessSpawnSpec`（`['bash', '-c', command]`），并保留自身配置、`resolve()` 默认值补全、基于融合 deadline 的 `timedOut`/`aborted` 分类、带 `[stderr]` 标记的后台读取合并及其消费游标，以及 `onProcessDone` 子类钩子。`dsh-bash-sandbox` 除了重新声明继承来的 inject 之外没有变化；它仍在命令字符串层面做包装，并重新进入继承的 spawn 路径。 - **`dsh-bash`（seam）**——把迁走的词汇从 `dsh-subprocess` 重导出，因此没有任何 bash 消费方需要改动导入；`BashExecRequest`/`BashExecSpec`/`BashProcess` 与沙箱事实仍归 bash 所有。 如今，每个加载 bash 执行器的组合都同时加载 `@deepseek-ai/dsh-subprocess-local`：CLI（命令行界面）、各示例、Python 捆绑运行时、create-sdk 的 bash 功能资源，以及各内联测试配置。 -后台进程的存续期从执行器移到了子进程服务：执行器不再保有存活进程集合，于是重载执行器后，后台工作会继续运行且仍可读取，而组合拆除（服务的 dispose）仍是先终止再等待退出的边界。一条行为 seam 随之挪动：后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr（对一个从未真正运行的进程，服务会 reject `done`，且不缓冲任何内容），因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 +后台进程的存续期从执行器移到了管理器：执行器不再保有存活进程集合，于是重载执行器后，后台工作会继续运行且仍可读取，而组合拆除（管理器的 dispose）仍是先终止再等待退出的边界。一条行为 seam 随之挪动：后台 spawn 失败不再能在管道内部被缓冲成伪造的 stderr（对一个从未真正运行的进程，管理器会 reject `done`，且不缓冲任何内容），因此执行器把 `spawn failed: …` 提示注入恰好一个 `readOutput()` 增量。 ## 曾考虑的替代方案 **把进程管道留在 `dsh-bash-local` 里（维持现状）。**否决的理由与[任务注册表拆分](2026-07-26-task-registry-seam.md)得以落地的理由相同：这条边界既稳定，也早已记录在代码里（`run.ts` 的模块文档曾写明「this layer reacts to an abort signal; the executor owns deadlines and classifies causes」），而若继续将它保持私有，未来每个非 shell 运行器就只能要么 fork 这套机制，要么为非 bash 工作去依赖一个以 bash 命名的包。这组堆叠变更对用户可见的动因正是这一拆分。 -**在同一变更中把仓库其余 spawn 调用点（lsp-local、pty-local、subagent-subprocess、sdk package-manager、test-support 各启动器）迁到 `ctx.subprocess` 上。**在本 PR（Pull Request）的规模下，作为带有真实设计风险的范围蔓延否决。这些调用点在流与生命周期上的需求存在实质差异：node-pty 所有权（pty）、长生命周期 stdio 上的 LSP 分帧加进程树终止回退（lsp）、以 stdin EOF 打头的 dispose 阶梯和完全不缓冲输出（subagent 传输层）。把它们强行纳入一个按有界批量输出塑形的句柄之下，要么会让这道 seam 膨胀，要么会让句柄与消费方错配。依照「接口围绕当前消费方塑形」的规则，该 seam 当时在其唯一真实的消费方家族上得到验证后交付。评审随后恰恰要求以堆叠 PR 的形式完成这项后续工作；[消费方迁移 Agent Note](2026-07-26-subprocess-consumer-migration.md) 记录了向 Node 形状的重塑，以及哪些调用点迁入（哪些因所有权归属而留在原地）。 +**在引入 seam 的同一变更中迁移仓库其余 spawn 调用点。**否决，因为在当时的规模下属于带有真实设计风险的范围蔓延。后续的[消费方迁移](2026-07-26-subprocess-consumer-migration.md)与[可移植执行环境](2026-07-28-portable-execution-world-consumers.md)决策围绕已观察到的 LSP、subagent、PTY 与 Code Runtime 消费方重塑接口；SDK 与 test-support 启动器因所有权归属仍留在服务之外。 **改把 `run_in_background`/任务语义放进进程 seam。**否决：那条边界已经存在。`ctx.tasks` 拥有 id、所有权与通知，bash 工具则把 `BashProcess` 适配成任务钩子。进程 seam 位于 bash 执行器*之下*，而不是与任务注册表并列。 -**把 `ENV_OVERRIDES`（TERM=dumb、PAGER=cat 等）移入子进程服务。**否决：通用子进程服务不得把终端呈现策略强加给非终端消费方；对环境中凭据形态名称与 `DSH_*` 名称的清除是安全与身份不变式，予以保留，但终端友好性是 bash 工具自己的选择，经 spec 的显式 env 表达，而调用方自己的条目依旧优先。 +**把 `ENV_OVERRIDES`（TERM=dumb、PAGER=cat 等）移入管理器。**否决：通用进程管理器不得把终端呈现策略强加给非终端消费方；对环境中凭据形态名称与 `DSH_*` 名称的清除是安全与身份不变式，予以保留，但终端友好性是 bash 工具自己的选择，经 spec 的显式 env 表达，而调用方自己的条目依旧优先。 ## 后果 -换来的是：「运行并管理一个进程」成为一项具备标准三包形态的可替换能力（消费方起步就有两个：`bash-local`、`bash-sandbox`）；容器化或远程进程后端可以直接接入，而不触碰 bash 语义；共享的 `DSH_*`/输出词汇有了一个不带 shell 含义的归属；后台进程也能在执行器重载后存活，与任务注册表的存续期模型一致。spawn 管道测试套件整体迁至 `dsh-subprocess-local`（现以 argv 为基础，外加 argv 校验与服务生命周期/dispose 套件）；执行器测试套件如今以真实服务为基准，固定 bash 自有的各层行为（分类、合并、spawn 失败提示、归服务所有的存续期）。 +换来的是：「运行并管理一个进程」成为 Bash、LSP、PTY、Code Runtime 与 ACP（Agent Client Protocol）消费方共用的可替换能力；容器化或远程进程后端可以直接接入，而无需改变各领域语义；共享的 `DSH_*`/输出词汇有了一个不带 shell 含义的归属；后台进程也能在执行器重载后存活，与任务注册表的存续期模型一致。进程与终端管道通过 `dsh-subprocess-local` 测试；消费方测试套件只需针对真实服务固定各自拥有的行为。 -代价是：多出一对包，而且凡加载 bash 执行器之处都多一行组合配置。若某次启动加载了执行器却没有加载子进程服务，`ctx.bash` 会因等待 `ctx.subprocess` 而保持挂起（标准的服务缺失行为）。迁移词汇的重导出让 `dsh-bash` 的导入继续可用，但也意味着两个包如今命名同一批类型；进程 seam 是所有者，bash seam 则记录这层重导出。spawn 失败提示经由读取路径变为单次交付，而旧管道曾把它保留在 stderr 缓冲区里，供重复的 `readFrom(0)` 读取；这一点可以接受，因为 bash 的后台读取路径本就是消费游标，该提示能到达唯一存在的那个读取方。 +代价是：多出一对包，而且凡加载 bash 执行器之处都多一行组合配置。若某次启动加载了执行器却没有加载管理器，`ctx.bash` 会因等待 `ctx.subprocess` 而保持挂起（标准的服务缺失行为）。迁移词汇的重导出让 `dsh-bash` 的导入继续可用，但也意味着两个包如今命名同一批类型；进程 seam 是所有者，bash seam 则记录这层重导出。spawn 失败提示经由读取路径变为单次交付，而旧管道曾把它保留在 stderr 缓冲区里，供重复的 `readFrom(0)` 读取；这一点可以接受，因为 bash 的后台读取路径本就是消费游标，该提示能到达唯一存在的那个读取方。 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml new file mode 100644 index 0000000000..8b4288edb4 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md +2026-07-28-portable-execution-world-consumers.md: 000fa1b98964eb85550d0c607281937f8d7932c6 +2026-07-28-portable-execution-world-consumers.zh.md: 94e6bfcf94e450fafea2b33e152384da400e4b63 diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md new file mode 100644 index 0000000000..000fa1b989 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md @@ -0,0 +1,48 @@ +# Agent Note: Portable consumers over filesystem and subprocess execution worlds + +Status: implemented + +English | [中文](2026-07-28-portable-execution-world-consumers.zh.md) + +## Problem + +The filesystem and subprocess seams made file and ordinary process access replaceable, but several higher capabilities still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY, LSP, and Code Runtime packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations. + +Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract. + +## Decision + +`ctx.fs` and `ctx.subprocess` together define one execution world. Providers mounted together must describe the same path namespace, executables, processes, and terminal sessions; higher capabilities consume those two interfaces rather than name the provider. + +The filesystem interface owns the path facts that another capability needs without exposing its opaque target identity: a canonical process path, canonical `file:` URI, containment, and a bounded stable-handle text read. The existing text and mutation operations remain filesystem-owned. + +The subprocess interface owns the process coordinates and primitives: canonical cwd, private runtime storage, executable lookup, ordinary raw or collected process spawning, and `spawnTerminal()`. The terminal operation is one deep primitive whose handle owns byte I/O, foreground groups, signalling, TERM-to-KILL session cleanup, and a quiescence wait. Prompt detection, idle inference, scrollback, sandbox policy, and owner lifecycle remain in the PTY consumer. + +Generic consumers use that execution world: + +- `dsh-bash-local` continues to map Bash semantics onto ordinary `ctx.subprocess.spawn()`. +- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and sends provider-owned file URIs. Its JSON-RPC, pooling, synchronization, cancellation, and normalization stay unchanged. +- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. +- `dsh-code-runtime-subprocess` materializes a dependency-free runner through `ctx.fs` and launches it through `ctx.subprocess`, preserving the Code Runtime binding and output contract across local or remote worlds. It shares host-side worker mechanics through the non-plugin `dsh-code-runtime-worker/runtime-host` subpath instead of copying them. + +`dsh-code-runtime-worker` remains a separate implementation. It is the smaller in-process backend and works in single-file distributions that cannot assume an installed Node executable. Remote filesystem/process compositions select `dsh-code-runtime-subprocess`; they do not need a provider-specific Code Runtime package. + +## Alternatives considered + +**Keep one PTY, LSP, and Code Runtime package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it. + +**Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches. + +**Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means. + +**Delete the worker-thread Code Runtime.** Rejected because portability does not erase its current deployment need. The subprocess backend requires a Node executable and filesystem materialization; the worker backend has neither requirement and remains the supported single-process path. + +**Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop. + +## Consequences + +A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, LSP, and subprocess Code Runtime compose above them, so fixes to those capabilities remain provider-neutral. + +The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers. + +The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal still waits for exact PID-identity-fenced descendants and the top-level terminal process to reach quiescence. diff --git a/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md new file mode 100644 index 0000000000..94e6bfcf94 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.zh.md @@ -0,0 +1,48 @@ +# Agent Note: 基于文件系统与进程管理执行世界的可移植消费方 + +Status: implemented + +[English](2026-07-28-portable-execution-world-consumers.md) | 中文 + +## 问题 + +文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性，但若干上层能力仍直接调用宿主 Node API。因此，即使领域行为没有变化，远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包（package）。这些包只会成为浅层适配器：每个包都仅为替换文件与进程操作而复制一个现有消费方。 + +普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号，以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作，最终不是泄漏提供方内部细节，就是削弱其生命周期契约。 + +## 决策 + +`ctx.fs` 与 `ctx.subprocess` 共同定义一个执行世界。共同挂载的提供方必须描述相同的路径命名空间、可执行文件、进程和终端会话；上层能力消费这两个接口，而不引用具体提供方。 + +文件系统接口负责其他能力需要的路径事实，同时不公开其不透明目标身份：规范化进程路径、规范化 `file:` URI、包含关系，以及通过稳定句柄执行的有界文本读取。现有文本与变更操作仍归文件系统负责。 + +进程管理接口负责进程运行坐标与原语：规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程，以及 `spawnTerminal()`。终端操作是一项深层原语，其句柄负责字节 I/O、前台进程组管理、信号发送、TERM→KILL 会话清理以及等待完全停稳。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。 + +通用消费方使用该执行世界： + +- `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。 +- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系，通过 `ctx.subprocess` 解析和启动语言服务器，并发送由提供方负责的文件 URI。其 JSON-RPC、池化、同步、取消和规范化保持不变。 +- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`；其他进程管理提供方则提供相同原语。 +- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner，并通过 `ctx.subprocess` 启动它，从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制，而不是复制这些机制。 + +`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端，可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统／进程组合选择 `dsh-code-runtime-subprocess`；它们不需要提供方专用的代码运行时包。 + +## 考虑过的替代方案 + +**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳，因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题：删除这些适配器不应使领域行为散落到远程提供方中；通用消费方本已负责这些行为。 + +**把终端建模为普通的管道子进程。** 不予采纳，因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小，也更能如实表达契约。 + +**把 PTY 就绪判断与会话策略移入进程管理服务。** 不予采纳，因为这些属于持久终端消费方的语义，而非 OS 进程机制。进程管理提供方负责只有其执行基底才能完成的操作；`dsh-pty-local` 负责 Harness 终端的语义。 + +**删除 worker 线程代码运行时。** 不予采纳，因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化，而 worker 后端两者都不需要，并且仍是受支持的单进程路径。 + +**在远程环境中运行整个 harness。** 不予采纳，因为这是另一种部署模型。让执行能力可移植，并不意味着移动模型调用、会话状态、插件状态或 agent loop（智能体循环）。 + +## 后果 + +远程执行提供方只需实现共享沙箱所有者，以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上，因此对这些能力的修复仍与提供方无关。 + +基础接口更宽，一对文件系统／进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制；模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。 + +本地实现承接 `node-pty` 和平台进程检查，因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸：dispose（资源释放）仍会等待受精确 PID 身份校验保护的后代进程和顶层终端进程完全停稳。 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml index bc05e497c5..a878942993 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md -2026-06-15-code-mode.md: 99bbed3edab32512f88ece9694d6519a1f89c2dd -2026-06-15-code-mode.zh.md: ca1bbe9ed3e412186763d1ed4fca9ed06669d4c3 +2026-06-15-code-mode.md: 7e51b9fa726afe4c34b87457b562b4092ee6c93c +2026-06-15-code-mode.zh.md: 4ba098d87d8c944e88c4cbe11ff78ac4383d3ac5 diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.md index 99bbed3eda..7e51b9fa72 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.md @@ -6,7 +6,7 @@ English | [中文](2026-06-15-code-mode.zh.md) ## Problem -In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and at the time of this note the loop dispatched each call through `ctx.tools.execute()` **sequentially** (parallel tool execution was an open TODO then; bounded parallel dispatch has since shipped — the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md), the rolling pool in [docs/architecture.md](../../../../docs/architecture.md)) — with **every** intermediate `tool-result` re-entering the model's context on the next request. +In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../../docs/architecture.md)), with **every** intermediate `tool-result` re-entering the model's context on the next request. For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each round-trip drags the entire intermediate result back into context whether the model needs it or not. @@ -20,7 +20,7 @@ Three decisions, each elaborated in its own section below: 1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (the default, contributing the visible capability schemas), `'code'` (the registry contributes only its reserved `run_code` transport plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas and the transport + SDK). The registry shapes its canonical contribution at the source; the cooperative prompt-assembly result remains authoritative, and the logged request header records exactly that returned presentation. 2. **Code execution is a capability seam** — `packages/code-runtime/` contains the interface package `@deepseek-ai/dsh-code-runtime`, which owns `ctx.codeRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another implementation package, not a redesign. -3. **The shipped implementation is `@deepseek-ai/dsh-code-runtime-worker`**: one fresh Node worker thread per run, executing the model's TypeScript after type-strip, with bindings bridged over the message port, an empty environment, configurable heap/output/time caps, and hard termination. Its trust posture is bash-equivalent by design — no unsafe-acknowledgement flags — because the harness already ships `dsh-bash-local`, which executes arbitrary model-written shell commands with strictly *more* ambient authority. +3. **Two implementations preserve one fresh-worker contract**: `@deepseek-ai/dsh-code-runtime-worker` runs the worker in the harness process, while `@deepseek-ai/dsh-code-runtime-subprocess` materializes a runner through `ctx.fs` and launches it through `ctx.subprocess` for another execution world. Both execute host-stripped TypeScript in a fresh Node worker with an empty environment, bridged bindings, configurable heap/output/time caps, and hard termination. Their trust posture is bash-equivalent by design; stronger isolation comes from the mounted execution world. This note owns Code Mode's presentation, composition, isolation, and settlement foundation. The later [typed tool-return Agent Note](2026-07-20-code-mode-typed-tool-returns.md) owns the generated output map, canonical binding values, `ToolCallError`, and the lossless outer-output boundary. @@ -32,7 +32,7 @@ This note owns Code Mode's presentation, composition, isolation, and settlement **Interaction with `toolOrder`, stated up front:** a configured `systemPrompt.toolOrder` naming native capabilities rejects every assembly under `mode: 'code'`, because those names are outside that mode's wire-validation universe. This is correct behavior, not a bug: a deployment using Code Mode updates its order config or drops it. -**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders the loaded runtime's language declarations plus fixed usage instructions for the scope's visible capabilities (TypeScript by default; the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md) added Python and the `ctx.codeRuntime.language` renderer table). It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. +**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders TypeScript declarations plus fixed usage instructions for the scope's visible capabilities. It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output. **Assembly ownership.** `run_code` and `tools:sdk` enter the trusted `system-prompt/assemble` waterfall as normal assembly inputs. A scoped `tools:sdk` section may shadow the global default before dispatch, and a listener may remove or replace either contribution. The waterfall's returned assembly is final, so whoever changes these inputs owns preserving a viable Code Mode protocol when the deployment expects Code Mode to remain usable; no restoration pass overrides deliberate composition. @@ -48,7 +48,7 @@ Under `'code'` and `'both'` the registry owns `run_code` as a reserved presentat **Sub-call contexts are deferred through the parent.** Injecting inside `run_code` would break parent call/result adjacency, so `ToolRunContext.deferContext()` collects every sub-result `additionalContexts` entry in dispatch order. The registry carries that array even when the program later throws, and the loop appends each entry only after the outer result and every sibling result in the step. An outer post-execute block discards tool-deferred entries and exposes only contexts explicitly attached by the blocking decision. -**Concurrency is bounded, not serialized.** Each run owns a dispatch queue that starts calls strictly in submission order and classifies each one through `registry.executionMode`, the same fail-closed `isConcurrencySafe` contract the native loop uses. Consecutive parallel-classified calls overlap up to `maxParallelSubCalls` (default 10; `1` restores serial dispatch); an exclusive call drains the pool and runs alone. Settlement abandons queued calls that have not started. This note shipped the serialized placeholder; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler that replaced it. +**Concurrency is serialized.** Each run owns a dispatch queue, so even `Promise.all` executes tool calls in submission order. Settlement abandons queued calls that have not started. Parallelism requires per-tool concurrency-safety metadata. **Presentation.** `run_code`'s render intent is decided here per the [render-intent Agent Note](../architecture/2026-07-02-tool-render-intent-union.md): `presentCall` creates a `generic` card with `kind: 'execute'`, the program text as its title, and the same program text as `rawInput`; `run_code` intentionally declares no `presentResult`, so the TUI and host/client runtime (Web) complete that card through their generic raw-content fallback using the final durable `tool/result.content`, including captured logs plus the returned value, failure, or post-policy spill preview. This is not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not. See the [result-card completeness note](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md). @@ -64,7 +64,7 @@ Each sub-dispatch appends a log-only `tool/code-dispatch-start` event at pool en - `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }` — the runtime exposes each namespace as a global object of async functions inside the program; the optional descriptor asks the runtime to inject a real program-visible rejection class without teaching the seam consumer-specific names. `CodeJsonValue` is this dependency-light seam's structural lossless-JSON type, so binding arguments and resolutions cross the implementation's serialization boundary whole. - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }` — program execution outcomes resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary. - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout, a lossy completion is not an overflow, and a substrate exit is none of them. -- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the first backend; a Python backend says `'python'` and pairs with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` accepts any `language` with a registered SDK renderer and `run_code` flavor (TypeScript and Python ship; see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)) and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). +- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for both shipped backends; a Python backend would pair with its own SDK generator) and `isolation` (`'worker-thread'` for both shipped backends; the subprocess provider may add a container boundary). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all). Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator. @@ -79,21 +79,23 @@ Requests contain every runtime input; implementations own validated timeout and 5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures. 6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md). +`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across a filesystem/subprocess execution world. It writes a dependency-free runner below `ctx.subprocess.runtimeRoot`, resolves Node through the provider, and carries binding traffic over bounded base64 JSON frames on raw pipes. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path. + ### Trust posture -The worker runtime provides containment, not a security boundary: model code can reach Node APIs and has authority comparable to the bash tool. `worker.terminate()` stops the thread but not OS processes it spawned. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary need a container-class backend for both code and bash; the runtime's isolation descriptor lets them distinguish that backend. +The runtimes provide containment, not an independent security boundary: model code can reach Node APIs and has the authority of its mounted execution world. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty worker environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary mount container-class filesystem/subprocess providers for both code and bash. ### What the model sees -The SDK instructs the model to write an async body in the loaded runtime's language (an erasable-TypeScript body by default; a Python `async` body under a Python runtime — see the [language-dispatch note](2026-07-31-code-mode-language-dispatch.md)), call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Both flavors state the same contract in their own primitive: independent read-only calls MAY overlap under `Promise.all` (TypeScript) or `asyncio.gather` (Python), mutating calls run alone in submission order, and dependent work sequences with `await`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. +The SDK instructs the model to write an async erasable-TypeScript body, call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Calls remain sequential even under `Promise.all`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching. ## Consequences -Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch starts in submission order under a bounded overlap pool, while per-call contexts retain their source, envelope, and metadata through the outer result. +Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch remains serialized, while per-call contexts retain their source, envelope, and metadata through the outer result. ## Testing -- **Worker runtime:** Real-worker tests cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. A built-package test runs the worker entry under plain Node. +- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. Built-package tests run both the direct worker entry and the filesystem/subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test. - **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup. - **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior. - **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards. @@ -106,7 +108,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Result elision / summarization over native tool-calling.** Addresses only the context-bloat half of the problem: trimming old `tool-result`s is cheap to add as a logged surface replacement under reconstructable requests, but still pays one model round-trip per call and cannot express loops, branches, or joins. Complementary, not competing; it can layer under Code Mode for residual native calls. -**Parallel native dispatch in the loop.** The other answer to round-trip cost at decision time; it was blocked on concurrency-safety metadata and offers no composition either way — it parallelizes calls the model already decided on in one step. Code Mode's queue decision kept the two compatible, and that is how it played out: the metadata landed as `isConcurrencySafe` (the [parallel tool-call note](2026-07-10-parallel-tool-call-execution.md)), and native rolling-pool dispatch and per-tool binding parallelism unlocked on the same classifier. +**Parallel native dispatch in the loop.** The other answer to round-trip cost; still valid future work (the open TODO), still blocked on concurrency-safety metadata, and still no composition — it parallelizes calls the model already decided on in one step. Code Mode's serialized-queue decision keeps the two compatible: when the metadata lands, both native parallel dispatch and per-tool binding parallelism unlock together. **Always-exclusive (Cloudflare-faithful, no mode).** Rejected for this SDK's primary consumer: a coding agent's bread-and-butter single calls (`bash`, `read`, `edit`) are already ideal as native calls, and forcing every edit through a program taxes the common case. The mode config keeps the faithful form (`'code'`) one line away without imposing it. @@ -118,7 +120,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem ## Risks -**The worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool, containment exceeds it, gating uses the same seams. Deployments needing more need a future `isolation: 'container'` backend — tracked as the seam's designed extension, not a TODO on this design. +**A worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool and gating uses the same seams. The subprocess implementation can run inside a container-class execution world, but its worker descriptor does not itself claim that boundary. **`stripTypeScriptTypes` is marked experimental.** It is the same engine (amaro/swc) behind Node's own native `.ts` execution, exposed as an API across this repo's whole engines range. Mitigations: the runtime's unit suite pins the behaviors relied on (position preservation, erasable-only rejection message shape loosely), the call sits behind one private function, and `amaro`/`sucrase` are drop-in replacements if the API shifts. The erasable-only subset is a model-facing contract line, and the error path is a working feedback loop, not a dead end. @@ -126,8 +128,8 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem **Registry scope growth.** `dsh-tools` absorbs codegen, a tool, a bridge, and an event. Contained by module boundaries inside the package (`ts-types.ts`, `code-mode.ts` beside `schema.ts`/`json-schema.ts`/`presentation.ts`) and by the seam: everything substrate-shaped lives behind `ctx.codeRuntime`. -**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The runtime validates both sides of the worker port and applies no per-binding byte cap; structured-clone cost and process or worker memory are the practical bounds. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is the only byte-capped boundary. +**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The direct worker runtime applies no per-binding byte cap; the subprocess runtime bounds each transport frame with `maxFrameBytes`, but repeated or concurrent calls can still consume process or worker memory. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is separately bounded by `maxOutputBytes`. -**Sub-dispatch overlap is bounded by tool safety claims, not by the caller.** A program's `Promise.all` or `asyncio.gather` buys wall-clock parallelism only across calls the tool itself classifies concurrency-safe; a run of exclusive calls still costs its round-trips in sequence, and models may over-expect. Both flavors' SDK instructions state the real contract. This note shipped the serialized placeholder that made the risk absolute; the [live-parallel Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) owns the scheduler and its overlap cap. +**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs. **Budget metering reads the event loop, not a flag.** Busy-time polling (`eventLoopUtilization()`) is coarser than an exact CPU meter — a budget expires up to one poll interval late — and its correctness claim ("a pending dispatch cannot pause it") is load-bearing against a hostile program. Both sides are unit-tested (hot loop with a pending decoy dispatch dies at `computeMs`; idle-on-slow-binding survives to `maxWallMs`), and the poll interval is an internal constant, not config — nothing a deployment could mis-tune into a bypass. `maxWallMs` is config, and it reaches `setTimeout`, which clamps a delay above `MAX_TIMER_DELAY_MS` (2^31-1 ms) to 1 ms; a positivity check alone therefore accepts a 25-day ceiling that expires on the first tick and times out every run. The worker runtime range-checks the field at load for that reason. `computeMs` needs no upper bound because it is compared against measured utilization instead of being handed to a timer. diff --git a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md index ca1bbe9ed3..4ba098d87d 100644 --- a/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md +++ b/.agents/notes/implemented/feature/2026-06-15-code-mode.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -在注册表的原生呈现方式下，agent loop（智能体循环）将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装，组装结果中的 `tools` 落到协议格式（wire format）上（也记录在请求头日志中），模型每步调用一个 `tool-call` 块，而在本 note 写作时，循环通过 `ctx.tools.execute()` **逐个**分发每次调用（并行工具执行当时还是 open TODO；此后有界的并行分发已经交付——见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md)，以及 [docs/architecture.md](../../../../docs/architecture.md) 中的 rolling pool）——且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 +在注册表的原生呈现方式下，agent loop（智能体循环）将每个可见能力以 JSON Schema 函数定义的形式通告给模型。`ToolRegistry` 将其 schema 贡献给系统提示词组装，组装结果中的 `tools` 落到协议格式（wire format）上（也记录在请求头日志中），模型每步调用一个 `tool-call` 块，循环通过 `ctx.tools.execute()` **逐个**分发每次调用（并行工具执行是 `dsh-tools` 和 [docs/architecture.md](../../../../docs/architecture.md) 中明确标注的 open TODO），且**每一个**中间 `tool-result` 都会在下一次请求时重新进入模型上下文。 对于多步工具操作，这种方式 token 开销大且串行。模型无法组合工具——遍历结果集、根据中间值分支、扇出、后处理——每次调用都需要一次完整的模型往返，而每次往返都会把完整的中间结果拖回上下文，不管模型是否需要。 @@ -19,8 +19,8 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 三项决策，各自在下方独立小节中展开： 1. **Code Mode 是 `ToolRegistry`（`dsh-tools`）的一等呈现模式**，通过经校验的 `mode` 配置选择：`'native'`（默认，贡献可见能力 schema）、`'code'`（注册表仅贡献其保留的 `run_code` 传输通道加一份生成的 SDK `.d.ts` 到系统提示词中）或 `'both'`（原生 schema 加传输通道 + SDK）。注册表在源头塑造其权威贡献；协作式提示词组装的结果仍具权威性，记录在日志中的请求头精确反映该返回的呈现。 -2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包`@deepseek-ai/dsh-code-runtime`，拥有 `ctx.codeRuntime`（[能力 seam](../architecture/2026-06-13-capability-seams.md)；消费方 = `dsh-tools`，core 消费 seam 的先例见 `agent-loop` → `dsh-llm`）。运行时对工具一无所知：它接收一段程序和命名的异步绑定，执行程序，报告 `{ value, logs, error? }`。语言和基底是后端属性，因此未来的 Python 或容器后端只是另一个实现包，而非重新设计。 -3. **交付的实现是 `@deepseek-ai/dsh-code-runtime-worker`**：每次运行 spawn 一个全新的 Node worker 线程，对模型的 TypeScript 进行 type-strip 后执行，绑定通过消息端口桥接，环境为空，堆/输出/时间上限可配置，并支持硬终止。其信任姿态在设计上等同于 bash——无需 unsafe-acknowledgement flag——因为 harness 已经交付了 `dsh-bash-local`，后者以严格*更高*的环境权限执行模型编写的任意 shell 命令。 +2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包 `@deepseek-ai/dsh-code-runtime`，拥有 `ctx.codeRuntime`（[能力 seam](../architecture/2026-06-13-capability-seams.md)；消费方 = `dsh-tools`，core 消费 seam 的先例见 `agent-loop` → `dsh-llm`）。运行时对工具一无所知：它接收一段程序和命名的异步绑定，执行程序，报告 `{ value, logs, error? }`。语言和基底是后端属性，因此未来的 Python 或容器后端只是另一个实现包，而非重新设计。 +3. **两种实现保持同一份全新 worker 契约**：`@deepseek-ai/dsh-code-runtime-worker` 在 harness 进程内运行 worker，`@deepseek-ai/dsh-code-runtime-subprocess` 则通过 `ctx.fs` 物化 runner，再通过 `ctx.subprocess` 在另一执行环境中启动。二者都在具有空环境的全新 Node worker 内执行由宿主剥离类型的 TypeScript，并提供桥接绑定、可配置的堆／输出／时间上限和硬终止。其信任姿态在设计上等同于 bash；更强的隔离来自挂载的执行环境。 本说明负责定义 Code Mode 的呈现、组合、隔离与结算基础。后续的[类型化工具返回值 Agent Note](2026-07-20-code-mode-typed-tool-returns.md)负责定义生成的输出映射、规范绑定值、`ToolCallError` 和无损外层输出边界。 @@ -32,7 +32,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **与 `toolOrder` 的交互，预先说明：** 如果配置的 `systemPrompt.toolOrder` 引用了原生能力名称，在 `mode: 'code'` 下会拒绝所有组装，因为那些名称不在该模式的协议校验范围内。这是正确行为而非 bug：使用 Code Mode 的部署需要更新其 order 配置或移除它。 -**SDK 提示词段。** 在 `'code'` 和 `'both'` 下，tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染所加载运行时语言的声明加固定的使用说明（默认 TypeScript；[语言分发 note](2026-07-31-code-mode-language-dispatch.md) 加入了 Python 与按 `ctx.codeRuntime.language` 选择的渲染器表）。它共享查找和执行可见性，排除 `run_code`，并按字典序排列工具以获得字节稳定的输出。 +**SDK 提示词段。** 在 `'code'` 和 `'both'` 下，tool-guidance order band 中的惰性 `tools:sdk` 段为当前 scope 的可见能力渲染 TypeScript 声明加固定的使用说明。它共享查找和执行可见性，排除 `run_code`，并按字典序排列工具以获得字节稳定的输出。 **组装所有权。** `run_code` 和 `tools:sdk` 作为正常的组装输入进入受信任的 `system-prompt/assemble` waterfall。一个 scoped 的 `tools:sdk` 段可以在分发前遮蔽全局默认值，监听器也可以移除或替换任一贡献。waterfall 返回的组装结果是最终的，因此修改这些输入的人有责任在部署期望 Code Mode 可用时保持协议面的完整性；没有恢复 pass 会覆盖有意的组合。 @@ -40,7 +40,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 ### run_code 工具与分发桥 -在 `'code'` 和 `'both'` 下，注册表拥有 `run_code` 作为保留的呈现传输通道，带两个必需参数 `{ code: string; description: string }`（description 为 UI 标注该调用，沿用 bash 的先例）。它由一个正常的 `ToolDefinition` 表示以供分发，但位于可过滤的能力层之外，因此限制规则不会意外移除 Code Mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调性守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致；权限插件可以在程序运行前检查程序文本，最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`： +在 `'code'` 和 `'both'` 下，注册表拥有 `run_code` 作为保留的呈现传输通道，带两个必需参数 `{ code: string; description: string }`（description 为 UI 标注该调用，沿用 bash 的先例）。它由一个正常的 `ToolDefinition` 表示以供分发，但位于可过滤的能力层之外，因此限制规则不会意外移除 Code Mode 的唯一入口。调用遍历完整的工具流水线——`tools/pre-execute` → 单调守卫 → `tools/execute` 包裹分发 → `tools/post-execute` → 由定义拥有的可选 `finalizeContent` → 不可变的 `tools/result` 通知——与原生调用完全一致；权限插件可以在程序运行前检查程序文本，最终结果观察者看到的是规范化的外层结果。其 `execute(args, exec)`： 1. **构建绑定。** 一个 run 级别的 signal 跟随外层取消，并在 run 结算时被 abort。每个可见工具绑定都会对无损 JSON 参数创建快照，进入原生契约的分发池（调度设计由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责），以确定性的 call id 和外层 token 作为 `parent` 执行，通过外层 execution 延后返回的上下文，并记录 `tool/code-dispatch-start`/`tool/code-dispatch` 事件对，其中结算侧携带完整渲染后的结果内容。成功时返回工具最终的规范 JSON 值；失败则变为程序可见的 `ToolCallError`。每个子调用保留自己不可变的执行标识，并遍历完整的工具流水线。 2. **运行程序**：`ctx.codeRuntime.run({ program: args.code, bindings: [{ global: 'tools', functions }], signal: runController.signal })`。运行时接收的是 run 级别的 signal 而非仅调用方的外层 signal，因此外层 run 以任何方式结算都会同时 abort 运行时内部的工作。 @@ -48,7 +48,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 **子调用上下文通过父调用延后。** 在 `run_code` 内部注入会破坏父调用/结果的相邻性，因此 `ToolRunContext.deferContext()` 按分发顺序收集每个子结果的 `additionalContexts` 条目。即使程序后来抛出异常，注册表仍携带该数组；循环只在外层结果与步骤中所有兄弟结果之后追加每个条目。外层 post-execute 阻止会丢弃工具延后的条目，只暴露阻止 decision 显式附加的上下文。 -**并发是有界的，而非被序列化。** 每次 run 拥有一个分发队列，严格按提交顺序启动调用，并通过 `registry.executionMode` 对每个调用分类——与原生循环所用的 fail-closed `isConcurrencySafe` 契约相同。连续的 parallel 类调用最多重叠 `maxParallelSubCalls` 个（默认 10；设为 `1` 恢复串行分发）；exclusive 类调用会排空池并单独运行。结算时放弃尚未开始的排队调用。本 note 交付的是被序列化的占位实现；取代它的调度器由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 +**并发被序列化。** 每次 run 拥有一个分发队列，因此即使 `Promise.all` 也按提交顺序执行工具调用。结算时放弃尚未开始的排队调用。并行化需要每个工具的并发安全元数据。 **呈现。** `run_code` 的 render intent 按[呈现意图 Agent Note](../architecture/2026-07-02-tool-render-intent-union.md)在此决定：`presentCall` 创建一个 `generic` 卡片，`kind: 'execute'`，以程序文本作为标题，并将同一程序文本作为 `rawInput`；`run_code` 有意不声明 `presentResult`，因此 TUI 和宿主/客户端运行时（Web）会通过通用原始内容回退机制，使用最终持久化的 `tool/result.content` 补全该卡片，其中包括捕获的日志，以及返回值、失败信息或 post-policy 输出落盘预览。这不是 `terminal` 卡片：该卡片的语义是「工作目录中的 shell 命令」，程序不是。参见[结果卡片完整性说明](../../archived/bug-fix/2026-07-20-code-mode-result-card-completeness.md)。 @@ -61,39 +61,41 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一 `packages/code-runtime/code-runtime/`——`@deepseek-ai/dsh-code-runtime`，仅依赖 `cordis`。一个抽象的 `CodeRuntime extends Service`（`super(ctx, 'codeRuntime')`）加上词汇： - `CodeRunRequest = { program: string; bindings: CodeBindingNamespace[]; signal?: AbortSignal }` -- `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露；可选描述符要求运行时注入真正的、程序可见的 reject 类，而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型，因此绑定参数与返回值可以完整跨越实现的序列化边界。 +- `CodeBindingNamespace = { global: string; functions: Record Promise>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露；可选描述符要求运行时注入真正的、程序可见的 reject 类，而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型，因此绑定参数与解析值可以完整跨越实现的序列化边界。 - `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }`——程序执行失败时，执行 promise 仍会 fulfill，并通过 `error` 字段返回失败结果。只有调用方/seam 误用（例如重复的绑定命名空间）时，`run()` 才会 reject；消费方仍在自己的错误边界处理不合规后端的拒绝。 - `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }`——按[防御性模式](../../../../docs/defensive-patterns.md)独立报告的正交结果；超时的 run 不是异常，abort 不是超时，有损完成值不是溢出，基底退出也与上述情况相互独立。 -- 两个只读的后端描述符，仅供信息参考而非门禁判定：`language`（程序必须使用的语言——首个后端为 `'typescript'`；Python 后端声明 `'python'`，并在呈现侧配对自己的 SDK 生成器）和 `isolation`（交付的后端为 `'worker-thread'`；未来可为 `'process'`、`'container'` 等）。`dsh-tools` 接受任何注册了 SDK 渲染器与 `run_code` flavor 的 `language`（TypeScript 与 Python 已交付；见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)），否则组装会大声失败，与 `toolOrder` 违规时的配置错误惯用法相同（如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`）。 +- 两个只读的后端描述符，仅供信息参考而非门禁判定：`language`（程序必须使用的语言——两种已交付后端均为 `'typescript'`；Python 后端会配对自己的 SDK 生成器）和 `isolation`（两种已交付后端均为 `'worker-thread'`；子进程提供方可以增加容器边界）。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败，与 `toolOrder` 违规时的配置错误惯用法相同（如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`）。 请求包含所有运行时输入；实现方拥有经校验的超时和上限默认值。注册表仅在组装 Code Mode 时查找可选的运行时，因此 native 模式不依赖它。缺失或语言不兼容的运行时会大声失败。替代基底或语言可以在同一 seam 背后替换实现，配对相应的 SDK 生成器。 ### worker-thread 运行时 -`@deepseek-ai/dsh-code-runtime-worker`，`packages/code-runtime/` 组的第二个包。每次 `run()`： +`@deepseek-ai/dsh-code-runtime-worker`，`packages/code-runtime/` 组的第二个包（package）。每次 `run()`： -1. **宿主侧 type-strip**，使用 Node 内置的 `stripTypeScriptTypes`（`node:module`；在本仓库的整个引擎范围 `^22.19.0 || >=24.0.0` 内可用，且会保留源码位置，因此运行时错误行号与模型源码一致）。仅剥离模式拒绝不可擦除的语法（`enum`、namespaces）——该拒绝以 `error.kind: 'exception'` 加 Node 的消息返回，SDK 说明写明「仅限可擦除 TypeScript」，模型像处理其他程序错误一样自我修正。语法级失败不会 spawn worker。 +1. **宿主侧 type-strip**，使用 Node 内置的 `stripTypeScriptTypes`（`node:module`；在本仓库的整个引擎范围 `^22.19.0 || >=24.0.0` 内可用，且保持位置不变，因此运行时错误行号与模型源码一致）。仅剥离模式拒绝不可擦除的语法（`enum`、namespaces）——该拒绝以 `error.kind: 'exception'` 加 Node 的消息返回，SDK 说明写明「仅限可擦除 TypeScript」，模型像处理其他程序错误一样自我修正。语法级失败不会 spawn worker。 2. **每次 run spawn 一个全新 `Worker`**，来自包自身的 bootstrap 模块：`env: {}`（真正为空——比 spawn 命令的 scrubbed-env 规则更严格），`resourceLimits` 来自配置，`stdout`/`stderr` 捕获到 `logs` 而非继承。不做池化，不跨 run 保留状态：程序的世界随 worker 消亡，这使得 run 仅从日志即可重建，状态泄漏不可表达。 3. **在 bootstrap 中执行**：剥离后的程序成为一个 `AsyncFunction` 的函数体，其参数是绑定全局变量、消费方声明的 reject 类和一个捕获式 `console` shim，因此顶层 `await` 和 `return` 可用。Code Mode 声明 `ToolCallError`，成员属性为 `toolName`；运行时无需硬编码工具即可实体化真正的构造函数。无损 JSON 完成值会精确跨越边界；`undefined` 仍表示缺席，有损值产生 `invalid-output`，过大的外层结果产生 `output-limit`，而不会退化为检查格式化后的字符串替代品。 4. **通过消息端口桥接绑定**：worker 中的每个绑定函数发送 `{ id, global, name, args }` 并等待回复；宿主根据请求的绑定校验名称、调用、并回复 `{ id, ok, value }` 或 `{ id, ok: false, message }`（宿主侧绑定拒绝变为程序侧 rejection）。worker 侧的命名空间对象通过 `defineProperty` 构建为 null-prototype，因此名为 `__proto__`、`constructor` 或 `toString` 的绑定是普通自有属性，而非原型链碰撞。未知名称、重复 id 和结算后消息被拒绝或忽略——端口协议假设对端是恶意的，因为对端运行的是模型代码。 5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间，允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间，包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合；中间绑定值没有字节数上限。到期、取消和完成都终止 worker，堆退出或外层溢出会作为显式失败报告。 -6. **dispose（资源释放）至完全停稳**：服务自身的 dispose 终止进行中的 worker 并*等待*其退出后再 resolve，遵循[防御性模式](../../../../docs/defensive-patterns.md)。 +6. **dispose 至完全停稳**：服务自身的 dispose（资源释放）终止进行中的 worker 并*等待*其退出后再 resolve，遵循[防御性模式](../../../../docs/defensive-patterns.md)。 + +`@deepseek-ai/dsh-code-runtime-subprocess` 在文件系统／子进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它在 `ctx.subprocess.runtimeRoot` 下写入一个无依赖 runner，通过提供方解析 Node，并在原始管道上使用有界 base64 JSON 帧承载绑定通信。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包；`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。 ### 信任姿态 -worker 运行时只能约束程序的运行，而不构成安全边界：模型代码可以访问 Node API，权限与 bash 工具相当。`worker.terminate()` 停止线程但不停止它 spawn 的 OS 进程。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁，并额外提供空环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署需要为代码和 bash 都使用容器级后端；运行时的 isolation 描述符让它们能区分该后端。 +这些运行时提供的是隔离，而非独立安全边界：模型代码可以访问 Node API，并拥有挂载的执行环境所授予的权限。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁，并额外提供空 worker 环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署应当为代码和 bash 都挂载容器级文件系统／子进程提供方。 ### 模型看到的内容 -SDK 指示模型编写一个所加载运行时语言的异步函数体（默认可擦除 TypeScript；Python 运行时下为 Python `async` 函数体——见[语言分发 note](2026-07-31-code-mode-language-dispatch.md)），通过 `await tools.name(args)` 调用工具，在需要时捕获被拒绝的工具调用，并仅 return 或 log 应重新进入上下文的输出。两种 flavor 用各自的原语陈述同一契约：相互独立的只读调用可以在 `Promise.all`（TypeScript）或 `asyncio.gather`（Python）下重叠，有副作用的调用按提交顺序单独运行，有依赖的工作用 `await` 排序。声明前缀可能与原生 schema 一样大，尤其在 `'both'` 下，但对提供方缓存保持稳定。 +SDK 指示模型编写一个异步的可擦除 TypeScript 函数体，通过 `await tools.name(args)` 调用工具，在需要时捕获被拒绝的工具调用，并仅 return 或 log 应重新进入上下文的输出。即使在 `Promise.all` 下调用仍保持顺序。声明前缀可能与原生 schema 一样大，尤其在 `'both'` 下，但对提供方缓存保持稳定。 ## 后果 -切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发在有界的重叠池下按提交顺序启动，而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 +切换到 `'code'` 的部署必须更新任何仅限 native 的 `toolOrder`。组装监听器有责任维护任何被重写的协议面的完整性。子分发保持序列化，而每次调用的上下文会通过外层结果保留其 source、信封与元数据。 ## 测试 -- **Worker 运行时：** 真实 worker 测试覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。一个构建后包测试在纯 Node 下运行 worker 入口。 +- **运行时实现：** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与文件系统／子进程组合；后者另有一个由 Loader 驱动的 `cordis.yml` 测试。 - **注册表集成：** 测试覆盖代码生成、所有呈现模式、保留名称和限制规则、scoped 可见性、权威组装重写、`toolOrder`、运行时兼容性失败、完整流水线子分发、parent-token 关联、序列化、取消和队列排空、JSON 规范化、错误传播、日志事件、成功与失败程序中的有序上下文延后、外层阻止抑制以及 HMR（热模块替换）清理。 - **带密钥 e2e：** 真实模型在一个程序中组合两次 bash 调用；另一个模型通过 Code Mode fs 分发发现嵌套的工作区指令。测试验证折叠的请求头、关联的分发事件、结果文件、延后上下文和模型行为。 - **快照：** `code-mode-turn`、`both-mode-turn` 和 `code-mode-workspace-context` fixture（测试前置数据）固定 SDK 文本、请求头工具列表、分发事件、延后上下文和结果卡片。 @@ -104,9 +106,9 @@ SDK 指示模型编写一个所加载运行时语言的异步函数体（默认 **`node:vm` 作为参考运行时，加固推迟。** 否决：`node:vm` 不是隔离（原型链逃逸可达宿主 realm）且无法中断热循环。worker 线程提供独立 isolate、空环境、`resourceLimits` 和可靠的 `terminate()`，信任等级等同于 bash，因此参考实现和生产实现是同一个包，无需 unsafe-acknowledgement 仪式。 -**在原生工具调用上做结果省略/摘要。** 仅解决问题中上下文膨胀这一半：裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低，但仍需每次调用一次模型往返，且无法表达循环、分支或汇合。互补而非竞争；它可以在 Code Mode 下为残余的原生调用分层。 +**在原生工具调用上做结果省略/摘要。** 仅解决问题的上下文膨胀一半：裁剪旧 `tool-result` 作为可重建请求下的日志化表面替换成本低，但仍需每次调用一次模型往返，且无法表达循环、分支或汇合。互补而非竞争；它可以在 Code Mode 下为残余的原生调用分层。 -**循环中的并行原生分发。** 决策当时对往返成本的另一个答案；它被并发安全元数据阻塞，且无论如何都不提供组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的队列决策保持了两者兼容，后续也正是这样落地的：元数据以 `isConcurrencySafe` 的形式就绪（见[并行工具调用 note](2026-07-10-parallel-tool-call-execution.md)），原生 rolling-pool 分发与每工具绑定并行化基于同一个分类器一起解锁。 +**循环中的并行原生分发。** 往返成本的另一个答案；仍是有效的未来工作（open TODO），仍被并发安全元数据阻塞，且仍无组合能力——它并行化的是模型在一步中已经决定的调用。Code Mode 的序列化队列决策保持两者兼容：当元数据就绪时，原生并行分发和每工具绑定并行化一起解锁。 **始终排他（忠于 Cloudflare，无模式）。** 否决，因为本 SDK 的主要消费方是编码 agent：其日常的单次调用（`bash`、`read`、`edit`）作为原生调用已经是最优的，强制每次编辑都通过程序会给常见场景增加负担。mode 配置让忠实形式（`'code'`）只需一行配置即可启用，而不强加于人。 @@ -114,20 +116,20 @@ SDK 指示模型编写一个所加载运行时语言的异步函数体（默认 **SDK 中的清洁化标识符别名**（`my-tool` → `my_tool`，Cloudflare 的做法）。否决：`declare const` 上的带引号键使每个名称可达，零别名碰撞逻辑；模型能正常处理 `tools["my-tool"](…)`。 -**REPL 风格的持久内核**（状态跨 `run_code` 调用存活）。在 MVP 中否决：跨调用状态对会话日志不可见，破坏了「每个请求是日志的纯函数」这一可重建性保证；每次 run 均使用全新实例则维持了这一保证。内核风格的后端在未来仍可通过同一 seam 表达，配合自己的日志方案。 +**REPL 风格的持久内核**（状态跨 `run_code` 调用存活）。在 MVP 中否决：跨调用状态对会话日志不可见，破坏了「每个请求是日志的纯函数」这一可重建性保证；每次 run 全新保持了这一点。内核风格的后端在未来仍可通过同一 seam 表达，配合自己的日志方案。 ## 风险 -**Worker 不是硬安全边界。** 有意为之且已文档化（§信任姿态）：姿态等同于既有的 bash 工具，约束能力强于它，门禁使用相同的 seam。需要更强隔离的部署需要未来的 `isolation: 'container'` 后端——作为 seam 设计的扩展点跟踪，而非本设计的 TODO。 +**worker 不是硬安全边界。** 有意为之且已文档化（§信任姿态）：姿态等同于既有的 bash 工具，门禁使用相同的 seam。子进程实现可以在容器级执行环境中运行，但其 worker 描述符本身不声称具备该边界。 **`stripTypeScriptTypes` 标记为 experimental。** 它与 Node 自身原生 `.ts` 执行背后的引擎（amaro/swc）相同，在本仓库的整个引擎范围内作为 API 暴露。缓解措施：运行时的单元测试套件固定了所依赖的行为（位置保持、可擦除限制的拒绝消息形状宽松匹配），调用位于一个私有函数之后，且 `amaro`/`sucrase` 是 API 变化时的直接替代品。仅可擦除子集是面向模型的契约线，错误路径是一个可工作的反馈循环，而非死胡同。 **SDK 的提示词成本，尤其在 `'both'` 下。** `.d.ts` 可能与它补充的原生 schema 体量相当；`'both'` 携带两种表示。前缀稳定性 + 提供方缓存摊销了每会话成本；mode 是每部署的；本 Agent Note 不做无条件节省的声明。何时优先使用哪种模式的量化指导明确属于上线后学习。 -**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。包内的模块边界（`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列）和 seam 共同约束了这一增长：所有基底相关的内容都在 `ctx.codeRuntime` 之后。 +**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。通过包内的模块边界（`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列）和 seam 约束：所有基底相关的内容都在 `ctx.codeRuntime` 之后。 -**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照，并完整返回规范 JSON 返回值。运行时会校验 worker 端口两侧，但不对单次绑定设置字节数上限；结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。 +**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照，并完整返回规范 JSON 解析值。直接 worker 运行时不对单次绑定设置字节数上限；子进程运行时使用 `maxFrameBytes` 限制每个传输帧，但重复或并发调用仍可能消耗进程或 worker 内存。包含日志、完成值和失败诊断的组合外层输出账本另由 `maxOutputBytes` 限制。 -**子分发的重叠由工具自身的安全声明限定，而非由调用方决定。** 程序里的 `Promise.all` 或 `asyncio.gather` 只在工具自己分类为并发安全的调用之间换来挂钟并行性；一串 exclusive 调用仍要按顺序付出各自的往返开销，模型可能过度期望。两种 flavor 的 SDK 说明都陈述了真实契约。本 note 交付的是使该风险绝对化的序列化占位实现；调度器及其重叠上限由[实时并行 Agent Note](2026-07-26-code-mode-live-parallel-dispatch.md) 负责。 +**仅序列化的子分发。** `Promise.all` 尚未获得挂钟并行性，仅减少往返次数；模型可能过度期望。说明中已声明；解除此限制与原生并行分发 TODO 所需的并发安全元数据绑定。 -**预算计量读取事件循环，而非 flag。** 忙碌时间轮询（`eventLoopUtilization()`）比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明（「pending 的分发不能暂停它」）是抵御恶意程序的关键。两种情况均有单元测试（带 pending 诱饵分发的热循环会在耗尽 `computeMs` 预算时终止；等待慢速绑定的空闲程序则会持续运行至 `maxWallMs`），轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项，且会传入 `setTimeout`，后者会把超过 `MAX_TIMER_DELAY_MS`（2^31-1 ms）的延迟夹到 1 ms；因此仅有正数校验会放行一个 25 天的上限，它在第一个 tick 就到期，使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界，因为它对照的是实测占用率，而不是交给定时器。 +**预算计量读取事件循环，而非 flag。** 忙碌时间轮询（`eventLoopUtilization()`）比精确 CPU 计量更粗糙——预算到期最多延迟一个轮询间隔——且其正确性声明（「pending 的分发不能暂停它」）对恶意程序是承重的。两侧都有单元测试（带 pending 诱饵分发的热循环在 `computeMs` 处死亡；在慢绑定上空闲的程序存活到 `maxWallMs`），轮询间隔是内部常量而非配置——部署无法将其误调为绕过手段。`maxWallMs` 是配置项，且会传入 `setTimeout`，后者会把超过 `MAX_TIMER_DELAY_MS`（2^31-1 ms）的延迟夹到 1 ms；因此仅有正数校验会放行一个 25 天的上限，它在第一个 tick 就到期，使每次运行都超时。worker 运行时正因如此在加载时对该字段做范围校验。`computeMs` 不需要上界，因为它对照的是实测占用率，而不是交给定时器。 diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml index 451a4ea0d3..6a801df865 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md -2026-07-16-persistent-pty-sessions.md: d7d06dc8517780a37889e4f94dd0b99475dc5432 -2026-07-16-persistent-pty-sessions.zh.md: 84ef8988ba657a87137f904df38e45808d32b483 +2026-07-16-persistent-pty-sessions.md: 65d265b83ab5ad89c2b919364043eb28e75977c7 +2026-07-16-persistent-pty-sessions.zh.md: 6ea92da9a1b2870f3773d84e7f5629bec220dd1c diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md index d7d06dc851..65d265b83a 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md @@ -23,10 +23,10 @@ The implementation supports interactive shells and line-oriented REPLs on Linux | Package | Role | ctx key | |---|---|---| | `dsh-pty` | `PtyService`, branded `PtySessionId`, backend registry, owner-scoped session contract, and result types | `ctx.pty` | -| `dsh-pty-local` | [`node-pty`](https://github.com/microsoft/node-pty)-based local backend, platform process inspection, bounded terminal buffer, sandbox resolution, and process-tree supervision | registers a backend on `ctx.pty` | +| `dsh-pty-local` | Persistent-shell backend over `ctx.subprocess.spawnTerminal()`: readiness, bounded terminal buffers, sandbox resolution, and owner-aware session lifecycle | registers a backend on `ctx.pty` | | `dsh-tool-pty` | Six model-facing tools, task-runtime integration for background sends, guidance, and UI render intents | registers on `ctx.tools` | -Idle detection is backend behavior, not a second public seam. A remote or container backend may have authoritative readiness signals that do not resemble local `/proc` inspection; every `PtyBackend` therefore returns the common send result while owning its detection mechanism internally. +Readiness remains PTY-backend behavior, not a second public seam. The terminal-process provider supplies only substrate facts such as the foreground process group and whether it can prove that group is waiting on input; `dsh-pty-local` combines those facts with prompt and silence evidence into the common send result. ### Agent ownership and identity @@ -40,12 +40,12 @@ Agent-scope disposal closes registrations first, then awaits quiescent teardown A registered `shell` backend constrains how a terminal starts; it does not constrain commands typed after startup. `dsh-pty-local` therefore applies two protections before spawning: -- It builds a scrubbed child environment using the same credential-shaped-name policy as `bash-local`, removing ambient `*KEY*`, `*PASSWORD*`, `*SECRET*`, `*TOKEN*`, and harness-managed variables unless an explicit trusted mapping supplies them. +- It supplies only terminal-specific environment overrides; the mounted subprocess provider applies the shared credential-shaped-name scrub before merging them. - It requires `ctx.sandbox` and the shared `ctx.sandboxPolicy`. At spawn, the backend resolves the owner's effective session mode over the deployment default and wraps the shell argv once; that mode and workspace root remain the process boundary for the PTY lifetime. A write that would change the effective `sandbox/mode` is rejected before commit while the owner has any open PTY or unpublished spawn, with an instruction to wait for creation to settle and close those sessions first; same-effective-mode writes remain valid. The pending reservation spans backend setup through publication, so there is no race in which a wider terminal appears after a downgrade. `danger-full-access` is the existing explicit unconfined choice rather than a PTY-specific bypass. Sandboxing confines local process effects but does not make arbitrary shell input safe: network calls and other external side effects remain governed by deployment policy. Tool descriptions state that PTY sessions are less auditable than one-shot tools and should be used only when persistence or interactive stdin is necessary. -The implementation uses only public `node-pty` capabilities: child PID, `data` and `exit` notifications, `write`, `resize`, and `kill`. It does not assume access to the native master fd or call `waitpid` from TypeScript. Platform process inspectors derive foreground process groups and parent/child identity from `/proc` on Linux and `ps` on macOS. +The local subprocess terminal primitive uses only public `node-pty` capabilities: child PID, `data` and `exit` notifications, `write`, and `kill`. It does not assume access to the native master fd or call `waitpid` from TypeScript. Platform process inspectors below that primitive derive foreground process groups and parent/child identity from `/proc` on Linux and `ps` on macOS. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns this process/consumer split. ### Six model-facing tools @@ -92,9 +92,9 @@ Background sends use the existing task completion notice and `task_output` resul ### Process-tree teardown -The top-level `node-pty` child is the ownership anchor. On close, the backend stops callbacks, snapshots its transitive descendants by parent PID in children-first order, sends `SIGTERM`, waits, rescans for children forked during shutdown, sends `SIGKILL` to the remaining descendant tree, and verifies that every non-zombie descendant left the process table while the shell is still alive. A matching Linux zombie has no executable work and therefore counts as quiescent, allowing shell shutdown to reap or reparent it. Only then does the backend stop the shell with its own TERM/grace/KILL sequence. Every captured PID includes process-start identity so reuse cannot redirect escalation. +The subprocess terminal handle owns the top-level terminal process and its session. On close it snapshots transitive descendants by parent PID in children-first order, sends `SIGTERM`, waits, rescans for children forked during shutdown, sends `SIGKILL` to the union, and verifies every non-zombie descendant left the process table before stopping the top-level process. A matching Linux zombie has no executable work and therefore counts as quiescent. Every captured PID includes process-start identity so reuse cannot redirect escalation. -Teardown reports root exit and survivor cleanup independently. It does not claim success merely because the shell exited; disposal resolves only after no captured non-quiescent tree member remains or returns a cleanup failure naming the survivors. A failed close is not cached forever: the registry and local session each clear the fence only when it still names that failed attempt, so a later explicit or lifecycle close retries after the external survivor condition changes without disturbing a newer concurrent attempt. Service disposal still clears its backend, reservation, and owner-detacher registries when a close fails. It never broadens ownership to every member of the root PID's POSIX session. +Teardown reports top-level exit and survivor cleanup independently. The PTY session does not claim success merely because the shell exited: it calls `SubprocessTerminalHandle.terminate()` and awaits whole-session quiescence, propagating a cleanup failure that names survivors. A failed close is not cached forever: the registry and local session clear the fence only when it still names that failed attempt, so a later explicit or lifecycle close retries without disturbing a newer concurrent attempt. Service disposal still clears its backend, reservation, and owner-detacher registries when a close fails. ### Composition and rollout @@ -108,6 +108,7 @@ plugins: mode: workspace-write workspaceRoot: . '@deepseek-ai/dsh-pty': + '@deepseek-ai/dsh-subprocess-local': '@deepseek-ai/dsh-pty-local': config: scrollbackLines: 10000 @@ -141,11 +142,11 @@ The package ships concise tool guidance explaining persistent state, owner isola **Add persistent mode to `bash`.** Rejected. Returning on readiness rather than process exit, retaining a process tree across calls, and exposing interactive stdin create a different ownership and failure contract. -**Require native master-fd access from `node-pty`.** Rejected. Its public API exposes no master fd. The local backend instead derives foreground groups and descendants from supported OS process metadata and treats unreadable metadata as a detector miss. +**Require native master-fd access from `node-pty`.** Rejected. Its public API exposes no master fd. The local subprocess terminal adapter derives foreground groups and descendants from supported OS process metadata and treats unreadable metadata as a detector miss. **Signal every member of the root PID's POSIX session.** Rejected. `node-pty` may expose a helper PID whose session belongs to the launcher, so SID-wide teardown can signal unrelated harness or desktop processes. A PID-identity-fenced descendant tree is narrower and safe by construction. -**Publish `PtyIdleDetector` as a replaceable registry.** Rejected. Only the local backend needs these platform probes, while remote backends may receive readiness over their own protocol. Backend replacement already provides the necessary extension point. +**Publish `PtyIdleDetector` as a replaceable registry.** Rejected. Substrate-specific foreground facts come from the mounted terminal-process primitive, while prompt/silence readiness remains one private policy in `dsh-pty-local`. The filesystem/subprocess execution-world replacement is the necessary extension point. **Add a PTY-specific `sleep` tool.** Rejected. `ctx.tasks` already owns bounded waiting, cancellation, completion notices, and model-facing collection. A second general wake mechanism would cross the agent-loop boundary and duplicate that contract. @@ -156,8 +157,8 @@ The package ships concise tool guidance explaining persistent state, owner isola ## Verification - Per-file coverage pins owner fencing, concurrent reservations, unpublished-spawn cancellation and awaited teardown, sandbox-mode change rejection, retriable lifecycle cleanup, readiness tiers, rejection of pre-write stdin waits, the configured handoff grace holding the idle fallback past one poll and its rejection below `pollIntervalMs`, sanitizer carry state, complete UTF-8 bounds, task integration, schemas, and exact render intents. -- Linux process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. -- Real `node-pty` tests exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT` after deliberately delayed child readiness under scenario-owned timing bounds, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. +- Subprocess process fixtures cover non-leader and non-main-thread stdin waits, zombie quiescence, unreadable process state, supported syscall tables, unsupported architectures, and false-positive rejection; macOS inspector logic is injected into the same unit suite. +- Real `node-pty` and PTY-consumer tests jointly exercise shell state, shared sandbox policy, environment scrubbing, raw-mode foreground `SIGINT`, a TERM-ignoring descendant, and immediate post-disposal quiescence on supported hosts. - A Loader-driven `cordis.yml` test mounts the real three-package composition. ACP and headless snapshots pin the six schemas, bounded results, and errors through opt-in overlays; TUI snapshots pin terminal and generic card presentation. - Package contracts, the architecture map, core data structures, generated catalogs, and the website API describe the same shipped surface. - The repository CI-equivalent sequence owns type, lint, coverage, snapshot, documentation, build, hygiene, demo, and built-entry verification. @@ -172,10 +173,10 @@ The package ships concise tool guidance explaining persistent state, owner isola **Persistent state can drift from the model's belief.** The model may forget its cwd or active REPL. Session summaries and retained output help recovery, but no prompt can make state persistence deterministic. -**A daemonized descendant can leave the captured tree.** A process that reparents before teardown is no longer discoverable from the `node-pty` root. The implementation accepts that cleanup gap instead of risking SID-wide signals to unrelated processes. +**A daemonized descendant can leave the local provider's captured tree.** A process that reparents before teardown is no longer discoverable from the `node-pty` root. The local terminal primitive accepts that cleanup gap instead of risking SID-wide signals to unrelated processes. **A shell can cause external side effects.** Session sandboxing and environment scrubbing reduce local exposure but do not undo pushes, API calls, or messages. Deployments that cannot tolerate those effects must omit PTY or add network policy. **Process loss destroys terminal state.** In-process sessions do not survive a harness crash or restart, and raw scrollback is not durable. Important work must be committed to files or another durable system. -**`node-pty` is a native dependency.** Installation, supported Node versions, prebuild availability, and platform behavior require built-artifact smokes on every supported OS. +**`node-pty` is a native dependency of `dsh-subprocess-local`.** Installation, supported Node versions, prebuild availability, and platform behavior require built-artifact smokes on every supported OS. diff --git a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md index 84ef8988ba..6ea92da9a1 100644 --- a/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md +++ b/.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.zh.md @@ -14,7 +14,7 @@ harness 可以运行前台与后台命令、编辑文件和委派工作，但无 ## 决策 -可选的 `packages/pty/` 能力家族提供由 agent 拥有、持久化且面向行式交互的 PTY 会话。它遵循仓库的 [capability pattern](../../implemented/architecture/2026-06-13-capability-seams.md)，与现有命令和文件系统工具并存，并且不修改 `agent-loop`。 +可选的 `packages/pty/` 功能家族提供由 agent 拥有、持久化且面向行式交互的 PTY 会话。它遵循仓库的 [capability pattern](../../implemented/architecture/2026-06-13-capability-seams.md)，与现有命令和文件系统工具并存，并且不修改 `agent-loop`。 当前实现在 Linux 和 macOS 上支持交互式 shell 与行式 REPL。全屏终端应用、按键序列、BEL 触发的控制流、进程丢失后的会话恢复以及跨 agent 共享会话都明确推迟。 @@ -23,10 +23,10 @@ harness 可以运行前台与后台命令、编辑文件和委派工作，但无 | 包 | 角色 | ctx key | |---|---|---| | `dsh-pty` | `PtyService`、branded `PtySessionId`、后端注册表、按 owner 隔离的会话契约和结果类型 | `ctx.pty` | -| `dsh-pty-local` | 基于 [`node-pty`](https://github.com/microsoft/node-pty) 的本地后端、平台进程检查、有界终端缓冲、沙箱解析和进程树监管 | 在 `ctx.pty` 上注册后端 | +| `dsh-pty-local` | 基于 `ctx.subprocess.spawnTerminal()` 的持久 shell 后端：就绪状态、有界终端缓冲、沙箱解析和感知 owner 的会话生命周期 | 在 `ctx.pty` 上注册后端 | | `dsh-tool-pty` | 6 个面向模型的工具、后台发送的 task 运行时集成、使用指引和 UI 渲染意图 | 注册到 `ctx.tools` | -idle 检测属于后端行为，不是第二条公共 seam。远程或容器后端可能拥有完全不同于本地 `/proc` 检查的权威就绪信号；因此每个 `PtyBackend` 都返回统一的发送结果，同时在内部拥有自己的检测机制。 +就绪判定仍属于 PTY 后端行为，不是第二条公共 seam。终端进程提供方只提供基底事实，例如前台进程组，以及能否证明该组正在等待输入；`dsh-pty-local` 将这些事实与提示符和静默证据组合成统一的发送结果。 ### agent 所有权与身份 @@ -34,18 +34,18 @@ idle 检测属于后端行为，不是第二条公共 seam。远程或容器后 实现不提供插件加载期 auto-start 会话。`terminal_open` 只在 agent 工具调用期间创建会话，此时所有权和所属的事件溯源会话都已确定。未来的声明式启动功能必须通过尚未发布的 agent setup 组合，而不能创建全局共享终端。 -agent scope dispose（资源释放）时先关闭注册，再等待全部所属 PTY 完全停稳。未发布的后端 setup 同样是受追踪的生命周期操作：owner 或服务 dispose 会中止服务自有的 signal，等待后端结算与回滚完成后才返回。即使后端 reject，或返回的会话在回滚 close 时失败，调用方取消仍会原样保留其 `AbortSignal.reason`；该清理失败不会替换调用方原因，而会继续受追踪，留待后续 owner 或服务 dispose 处理。由 lifecycle dispose 触发的回滚 close 失败会使 spawn 与该 lifecycle dispose 都 reject，而 `PtyBackendCleanupError` 让后端在不替换调用方取消的前提下，为该 lifecycle dispose 保留自身的启动清理失败。若调用方取消已完成结算，而 dispose 尚未发生，该清理失败会继续作为受追踪的 owner activity 保留，直到后续 owner 或服务 dispose 消费并报告它，因此沙箱模式策略不会把清理失败误判为完全停稳。后端或工具插件 reload 不会遗留会话：所有权持续存放在 `PtyService` 中，直到 agent 结束，与 [`ctx.tasks`](../../../../packages/tasks/tasks/README.md) 的服务持有记录模式一致。服务会先同步把会话预留给一次活跃发送，再返回该操作；后台发送同样会在 task id 对外可见前完成预留。第二次发送会以 `SEND_ACTIVE` 失败，因此输出与取消无法跨越操作所有权。 +agent scope dispose 时先关闭注册，再等待全部所属 PTY 静默退出。未发布的后端 setup 同样是受追踪的生命周期操作：owner 或服务 dispose 会中止服务自有的 signal，等待后端结算与回滚完成后才返回。即使后端 reject，或返回的会话在回滚 close 时失败，调用方取消仍会原样保留其 `AbortSignal.reason`；该清理失败不会替换调用方原因，而会继续受追踪，留待后续 owner 或服务 dispose 处理。由 lifecycle dispose 触发的回滚 close 失败会使 spawn 与该 lifecycle dispose 都 reject，而 `PtyBackendCleanupError` 让后端在不替换调用方取消的前提下，为该 lifecycle dispose 保留自身的启动清理失败。若调用方取消先于 dispose 完成结算，该清理失败会继续作为受追踪的 owner activity 保留，直到后续 owner 或服务 dispose 消费并报告它，因此沙箱模式策略不会把清理失败误判为静默。后端或工具插件 reload 不会遗留会话：所有权持续存放在 `PtyService` 中，直到 agent 结束，与 [`ctx.tasks`](../../../../packages/tasks/tasks/README.md) 的服务持有记录模式一致。服务会先同步把会话预留给一次活跃发送，再返回该操作；后台发送同样会在 task id 对外可见前完成预留。第二次发送会以 `SEND_ACTIVE` 失败，因此输出与取消无法跨越操作所有权。 ### 安全与进程边界 注册的 `shell` 后端只约束终端如何启动，不约束启动后输入的命令。因此 `dsh-pty-local` 在 spawn 前应用两层保护： -- 它使用与 `bash-local` 相同的凭证形态名称策略构建清洗后的子进程环境，移除环境中的 `*KEY*`、`*PASSWORD*`、`*SECRET*`、`*TOKEN*` 和 harness 管理的变量，除非显式的可信映射提供这些值。 +- 它只提供终端专用的环境覆盖；挂载的子进程提供方先清除名称形似凭据的环境变量，再合并这些覆盖。 - 它要求 `ctx.sandbox` 和共享的 `ctx.sandboxPolicy`。后端在 spawn 时，以部署默认值为底折叠 owner 的有效 session mode，并只包装一次 shell argv；该 mode 与 workspace root 在 PTY 的整个生命周期中充当进程边界。只要 owner 有任何已打开的 PTY 或尚未发布的 spawn，任何会改变生效 `sandbox/mode` 的写入都会在提交前被拒绝，并提示先等待创建操作结算，再关闭这些会话；不会改变生效模式的写入仍然有效。这项进行中的预留从后端 setup 持续到发布完成，因此不存在降级后又出现权限更宽的终端这一竞态。`danger-full-access` 是现有的显式无约束选择，不另设 PTY 私有 bypass。 沙箱限制本地进程副作用，但不会让任意 shell 输入自动安全：网络调用和其他外部副作用仍由部署策略治理。工具描述会说明 PTY 会话比一次性工具更难审计，只应在确实需要持久状态或交互式 stdin 时使用。 -实现只使用 `node-pty` 的公共功能：子进程 PID、`data` 与 `exit` 通知、`write`、`resize` 和 `kill`。它不假设能访问原生 master fd，也不从 TypeScript 调用 `waitpid`。平台进程检查器在 Linux 上通过 `/proc`、在 macOS 上通过 `ps` 推导前台进程组和父子进程身份。 +本地子进程终端原语只使用 `node-pty` 的公共功能：子进程 PID、`data` 与 `exit` 通知、`write` 和 `kill`。它不假设能访问原生 master fd，也不从 TypeScript 调用 `waitpid`。该原语下的平台进程检查器在 Linux 上通过 `/proc`、在 macOS 上通过 `ps` 推导前台进程组和父子进程身份。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义这种进程／消费方拆分。 ### 6 个面向模型的工具 @@ -55,7 +55,7 @@ agent scope dispose（资源释放）时先关闭注册，再等待全部所属 | `terminal_send` | 发送文本、可选提交 Enter，并等待就绪或注册一个后台任务 | 有界 viewport、等待状态和会话状态；后台模式还返回 `taskId` | | `terminal_read` | 从保留的 scrollback 读取一个有界页 | `{ text, totalLines, lineBegin, lineEnd, truncated }` | | `terminal_signal` | 向当前前台进程组发送一种允许的信号 | `{ delivered, targetPgid }` | -| `terminal_close` | 关闭一个会话并等待进程树完全停稳 | `{ killed }` | +| `terminal_close` | 关闭一个会话并等待进程树静默退出 | `{ killed }` | | `terminal_list` | 列出调用方的活会话 | 按 owner 隔离的会话摘要 | UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发送使用 terminal 调用卡片和结果卡片；后台形式使用通用 `execute` 卡片。`terminal_open`、`terminal_read`、`terminal_signal`、`terminal_close` 和 `terminal_list` 分别使用通用 `execute`、`read`、`execute`、`delete` 和 `read` 卡片。所有 PTY 工具都不发出 `locations`。 @@ -66,7 +66,7 @@ UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发 当 `run_in_background: true` 时，`dsh-tool-pty` 在 `ctx.tasks` 上注册进行中的发送，并立即返回 `taskId`。生产方把 `maxResultBytes` 写入 task 快照，使 `task_output`、kill 返回的终态状态和完成通知在加上通用元数据后，仍对完整结果执行同一上限。`task_output(wait: true)` 负责等待、读取增量输出并记录最终结果；`task_kill` 会解析当前前台 PGID 并发送真正的 `SIGINT`，即使应用已禁用终端 `ISIG` 也同样如此，且后续升级仍只通过 PTY 后端拥有的 teardown 路径进行。若 task 对外接口不存在，后台模式必须在写入输入前失败。设计不新增 PTY 专用的 `sleep` 工具或通用唤醒 seam。 -`terminal_read` 从最新保留行起，朝更早的行分页。后端同时对保留的 scrollback 和返回页载荷执行行数与 UTF-8 字节上限，因此单个超长行无法绕过后端上限；工具随后再限制包含分页与截断元数据的完整渲染页。`truncated` 用于区分保留数据丢失与普通 viewport 增量。 +`terminal_read` 从最新保留行向后分页。后端同时对保留的 scrollback 和返回页载荷执行行数与 UTF-8 字节上限，因此单个超长行无法绕过后端上限；工具随后再限制包含分页与截断元数据的完整渲染页。`truncated` 用于区分保留数据丢失与普通 viewport 增量。 `terminal_signal` 接受闭合集 `SIGINT | SIGTERM | SIGKILL | SIGTSTP | SIGHUP`。后端在执行时解析终端前台进程组。当目标组是顶层 shell 时拒绝 `SIGKILL`，并指引调用方使用 `terminal_close`；进程组解析失败时操作直接失败，而不是向猜测的 PID 发送信号。 @@ -76,7 +76,7 @@ UI 渲染契约精确且不携带位置信息。`terminal_send` 只为前台发 在 Linux 上，检查器从 `/proc//stat` 读取 shell 的终端前台 PGID，枚举该进程组中的每个进程与线程，并检查它们当前的 syscall。Tier 1 只有观察到 stdin 等待才返回正结果：直接 `read(0)`、获准读取且含 fd 0 的 `select`/`pselect6` 或 `poll`/`ppoll` 参数，或者含 fd 0 的 epoll interest list。终端输入前就已存在的等待并不代表写入后就绪：必须先观察到同一 PGID 脱离该等待，之后再次进入等待才能使该次 send 完成；前台 PGID 发生变化则构成新的证据。无法读取的进程内存和未识别的 syscall 都是 miss，绝不作为正向猜测。架构表只包含对应 Linux UAPI 定义的 syscall number；不支持的架构跳过 Tier 1。 -macOS 没有精确 syscall 层。任何前台进程组输出静默都会返回 `inferred_idle`，包括 Python 和 `gdb`；从 `ps` 推导的终端 PGID 只用于发送信号，不作为「只有 shell 才能 idle」的证明。纯进程检查逻辑可注入，并在 Linux 上进行单元测试，同时由 macOS CI job 驱动真实 PTY 和进程表路径。 +macOS 没有精确 syscall 层。任何前台进程组输出静默都会返回 `inferred_idle`，包括 Python 和 `gdb`；从 `ps` 推导的终端 PGID 只用于发送信号，不作为「只有 shell 才能 idle」的证明。纯进程检查逻辑可注入并在 Linux 上完成 unit 覆盖率，同时由 macOS CI job 驱动真实 PTY 和进程表路径。 Tier 2 在持续 `idleSilenceMs` 没有输出后返回 `inferred_idle`，因此 sleep 或网络阻塞的命令可能看似 ready。如果此前已经见过 prompt marker，Tier 2 会再等待 `handoffGraceMs`，使恰好落在静默边界上的 bash 前台交接仍然以精确的 `stdin_read` 归因结束，而不是退到较弱的推断；该宽限是由部署方拥有的配置字段，并被校验为至少覆盖一个 `pollIntervalMs`——短于轮询周期的宽限装不下一次就绪轮询，因此不可能改变任何结果。它只约束见过 marker 的 send，代价是这一种情况的交互返回延迟，而不是每一次 send。Tier 3 在 `timeoutMs` 后返回 `timeout`，避免前台工具调用无限占住 agent。结果保留这些区别；调用方可以通过 `ctx.tasks` 等待、向前台组发信号，或从另一个会话排查。 @@ -88,13 +88,13 @@ Tier 2 在持续 `idleSilenceMs` 没有输出后返回 `inferred_idle`，因此 现有持久化 `tool/call` 与 `tool/result` 事件是模型发送文本和返回给模型的渲染输出的真源。`terminal_open` 通过已记录的工具结果返回 MOTD；前台 `send`/`read`/`list`/`signal`/`close` 结果走同一路径记录。PTY 包不会把原始字节流重复写入自定义会话事件。 -后台发送复用现有后台任务完成通知和 `task_output` 结果路径，因此进入后续模型请求的任何输出同样持久化。原始终端字节只作为有界的进程内状态存在，既不持久化也不可恢复。未来的 opt-in transcript（文本记录）sink 必须拥有独立的保留、凭证和隐私契约。 +后台发送复用现有后台任务完成通知和 `task_output` 结果路径，因此进入后续模型请求的任何输出同样持久化。原始终端字节只作为有界的进程内状态存在，既不持久化也不可恢复。未来的 opt-in transcript sink 必须拥有独立的保留、凭证和隐私契约。 ### 进程树 teardown -顶层 `node-pty` 子进程是所有权锚点。关闭时，后端先停止 callback，再按父 PID 以子进程优先顺序捕获其传递子进程、发送 `SIGTERM` 并等待，然后重新扫描关停期间 fork 出的子进程，向剩余子孙进程树发送 `SIGKILL`，并在 shell 仍存活时验证每个非僵尸子孙进程都已离开进程表。身份匹配的 Linux 僵尸进程已无可执行工作，因此视为完全停稳；shell 关闭时会回收它或将其重新挂接给负责回收的父进程。完成这些步骤后，后端才用 shell 自身的 TERM、宽限等待、KILL 序列停止 shell。每个捕获的 PID 都包含进程启动身份，避免 PID 复用把升级信号发给无关进程。 +子进程终端句柄拥有顶层终端进程及其会话。关闭时，它按父 PID 以子进程优先顺序捕获传递后代、发送 `SIGTERM` 并等待，然后重新扫描关停期间 fork 出的子进程，向二者并集发送 `SIGKILL`，并在停止顶层进程前验证每个非僵尸后代都已离开进程表。身份匹配的 Linux 僵尸进程已无可执行工作，因此视为完全停稳。每个捕获的 PID 都包含进程启动身份，避免 PID 复用把升级信号发给无关进程。 -teardown 独立报告根进程退出与存活进程清理。它不会只因 shell 退出就声称成功；dispose 只有在已捕获的进程树中不再存在尚未完全停稳的成员后才完成，否则返回清理失败并列出存活者。失败的 close 不会永久缓存：注册表与本地会话各自仅在关闭围栏仍指向该次失败尝试时才将其清除，因此外部存活进程状态改变后，后续的显式 close 或生命周期 close 会重试，且不会干扰较新的并发尝试。即使某个 close 失败，服务 dispose 仍会清空其后端、预留与 owner detacher 注册表。所有权绝不会扩大到根 PID 所属 POSIX 会话的全部成员。 +teardown 独立报告顶层进程退出与存活进程清理。PTY 会话不会只因 shell 退出就声称成功：它会调用 `SubprocessTerminalHandle.terminate()` 并等待整个会话完全停稳，若清理失败则向外传播并列出存活者。失败的 close 不会永久缓存：注册表与本地会话各自仅在关闭围栏仍指向该次失败尝试时才将其清除，因此后续的显式 close 或生命周期 close 会重试，且不会干扰较新的并发尝试。即使某个 close 失败，服务 dispose 仍会清空其后端、预留与 owner detacher 注册表。 ### 组合与推行 @@ -108,6 +108,7 @@ plugins: mode: workspace-write workspaceRoot: . '@deepseek-ai/dsh-pty': + '@deepseek-ai/dsh-subprocess-local': '@deepseek-ai/dsh-pty-local': config: scrollbackLines: 10000 @@ -141,11 +142,11 @@ plugins: **给 `bash` 增加持久模式。**拒绝。按就绪而不是进程退出返回、跨调用保留进程树、暴露交互式 stdin 会形成不同的所有权和失败契约。 -**要求从 `node-pty` 获取原生 master fd。**拒绝。它的公共 API 不暴露 master fd。本地后端改为从受支持的 OS 进程元数据推导前台组与子孙进程，并把不可读元数据视为 detector miss。 +**要求从 `node-pty` 获取原生 master fd。**拒绝。它的公共 API 不暴露 master fd。本地子进程终端适配器改为从受支持的 OS 进程元数据推导前台组与子孙进程，并把不可读元数据视为 detector miss。 **向根 PID 所属 POSIX 会话的全部成员发送信号。**拒绝。`node-pty` 可能暴露属于启动器会话的 helper PID，因此按 SID 清理可能向无关的 harness 或桌面进程发送信号。带 PID 启动身份校验的子孙进程树范围更窄，其安全边界由结构保证。 -**发布可替换注册表 `PtyIdleDetector`。**拒绝。只有本地后端需要这些平台 probe，远程后端可能通过自己的协议接收就绪状态。替换后端已经提供所需扩展点。 +**发布可替换注册表 `PtyIdleDetector`。**拒绝。基底专用的前台事实来自挂载的终端进程原语，提示符／静默就绪判定则仍是 `dsh-pty-local` 内部的一项私有策略。替换文件系统／子进程执行环境就是所需扩展点。 **新增 PTY 专用 `sleep` 工具。**拒绝。`ctx.tasks` 已经拥有有界等待、取消、完成通知和面向模型的收集。第二套通用唤醒机制会跨越 agent loop（智能体循环）边界并重复该契约。 @@ -155,9 +156,9 @@ plugins: ## 验证 -- 逐文件测试覆盖并固定 owner 隔离、并发预留、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 -- Linux 进程 fixture（测试前置数据）覆盖非 leader 与非主线程的 stdin 等待、僵尸进程的完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝；同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 -- 真实 `node-pty` 测试在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、在由场景掌控的时间界限内先有意延迟子进程就绪，再对 raw mode 前台进程发送 `SIGINT`、忽略 `SIGTERM` 的子进程，以及 dispose 返回后立即完全停稳。 +- 每文件覆盖率固定 owner 隔离、并发预留、未发布 spawn 的取消与等待式 teardown、沙箱模式变更拒绝、可重试的生命周期清理、就绪层级、对写入前 stdin 等待的拒绝、配置化交接宽限把 idle fallback 顶过一次轮询以及低于 `pollIntervalMs` 时的拒绝、sanitizer carry state、完整 UTF-8 结果上限、task 集成、schema 和精确 render intent。 +- 子进程 fixture 覆盖非 leader 与非主线程的 stdin 等待、僵尸进程完全停稳、不可读进程状态、受支持的 syscall 表、不支持的架构和误报拒绝；同一单元测试套件通过注入覆盖 macOS 检查器逻辑。 +- 真实 `node-pty` 与 PTY 消费方测试共同在受支持宿主上覆盖 shell 状态、共享沙箱策略、环境清洗、raw mode 前台 `SIGINT`、忽略 `SIGTERM` 的后代进程，以及 dispose 返回后立即完全停稳。 - Loader 驱动的 `cordis.yml` 测试挂载真实三包组合。ACP 与 headless 快照通过 opt-in overlay 固定 6 个 schema、有界结果和错误；TUI 快照固定 terminal 与 generic 卡片展示。 - 包契约、架构图、核心数据结构、生成目录和 website API 描述同一个已发布接口。 - 仓库 CI 等价序列负责类型、lint、覆盖率、快照、文档、构建、hygiene、demo 和 built-entry 验证。 @@ -172,10 +173,10 @@ plugins: **持久状态可能偏离模型认知。**模型可能忘记 cwd 或活跃 REPL。会话摘要和保留输出有助恢复，但任何 prompt 都无法让状态持久化变成确定行为。 -**daemonized 子进程可能离开捕获树。**在 teardown 前 reparent 的进程无法再从 `node-pty` 根进程发现。实现接受这个清理缺口，不冒险按 SID 向无关进程发送信号。 +**daemonized 后代进程可能离开本地提供方捕获的进程树。**在 teardown 前 reparent 的进程无法再从 `node-pty` 根进程发现。本地终端原语接受这个清理缺口，不冒险按 SID 向无关进程发送信号。 **Shell 可以造成外部副作用。**会话沙箱和环境清洗降低本地暴露，但无法撤销 push、API 调用或消息发送。无法容忍这些副作用的部署必须省略 PTY 或增加网络策略。 **进程丢失会销毁终端状态。**进程内会话无法跨 harness crash 或 restart 存活，原始 scrollback 也不持久化。重要工作必须提交到文件或其他持久系统。 -**`node-pty` 是原生依赖。**安装、支持的 Node 版本、prebuild 可用性和平台行为都需要在每个支持 OS 上运行构建产物冒烟测试。 +**`node-pty` 是 `dsh-subprocess-local` 的原生依赖。**安装、支持的 Node 版本、prebuild 可用性和平台行为都需要在每个支持 OS 上运行 built-artifact smoke。 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 1f81974b21..c07e81089c 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -284,6 +284,30 @@ export interface Config { Source: [`packages/client/hmr/src/index.ts:29`](../packages/client/hmr/src/index.ts) +## `@deepseek-ai/dsh-code-runtime-subprocess` + +Requires: `fs` · `subprocess` + +```ts config-catalog +/** Runtime configuration; every execution and bridge bound is deployment-tunable. */ +export interface Config { + /** Worker measured event-loop busy-time budget. */ + computeMs?: number + /** Host-observed wall-clock ceiling. */ + maxWallMs?: number + /** Combined serialized outer logs/value/diagnostic cap. */ + maxOutputBytes?: number + /** Worker old-generation heap cap in MiB. */ + maxOldGenerationSizeMb?: number + /** Largest decoded bridge frame, including binding traffic. */ + maxFrameBytes?: number + /** Process-tree TERM-to-KILL grace. */ + killGraceMs?: number +} +``` + +Source: [`packages/code-runtime/code-runtime-subprocess/src/index.ts:28`](../packages/code-runtime/code-runtime-subprocess/src/index.ts) + ## `@deepseek-ai/dsh-code-runtime-worker` ```ts config-catalog @@ -317,7 +341,7 @@ export interface Config { } ``` -Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:25`](../packages/code-runtime/code-runtime-worker/src/index.ts) +Source: [`packages/code-runtime/code-runtime-worker/src/index.ts:30`](../packages/code-runtime/code-runtime-worker/src/index.ts) ## `@deepseek-ai/dsh-compact-basic` @@ -423,7 +447,7 @@ export interface Config { } ``` -Source: [`packages/fs/fs-local/src/index.ts:38`](../packages/fs/fs-local/src/index.ts) +Source: [`packages/fs/fs-local/src/index.ts:40`](../packages/fs/fs-local/src/index.ts) ## `@deepseek-ai/dsh-fs-sandbox` @@ -906,7 +930,7 @@ Source: [`packages/llm/llm-retry/src/index.ts:46`](../packages/llm/llm-retry/src ## `@deepseek-ai/dsh-lsp-local` -Requires: `lsp` · `subprocess` +Requires: `fs` · `lsp` · `subprocess` ```ts config-catalog /** Plugin configuration: provider id → local language-server configuration. */ @@ -942,7 +966,7 @@ export interface LspLocalServerConfig { } ``` -Source: [`packages/lsp/lsp-local/src/index.ts:87`](../packages/lsp/lsp-local/src/index.ts) +Source: [`packages/lsp/lsp-local/src/index.ts:82`](../packages/lsp/lsp-local/src/index.ts) ## `@deepseek-ai/dsh-mcp-client` @@ -1048,7 +1072,7 @@ Source: [`packages/plan/plan-mode/src/index.ts:69`](../packages/plan/plan-mode/s ## `@deepseek-ai/dsh-pty-local` -Requires: `pty` · `sandbox` · `sandboxPolicy` +Requires: `pty` · `sandbox` · `sandboxPolicy` · `subprocess` ```ts config-catalog /** Public plugin configuration. */ diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index edfd046430..7015d39bc8 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -410,7 +410,7 @@ Single-slot decision for the next FileSystem.editText. Calling `next()` yields a Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:62`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:64`](../../packages/fs/fs/src/index.ts) ### `fs/observed` — emit @@ -430,7 +430,7 @@ Record a successful observation. Listeners must be synchronous recorders: throws Types: [FsTarget](../core-data-structures/filesystem.md) · [FsVersion](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:71`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:73`](../../packages/fs/fs/src/index.ts) ### `fs/write-intent` — waterfall @@ -450,7 +450,7 @@ Single-slot decision for the next FileSystem.writeText. Calling `next()` yields Types: [FsTarget](../core-data-structures/filesystem.md) · [FsWriteIntent](../core-data-structures/filesystem.md) -Source: [`packages/fs/fs/src/index.ts:54`](../../packages/fs/fs/src/index.ts) +Source: [`packages/fs/fs/src/index.ts:56`](../../packages/fs/fs/src/index.ts) ## `goal/*` diff --git a/docs/core-data-structures/filesystem.i18n.yaml b/docs/core-data-structures/filesystem.i18n.yaml index 91ac0ed81d..129c5569b9 100644 --- a/docs/core-data-structures/filesystem.i18n.yaml +++ b/docs/core-data-structures/filesystem.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/filesystem.md -filesystem.md: 110c1fd428b15c5094f9dcc94050cad61c324373 -filesystem.zh.md: 010ec22a5d4a29555425c3deede08b317cba7004 +filesystem.md: 50b936bdef7535769f19d6cc0a75eb11eea9a869 +filesystem.zh.md: 61a885203b0718b34b0fd936c318b03c65ce4770 diff --git a/docs/core-data-structures/filesystem.md b/docs/core-data-structures/filesystem.md index 110c1fd428..50b936bdef 100644 --- a/docs/core-data-structures/filesystem.md +++ b/docs/core-data-structures/filesystem.md @@ -12,6 +12,8 @@ Provider source: [`packages/fs/fs/src/types.ts`](../../packages/fs/fs/src/types. Every operation resolves a user-supplied path to an opaque backend target first. Consumers may display `displayPath`, but must not parse `targetKey` (a branded opaque id) or assume it is a local absolute path. +Consumers that share the filesystem's execution world obtain cross-capability coordinates through the provider instead of interpreting that identity: `processPath(target)` returns the canonical absolute path a subprocess can open, `fileUrl(target)` returns its provider-platform `file:` URI, and `contains(parent, child)` tests canonical identity or descendant containment. + ```ts type-equiv /** * A path resolved by a backend into a stable identity. `resolve()` produces @@ -50,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. +`stat` returns metadata (never content), or `undefined` when the target is absent. `type` lets the tool reject directories/special files before reading, and `size` lets it choose `readText` vs `streamText` without probing by failure. Protocol consumers use `readTextBounded(target, maxBytes)` when size validation and the complete UTF-8 read must remain one backend-owned stable operation; composing `stat` with `readText` would admit growth and replacement races. ```ts type-equiv /** @@ -256,4 +258,4 @@ type FsErrorCode = ## The service and the plugin -`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `stat`, `lstat`, `readText`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). +`FileSystem` (`ctx.fs`, abstract) owns the provider primitives: `resolve`, `processPath`, `fileUrl`, `contains`, `stat`, `lstat`, `readText`, `readTextBounded`, `streamText`, `listDir`, `writeText`, and `editText`. `dsh-fs-policy` registers **no service** — it is a plugin that adds policy through the `fs/*` event gate: it decides the write/edit intent waterfalls (supplying `createIfAbsent`/`replaceIfVersion`/`{ version }` or throwing `FS_NOT_OBSERVED`) and records on `fs/observed`. The executor is `dsh-tool-fs`: it reads/writes/edits through `ctx.fs`, dispatches the waterfalls, and emits the recording event. The generated wiring catalog shows the exact `ctx.fs` signatures on [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam). diff --git a/docs/core-data-structures/filesystem.zh.md b/docs/core-data-structures/filesystem.zh.md index 010ec22a5d..6b7642b3f7 100644 --- a/docs/core-data-structures/filesystem.zh.md +++ b/docs/core-data-structures/filesystem.zh.md @@ -12,6 +12,8 @@ 每个操作首先将用户提供的路径解析为不透明的后端目标。消费方可以显示 `displayPath`，但禁止解析 `targetKey`（一个品牌化的不透明 id），也不得假设它是本地绝对路径。 +与文件系统共享执行世界的消费方通过提供方获取跨能力坐标，而不是解释该身份：`processPath(target)` 返回子进程可以打开的规范化绝对路径，`fileUrl(target)` 返回采用提供方平台语法的 `file:` URI，`contains(parent, child)` 则检查规范化身份相等或后代包含关系。 + ```ts type-equiv /** * A path resolved by a backend into a stable identity. `resolve()` produces @@ -50,7 +52,7 @@ type FsTargetKey = Branded<'FsTargetKey'> type FsVersion = Branded<'FsVersion'> ``` -`stat` 返回元数据（从不返回内容），目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件；`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。 +`stat` 返回元数据（从不返回内容），目标不存在时返回 `undefined`。`type` 让工具在读取前拒绝目录或特殊文件；`size` 让工具无需通过失败探测即可选择 `readText` 还是 `streamText`。当大小校验与完整 UTF-8 读取必须保持为一个由后端负责的稳定操作时，协议消费方使用 `readTextBounded(target, maxBytes)`；组合 `stat` 与 `readText` 会容许文件增长与替换竞态。 ```ts type-equiv /** @@ -256,4 +258,4 @@ type FsErrorCode = ## 服务与插件 -`FileSystem`（`ctx.fs`，abstract）拥有提供方原语：`resolve`、`stat`、`lstat`、`readText`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件：对写入/编辑意图 waterfall 作出决策（提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`，或抛出 `FS_NOT_OBSERVED`），并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`：它通过 `ctx.fs` 读取/写入/编辑，分发 waterfall，并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 +`FileSystem`（`ctx.fs`，abstract）拥有提供方原语：`resolve`、`processPath`、`fileUrl`、`contains`、`stat`、`lstat`、`readText`、`readTextBounded`、`streamText`、`listDir`、`writeText` 与 `editText`。`dsh-fs-policy` **不注册服务**——它是一个通过 `fs/*` 事件门禁添加策略的插件：对写入/编辑意图 waterfall 作出决策（提供 `createIfAbsent`/`replaceIfVersion`/`{ version }`，或抛出 `FS_NOT_OBSERVED`），并在 `fs/observed` 上记录。执行器是 `dsh-tool-fs`：它通过 `ctx.fs` 读取/写入/编辑，分发 waterfall，并 emit 记录事件。生成的 wiring 目录在 [services.md](../cordis-catalog/services.md#ctxfs--filesystem-abstract-seam) 中展示确切的 `ctx.fs` 签名。 diff --git a/docs/core-data-structures/subprocess.i18n.yaml b/docs/core-data-structures/subprocess.i18n.yaml index 5d59f6fd16..fca7947aae 100644 --- a/docs/core-data-structures/subprocess.i18n.yaml +++ b/docs/core-data-structures/subprocess.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/core-data-structures/subprocess.md -subprocess.md: 8189795c5acf2fc1882d9e5bc1c006f49c327e2f -subprocess.zh.md: 370a1a05807eefbe09a896ea8bcca1e4ab0a6b83 +subprocess.md: eae497530ff54a7a259c23a2d0f665f3313488f5 +subprocess.zh.md: c3504ef7cd42448a15341f7f92d0f309a6216de1 diff --git a/docs/core-data-structures/subprocess.md b/docs/core-data-structures/subprocess.md index 8189795c5a..c866276b8e 100644 --- a/docs/core-data-structures/subprocess.md +++ b/docs/core-data-structures/subprocess.md @@ -2,9 +2,13 @@ English | [中文](subprocess.zh.md) -The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends — the [bash executor family](bash.md) (collect-mode batch output), the LSP host (piped protocol streams + a collected stderr tail), and the ACP subagent backend (piped protocol streams + inherited stderr). This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. +The subprocess seam is split across interface ([dsh-subprocess](../../packages/subprocess/subprocess), `ctx.subprocess`) and implementation ([dsh-subprocess-local](../../packages/subprocess/subprocess-local)); its consumers are other capability seams and out-of-process backends: the [bash executor family](bash.md) uses collected batch output, the LSP and Code Runtime hosts use raw protocol pipes, the PTY backend uses the terminal primitive, and the ACP subagent backend uses piped ndjson plus inherited stderr. This seam owns the managed `DSH_*` environment namespace, the shared credential scrub (`scrubbedParentEnv`), and the `CollectedOutput` shape; [dsh-bash](../../packages/bash/bash) re-exports the vocabulary so bash consumers keep one import root. -Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) +Source: [`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) and [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) + +## Execution-world coordinates + +One provider's `cwd`, `runtimeRoot`, executable paths, ordinary processes, and terminal sessions inhabit the same path and process namespace as the mounted filesystem provider. `resolveExecutable(command, env?, signal?)` verifies absolute executable paths or resolves bare names through the provider's scrubbed `PATH` plus deliberate overrides. Consumers use `runtimeRoot` for private materialized helpers and never assume a host path exists in that world. ## Managed environment namespace and captured output @@ -234,6 +238,12 @@ interface SubprocessOutcome { } ``` +## Terminal-process primitive + +`spawnTerminal(spec)` is the non-pipe process primitive. The provider allocates the controlling terminal and owns raw UTF-8 byte transport, foreground-process-group inspection and signalling, TERM-to-KILL cleanup, and whole-session quiescence. The PTY backend remains responsible for prompt detection, readiness inference, scrollback, sandbox policy, and persistent-session ownership; ordinary `spawn()` cannot reconstruct controlling-terminal semantics. + +The terminal spec fully specifies argv, cwd, environment overrides, dimensions, cleanup grace, and optional cancellation. Its handle exposes `pid`, ordered output, `done`, `write`, `inspectForeground`, `signalForeground`, `terminate`, and `waitForExit`; the exact public shapes are generated into the [`ctx.subprocess` service catalog](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam). + ## Service behavior -The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines `spawn` only; [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) is the local implementation (detached trees, per-disposition wiring, credential scrub, terminate-and-join disposal). See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the seam contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for the mechanics. +The abstract [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam defines execution-world coordinates, executable lookup, ordinary `spawn`, and `spawnTerminal`. [`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) implements them with temporary runtime storage, detached process trees, per-disposition wiring, credential scrubbing, `node-pty`, platform process inspection, and terminate-and-join disposal. See [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md) for the interface contract and [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md) for local mechanics. diff --git a/docs/core-data-structures/subprocess.zh.md b/docs/core-data-structures/subprocess.zh.md index 370a1a0580..c3504ef7cd 100644 --- a/docs/core-data-structures/subprocess.zh.md +++ b/docs/core-data-structures/subprocess.zh.md @@ -1,14 +1,18 @@ -# 子进程 +# 进程管理器 [English](subprocess.md) | 中文 -子进程 seam 分为接口（[dsh-subprocess](../../packages/subprocess/subprocess)，`ctx.subprocess`）与实现（[dsh-subprocess-local](../../packages/subprocess/subprocess-local)）；它的消费方是其他能力 seam 与进程外后端：[bash 执行器家族](bash.md)使用收集模式（collect）的批量输出，LSP 主机使用管道化的协议流 + 收集的 stderr 尾部，ACP（Agent Client Protocol）subagent 后端则使用管道化的协议流 + inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除（`scrubbedParentEnv`）与 `CollectedOutput` 形状；[dsh-bash](../../packages/bash/bash) 重导出这套词汇，使 bash 消费方保持单一导入入口。 +进程管理器 seam 分为接口（[dsh-subprocess](../../packages/subprocess/subprocess)，`ctx.subprocess`）与实现（[dsh-subprocess-local](../../packages/subprocess/subprocess-local)）；它的消费方是其他能力 seam 与进程外后端：[bash 执行器家族](bash.md)使用收集模式的批量输出，LSP 与 Code Runtime 主机使用原始协议管道，PTY 后端使用终端原语，ACP（Agent Client Protocol）subagent 后端则使用管道化 ndjson 加 inherit 的 stderr。该 seam 拥有受管的 `DSH_*` 环境命名空间、共享的凭据清除（`scrubbedParentEnv`）与 `CollectedOutput` 形状；[dsh-bash](../../packages/bash/bash) 重导出这套词汇，使 bash 消费方保持单一导入入口。 -源码：[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) +源码：[`packages/subprocess/subprocess/src/types.ts`](../../packages/subprocess/subprocess/src/types.ts) 与 [`packages/subprocess/subprocess/src/index.ts`](../../packages/subprocess/subprocess/src/index.ts) + +## 执行世界坐标 + +一个提供方的 `cwd`、`runtimeRoot`、可执行文件路径、普通进程与终端会话，和挂载的文件系统提供方处于同一路径与进程命名空间。`resolveExecutable(command, env?, signal?)` 验证绝对可执行文件路径，或通过提供方清理后的 `PATH` 加有意覆盖来解析裸名称。消费方使用 `runtimeRoot` 存放私有物化辅助程序，绝不假设该执行世界中存在某条宿主路径。 ## 受管环境命名空间与捕获的输出 -`DSH_*` 变量是归 Harness 所有的子进程事实；实现会在合并调用方显式 `env` 之前丢弃环境中已有的 `DSH_*` 名称，因此当前事实只会以有意提供的字符串条目形式到达，而显式的 `undefined` tombstone 会删除普通环境中已有的值。每条被收集的流都通过 `CollectedOutput` 报告自身的截断与 spill 恢复状态。 +`DSH_*` 变量是归 Harness 所有的子进程事实；实现会在合并调用方显式 `env` 之前丢弃环境中已有的 `DSH_*` 名称，因此当前事实只会以有意提供的条目形式到达，每条被收集的流都通过 `CollectedOutput` 报告自身的截断与 spill 恢复状态。 ```ts type-equiv /** One environment key inside the managed {@link DSH_ENV_PREFIX} namespace. */ @@ -32,7 +36,7 @@ interface CollectedOutput { } ``` -## Node 风格的 stdio 处置方式（disposition） +## Node 形状的 stdio 处置方式（disposition） 每条流的处置方式都显式给出，由各消费方自行选择：原始管道用于协议分帧（LSP JSON-RPC、ACP ndjson），inherit 用于直通的诊断输出，收集模式用于有界的批量输出；其中 spill 文件是可选的，因此诊断尾部（语言服务器的 stderr）可以只在内存中缓冲，不留下任何文件。 @@ -84,7 +88,7 @@ interface SubprocessStdio { ## 完全显式的 spawn spec -该 seam 不应用任何默认值：每项处置方式、限制与目录都在 spec 上显式给出，因此由调用方自己的配置决定它们，而不是由某个隐藏的子进程服务默认值决定。`argv` 绝不经过 shell 解释。 +该 seam 不应用任何默认值：每项处置方式、限制与目录都在 spec 上显式给出，因此由调用方自己的配置决定它们，而不是由某个隐藏的进程管理器默认值决定。`argv` 绝不经过 shell 解释。 ```ts type-equiv /** @@ -101,11 +105,10 @@ interface SubprocessSpawnSpec { /** Per-stream stdio dispositions. */ stdio: SubprocessStdio /** - * Positive finite grace period in milliseconds, no greater than - * `MAX_TIMER_DELAY_MS`, for the {@link SubprocessHandle.terminate} escalation - * and for draining still-open collected pipes after the process exits (an - * inherited descriptor held by a surviving descendant cannot hold the - * outcome open indefinitely). + * Grace period in milliseconds for the {@link SubprocessHandle.terminate} + * escalation and for draining still-open collected pipes after the process + * exits (an inherited descriptor held by a surviving descendant cannot hold + * the outcome open indefinitely). */ graceMs: number /** @@ -116,18 +119,19 @@ interface SubprocessSpawnSpec { signal?: AbortSignal | undefined /** * Explicit environment entries merged onto the implementation's scrubbed - * parent base (see `scrubbedParentEnv`), with no namespace validation. A - * string is a deliberate caller opt-in, so a forwarded credential-shaped - * entry or current `DSH_*` fact survives the scrub; `undefined` is a - * tombstone that removes an ordinary ambient entry from the child. + * parent base (see `scrubbedParentEnv`), with no namespace validation: + * every entry is a deliberate caller opt-in, so a forwarded + * credential-shaped entry or a current `DSH_*` fact survives precisely + * because this layer merges after the scrub that drops its ambient + * namesake. */ - env?: NodeJS.ProcessEnv | undefined + env?: Record | undefined } ``` ## 句柄：流、读取器与以进程树为范围的终止 -spawn 会立即返回一个活动句柄。收集模式的读取器接受全流字节偏移量且从不消费，因此独立的读取器不会抢走彼此的增量；管道化的流归调用方所有。终止在每个平台上都以进程树为范围：`terminate()`（唯一的终止动词）执行 SIGTERM→宽限期→SIGKILL 升级，`waitForExit()` 观察整棵进程树。这足以让消费方构建自己的拆卸阶梯；ACP 后端的 `disposeAcpChild` 以 stdin EOF 开始，即为仓库内模板。 +spawn 会立即返回一个实时句柄。收集模式的读取器接受全流字节偏移量且从不消费，因此独立的读取器不会抢走彼此的增量；管道化的流归调用方所有。终止在每个平台上都以进程树为范围：`terminate()`（唯一的终止动词）执行 SIGTERM→宽限期→SIGKILL 升级，`waitForExit()` 观察整棵进程树——这足以让消费方构建自己的拆卸阶梯（ACP 后端以 stdin EOF 打头的 `disposeAcpChild` 即是模板）。 ```ts type-equiv /** @@ -234,6 +238,12 @@ interface SubprocessOutcome { } ``` +## 终端进程原语 + +`spawnTerminal(spec)` 是非管道进程原语。提供方分配控制终端，并负责原始 UTF-8 字节传输、前台进程组检查与信号发送、TERM→KILL 清理，以及整个会话的完全停稳。PTY 后端仍负责提示符检测、就绪推断、scrollback、沙箱策略和持久会话所有权；普通 `spawn()` 无法重建控制终端语义。 + +终端 spec 完全指定 argv、cwd、环境覆盖、尺寸、清理宽限期与可选取消。其句柄公开 `pid`、有序输出、`done`、`write`、`inspectForeground`、`signalForeground`、`terminate` 和 `waitForExit`；确切的公共形状生成到 [`ctx.subprocess` 服务目录](../cordis-catalog/services.md#ctxsubprocess--subprocessservice-abstract-seam)中。 + ## 服务行为 -抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 只定义 `spawn`；[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 是本地实现（detached 进程树、按处置方式接线的流、凭据清除、先终止再等待退出的 dispose（资源释放））。seam 契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md)，具体机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 +抽象的 [`SubprocessService`](../../packages/subprocess/subprocess/src/index.ts) seam 定义执行世界坐标、可执行文件查找、普通 `spawn` 与 `spawnTerminal`。[`LocalSubprocessService`](../../packages/subprocess/subprocess-local/src/index.ts) 以临时运行时存储、detached 进程树、按处置方式接线、凭据清除、`node-pty`、平台进程检查，以及先终止再等待退出的资源释放实现这些能力。接口契约见 [`dsh-subprocess`](../../packages/subprocess/subprocess/README.md)，本地机制见 [`dsh-subprocess-local`](../../packages/subprocess/subprocess-local/README.md)。 diff --git a/knip.json b/knip.json index ede8b2c738..20bcb5ff2d 100644 --- a/knip.json +++ b/knip.json @@ -383,6 +383,16 @@ "tests/**/*.ts" ] }, + "packages/code-runtime/code-runtime-subprocess": { + "entry": [ + "tests/**/*.spec.ts", + "tests/**/*.e2e.ts" + ], + "project": [ + "src/**/*.ts", + "tests/**/*.ts" + ] + }, "packages/llm/llm-deepseek": { "entry": [ "tests/**/*.spec.ts", diff --git a/packages/README.i18n.yaml b/packages/README.i18n.yaml index 8aa9b92b91..c5be7e67d6 100644 --- a/packages/README.i18n.yaml +++ b/packages/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/README.md -README.md: 229feae568ba6e40a9c633696097eff46fd5bc95 -README.zh.md: b84aef020a7e3edf305df709d399fbc7b093b6a3 +README.md: 7fcd98bd19ca3291f0472af57841f2f763bb346f +README.zh.md: 7c4aed9b343ec57001883e094ea3dd0d73e2a920 diff --git a/packages/README.md b/packages/README.md index 229feae568..a711e4d303 100644 --- a/packages/README.md +++ b/packages/README.md @@ -19,7 +19,7 @@ Packages live at `packages///`; groups are containers, while names r | [`subprocess/`](subprocess/README.md) | Subprocess capability family: spawn seam + local process-tree implementation | Product — stable surface | | [`bash/`](bash/README.md) | Bash capability family: executor seam, local impl, model-facing tool | Product — stable surface | | [`pty/`](pty/README.md) | Persistent PTY capability family: owner-scoped sessions, local implementation, and model-facing tools | Product — stable surface | -| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: the runtime seam for model-written programs + a worker-thread backend | Product — stable surface | +| [`code-runtime/`](code-runtime/README.md) | Code-execution capability family: runtime seam plus local worker and filesystem/subprocess backends | Product — stable surface | | [`sandbox/`](sandbox/README.md) | Process-confinement seam; bwrap/Landlock/Seatbelt backends | Product — stable surface | | [`fs/`](fs/README.md) | Filesystem capability family: seam, local impl, model-facing file tools, bash-backed discovery tools | Product — stable surface | | [`lsp/`](lsp/README.md) | LSP capability family: seam, generic stdio provider, and the `lsp` tool | Product — stable surface | diff --git a/packages/README.zh.md b/packages/README.zh.md index b84aef020a..41380317f8 100644 --- a/packages/README.zh.md +++ b/packages/README.zh.md @@ -19,7 +19,7 @@ | [`subprocess/`](subprocess/README.md) | 进程管理能力系列：spawn seam + 本地进程树实现 | 产品：稳定表面 | | [`bash/`](bash/README.md) | Bash 能力系列：执行器 seam、本地实现、面向模型的工具 | 产品：稳定表面 | | [`pty/`](pty/README.md) | 持久 PTY 能力系列：按所有者隔离的会话、本地实现和面向模型的工具 | 产品：稳定表面 | -| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列：面向模型所写程序的运行时 seam + worker 线程后端 | 产品：稳定表面 | +| [`code-runtime/`](code-runtime/README.md) | 代码执行能力系列：运行时 seam、本地 worker 后端及文件系统／进程管理后端 | 产品：稳定表面 | | [`sandbox/`](sandbox/README.md) | 进程限制 seam；bwrap/Landlock/Seatbelt 后端 | 产品：稳定表面 | | [`fs/`](fs/README.md) | 文件系统能力系列：seam、本地实现、面向模型的文件工具、bash 后端发现工具 | 产品：稳定表面 | | [`lsp/`](lsp/README.md) | LSP 能力系列：seam、通用 stdio 提供方和 `lsp` 工具 | 产品：稳定表面 | diff --git a/packages/code-runtime/code-runtime-worker/README.md b/packages/code-runtime/code-runtime-worker/README.md index 590b79dcd1..3de3bc622e 100644 --- a/packages/code-runtime/code-runtime-worker/README.md +++ b/packages/code-runtime/code-runtime-worker/README.md @@ -34,7 +34,7 @@ Every field is validated and defaulted; `maxOutputBytes` is a safe integer of at Source mode loads erasable-only `src/worker.ts` through Node's native type stripping. Its transitive runtime closure contains only Node built-ins and relative source modules, so a fresh checkout never requires a sibling workspace package's unbuilt `lib/` export. The worker-local and session-owned JSON boundaries both flatten and rebuild validated values around the message port so application nesting never reaches structured clone. Built mode passes the sibling `lib/worker.cjs` as a filesystem path because pkg's VFS Worker hook expects CommonJS; the same path works under ordinary Node. The repository-wide requirement to exercise this published entry path belongs to the [testing policy](../../../docs/testing.md). -The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers are source-private implementation details. +The SDK surface is the default/named `WorkerCodeRuntime` class plus `Config`. The `./runtime-host` subpath shares type stripping, binding validation/dispatch, lossless JSON transport, and output accounting with sibling worker-based implementations; it is implementation support, not a plugin. The operational `./worker` subpath exists only as the packaged spawn entry; the wire protocol and bootstrap helpers remain source-private. ## Model Experience diff --git a/packages/code-runtime/code-runtime-worker/src/runtime-host.ts b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts new file mode 100644 index 0000000000..334bd82e38 --- /dev/null +++ b/packages/code-runtime/code-runtime-worker/src/runtime-host.ts @@ -0,0 +1,222 @@ +/** Shared host mechanics for local and subprocess-hosted TypeScript worker runtimes. */ + +import { stripTypeScriptTypes } from 'node:module' +import type { + CodeBindingNamespace, + CodeJsonValue, + CodeRunFailure, + CodeRunRequest, + CodeRunResult, +} from '@deepseek-ai/dsh-code-runtime' +import { jsonStringBytesUpTo, jsonValueBytesUpTo, truncateJsonStringBytes } from './output-json.ts' +import { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' +import type { WorkerJsonWire } from './worker-json.ts' + +/** Smallest cap that can represent an empty log array and failure message. */ +export const MIN_RUNTIME_OUTPUT_BYTES = 4 + +const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/ +const RESERVED_WORDS = new Set([ + 'await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', 'delete', 'do', + 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if', 'import', 'in', + 'instanceof', 'new', 'null', 'return', 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', + 'var', 'void', 'while', 'with', 'yield', 'let', 'static', 'implements', 'interface', 'package', + 'private', 'protected', 'public', 'arguments', 'eval', +]) +const RESERVED_ERROR_PROPERTIES = new Set(['name', 'message', 'stack']) +const STRIP_WRAP = { prefix: 'async function __dsh_program__() {\n', suffix: '\n}' } as const + +/** One validated binding call received from an isolated worker. */ +export interface RuntimeBindingCall { + /** Correlation id supplied by the isolated worker. */ + readonly id: number + /** Injected namespace global. */ + readonly global: string + /** Declared namespace function. */ + readonly name: string + /** Untrusted lossless-JSON wire payload. */ + readonly args: unknown +} + +/** One host reply to an isolated worker binding call. */ +export type RuntimeBindingReply = + | { readonly type: 'reply'; readonly id: number; readonly ok: true; readonly value: WorkerJsonWire } + | { readonly type: 'reply'; readonly id: number; readonly ok: false; readonly message: string } + +/** + * Render an unknown thrown value without assuming it is an Error. + * @param error - thrown or rejected value. + * @returns the caller-facing diagnostic text. + */ +export function runtimeErrorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error) +} + +/** + * Strip erasable TypeScript while preserving the program's body coordinates. + * @param program - model-written async-function body. + * @returns JavaScript source with the wrapper removed. + */ +export function stripRuntimeProgram(program: string): string { + const stripped = stripTypeScriptTypes(STRIP_WRAP.prefix + program + STRIP_WRAP.suffix) + return stripped.slice(STRIP_WRAP.prefix.length, stripped.length - STRIP_WRAP.suffix.length) +} + +/** + * Validate binding globals and typed-error declarations shared by worker runtimes. + * @param request - code-runtime request carrying the namespaces. + * @param implementationName - package name used in seam-misuse diagnostics. + * @returns namespaces indexed by their injected global. + */ +export function validateRuntimeBindings( + request: CodeRunRequest, + implementationName: string, +): Map { + const bindings = new Map() + for (const namespace of request.bindings) { + if (!IDENTIFIER.test(namespace.global) || RESERVED_WORDS.has(namespace.global)) { + throw new Error(`${implementationName}: binding global ${JSON.stringify(namespace.global)} is not a usable identifier`) + } + if (namespace.global === 'console' || bindings.has(namespace.global)) { + throw new Error(`${implementationName}: duplicate binding global ${JSON.stringify(namespace.global)}`) + } + bindings.set(namespace.global, namespace) + } + + const errorClassNames = new Set() + for (const namespace of request.bindings) { + const descriptor = namespace.errorClass + if (descriptor === undefined) continue + if (!IDENTIFIER.test(descriptor.name) || RESERVED_WORDS.has(descriptor.name)) { + throw new Error(`${implementationName}: binding error class ${JSON.stringify(descriptor.name)} is not a usable identifier`) + } + if (descriptor.name === 'console' || bindings.has(descriptor.name) || errorClassNames.has(descriptor.name)) { + throw new Error(`${implementationName}: duplicate injected global ${JSON.stringify(descriptor.name)}`) + } + if (descriptor.memberNameProperty.length === 0 || RESERVED_ERROR_PROPERTIES.has(descriptor.memberNameProperty)) { + throw new Error(`${implementationName}: binding error member property ${JSON.stringify(descriptor.memberNameProperty)} is not usable`) + } + errorClassNames.add(descriptor.name) + } + return bindings +} + +/** + * Resolve one untrusted worker call through a declared host binding. + * @param call - parsed call envelope from the isolated worker. + * @param bindings - namespaces returned by {@link validateRuntimeBindings}. + * @returns a lossless-JSON success or stable rejection reply. + */ +export async function invokeRuntimeBinding( + call: RuntimeBindingCall, + bindings: ReadonlyMap, +): Promise { + const functions = bindings.get(call.global)?.functions + const fn = functions !== undefined && Object.hasOwn(functions, call.name) ? functions[call.name] : undefined + if (typeof fn !== 'function') { + return { type: 'reply', id: call.id, ok: false, message: `unknown binding ${JSON.stringify(`${call.global}.${call.name}`)}` } + } + const args = decodeWorkerJson(call.args) + if (args === undefined) { + return { type: 'reply', id: call.id, ok: false, message: 'binding arguments must be lossless JSON' } + } + try { + const resolved = await fn(args) + let value: CodeJsonValue | undefined + try { + value = snapshotCodeJsonValue(resolved) + } catch { + value = undefined + } + if (value === undefined) { + return { type: 'reply', id: call.id, ok: false, message: 'binding resolution must be lossless JSON' } + } + return { type: 'reply', id: call.id, ok: true, value: encodeWorkerJson(value) } + } catch (error: unknown) { + return { type: 'reply', id: call.id, ok: false, message: runtimeErrorMessage(error) } + } +} + +/** One run's combined outer-output ledger; binding values never enter it. */ +export class RuntimeOutputLedger { + private bytes = 2 + private entries = 0 + + /** @param maxBytes - hard cap for logs plus completion or failure payload. */ + constructor(private readonly maxBytes: number) {} + + /** + * Admit one exact log entry. + * @param text - candidate log entry. + * @param sink - ordered retained log list. + * @returns false when the hard cap was crossed. + */ + admit(text: string, sink: string[]): boolean { + const separatorBytes = this.entries > 0 ? 1 : 0 + const stringBytes = jsonStringBytesUpTo(text, this.maxBytes - this.bytes - separatorBytes) + if (stringBytes === undefined) return false + this.bytes += stringBytes + separatorBytes + this.entries += 1 + sink.push(text) + return true + } + + /** + * Finalize a successful completion against the combined cap. + * @param logs - retained ordered logs. + * @param value - optional lossless-JSON completion. + * @returns the completion or output-limit result. + */ + success(logs: string[], value?: CodeJsonValue): CodeRunResult { + if (value !== undefined && jsonValueBytesUpTo(value, this.maxBytes - this.bytes) === undefined) return this.limit(logs) + return { logs, ...value !== undefined ? { value } : {} } + } + + /** + * Finalize one failure diagnostic against the combined cap. + * @param logs - retained ordered logs. + * @param error - structured runtime failure. + * @returns the failure or output-limit result. + */ + failure(logs: string[], error: CodeRunFailure): CodeRunResult { + if (jsonStringBytesUpTo(error.message, this.maxBytes - this.bytes) === undefined) return this.limit(logs) + return { logs, error } + } + + /** + * Build an explicit output-limit failure with a fitting log prefix. + * @param logs - ordered logs observed before the limit. + * @returns bounded output-limit result. + */ + limit(logs: string[]): CodeRunResult { + const fullMessage = `outer output exceeded ${this.maxBytes} bytes` + const messageBytes = fullMessage.length + 2 + const retained: string[] = [] + let retainedBytes = 2 + const logBudget = this.maxBytes - messageBytes + for (const text of logs) { + const separatorBytes = retained.length > 0 ? 1 : 0 + const availableBytes = logBudget - retainedBytes - separatorBytes + const stringBytes = jsonStringBytesUpTo(text, availableBytes) + if (stringBytes !== undefined) { + retained.push(text) + retainedBytes += stringBytes + separatorBytes + continue + } + const prefix = truncateJsonStringBytes(text, availableBytes) + if (prefix.length > 0) { + const prefixBytes = jsonStringBytesUpTo(prefix, availableBytes) + /* v8 ignore next -- truncateJsonStringBytes guarantees the same bound. */ + if (prefixBytes === undefined) throw new Error('output ledger produced an oversized log prefix') + retained.push(prefix) + retainedBytes += prefixBytes + separatorBytes + } + break + } + const message = truncateJsonStringBytes(fullMessage, this.maxBytes - retainedBytes) + return { logs: retained, error: { kind: 'output-limit', message } } + } +} + +export { decodeWorkerJson, encodeWorkerJson, snapshotCodeJsonValue } from './worker-json.ts' +export type { WorkerJsonWire } from './worker-json.ts' diff --git a/packages/code-runtime/code-runtime-worker/tsconfig.json b/packages/code-runtime/code-runtime-worker/tsconfig.json index 555b3d161e..340e68e848 100644 --- a/packages/code-runtime/code-runtime-worker/tsconfig.json +++ b/packages/code-runtime/code-runtime-worker/tsconfig.json @@ -8,9 +8,6 @@ "src" ], "references": [ - { - "path": "../../core/session" - }, { "path": "../../../vendor/cosmokit" }, diff --git a/packages/code-runtime/code-runtime-worker/tsdown.config.ts b/packages/code-runtime/code-runtime-worker/tsdown.config.ts index 1c40637722..dac6f12a65 100644 --- a/packages/code-runtime/code-runtime-worker/tsdown.config.ts +++ b/packages/code-runtime/code-runtime-worker/tsdown.config.ts @@ -1,9 +1,10 @@ import { defineConfig } from 'tsdown' /** - * Build the index and worker as separate single-entry bundles. The sibling `worker.cjs` is loaded - * by file and must be CommonJS for pkg's VFS Worker hook. A multi-entry build emits an unlisted - * shared chunk omitted by the package's exact `files` whitelist; separate builds inline it. + * Build the plugin, reusable runtime host, and worker as separate bundles. The + * sibling `worker.cjs` is loaded by file and must be CommonJS for pkg's VFS + * Worker hook. Separate builds inline shared implementation instead of + * emitting an unlisted chunk outside the exact `files` whitelist. */ export default defineConfig([ { @@ -16,6 +17,16 @@ export default defineConfig([ dts: false, clean: false, }, + { + entry: ['lib/types/runtime-host.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, + }, { entry: ['lib/types/worker.js'], outDir: 'lib', diff --git a/packages/code-runtime/code-runtime/README.md b/packages/code-runtime/code-runtime/README.md index bb1c20d00a..7014bf517a 100644 --- a/packages/code-runtime/code-runtime/README.md +++ b/packages/code-runtime/code-runtime/README.md @@ -34,5 +34,5 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`run()` is one-shot** — `logs` arrive only on the resolved `CodeRunResult`; the seam exposes no streaming-log or progress surface for a live program's output. - **A persistent REPL-style kernel is recorded future work** — the no-state-between-runs contract stands until a persistent-kernel backend brings its own logging story ([Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md)). -- **Only the worker-thread backend ships** — `'process'`/`'container'` are declared well-known `isolation` values with no implementation; a hard security boundary awaits a container backend. -- **Intermediate binding values have no byte cap** — implementations remain subject to structured-clone cost and process memory, while a provider or executor may already have imposed its own acquisition bound. +- **No runtime claims a hard security boundary** — both shipped implementations use fresh worker threads; the filesystem/subprocess backend can place them inside a stronger execution world, but no runtime reports `'container'` today. +- **Intermediate binding values are implementation-bounded** — the direct worker backend has no per-binding byte cap; the filesystem/subprocess backend bounds each bridge frame, but repeated or concurrent binding traffic remains subject to process memory. diff --git a/packages/context/workspace-context/tests/workspace-context.spec.ts b/packages/context/workspace-context/tests/workspace-context.spec.ts index 0af12596cc..8620c38981 100644 --- a/packages/context/workspace-context/tests/workspace-context.spec.ts +++ b/packages/context/workspace-context/tests/workspace-context.spec.ts @@ -75,6 +75,14 @@ class RecordingFileSystem extends FileSystem { return { targetKey: FsTargetKey(absolute), displayPath: absolute } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { if (signal !== undefined) this.signals.push(signal) signal?.throwIfAborted() @@ -109,6 +117,12 @@ class RecordingFileSystem extends FileSystem { return this.entries.get(target.targetKey)?.content ?? '' } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + const text = await this.readText(target, signal) + if (Buffer.byteLength(text) > maxBytes) throw new Error('too large') + return text + } + override async streamText(target: FsTarget, signal?: AbortSignal): Promise> { if (signal !== undefined) this.signals.push(signal) signal?.throwIfAborted() diff --git a/packages/cordis/tool-cordis/src/api-catalog.ts b/packages/cordis/tool-cordis/src/api-catalog.ts index bf179462ee..c7b6df254e 100644 --- a/packages/cordis/tool-cordis/src/api-catalog.ts +++ b/packages/cordis/tool-cordis/src/api-catalog.ts @@ -312,6 +312,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ signature: 'abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise', jsDoc: '/**\n * Resolve a model/plugin-supplied path into a stable {@link FsTarget}. May perform I/O (a\n * remote/sandboxed backend may need a round-trip to map a path to a stable identity), hence\n * async even though the local backend only normalizes + realpaths.\n *\n * @param path - the path to resolve; relative paths resolve against `opts.cwd`.\n * @param opts - optional cwd override and cancellation signal.\n * @returns the stable target; the same file yields the same `targetKey`.\n */', }, + { + signature: 'abstract processPath(target: FsTarget): string', + jsDoc: '/**\n * Return the canonical absolute path a subprocess in this filesystem\'s\n * execution world can open. The path is deliberately separate from\n * {@link FsTarget.targetKey}: consumers may pass this value to another OS\n * capability, but must continue treating the target key as opaque.\n * @param target - the resolved target whose process path is required.\n * @returns an absolute path in the backend\'s execution world.\n */', + }, + { + signature: 'abstract fileUrl(target: FsTarget): string', + jsDoc: '/**\n * Return the canonical `file:` URI for a target in this filesystem\'s\n * execution world. Backends own URI encoding because the host platform may\n * differ from the execution platform.\n * @param target - the resolved target to encode.\n * @returns the target\'s canonical file URI.\n */', + }, + { + signature: 'abstract contains(parent: FsTarget, child: FsTarget): boolean', + jsDoc: '/**\n * Test canonical containment without exposing or parsing backend target\n * keys. Both targets must come from this provider.\n * @param parent - canonical directory target.\n * @param child - canonical candidate target.\n * @returns true when `child` is `parent` or a descendant of it.\n */', + }, { signature: 'abstract stat(target: FsTarget, signal?: AbortSignal): Promise', jsDoc: '/**\n * Return target metadata, or `undefined` when the target does not exist.\n * @param target - the resolved target to stat.\n * @param signal - aborts the metadata round-trip.\n * @returns metadata only, never content; undefined for an absent target.\n */', @@ -324,6 +336,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ signature: 'abstract readText(target: FsTarget, signal?: AbortSignal): Promise', jsDoc: '/**\n * Read the whole regular text file as a single decoded string.\n * @param target - the resolved target to read.\n * @param signal - aborts the read.\n * @returns the full decoded UTF-8 content.\n */', }, + { + signature: 'abstract readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise', + jsDoc: '/**\n * Read one regular UTF-8 text file through a backend-owned stable handle,\n * rejecting before more than `maxBytes` are retained. The size check and\n * bytes read are one operation: a caller must not emulate this with\n * {@link stat} followed by {@link readText}, which admits growth and path\n * replacement races between the two calls.\n * @param target - the resolved target to read.\n * @param maxBytes - positive safe-integer byte ceiling.\n * @param signal - aborts the open/read operation.\n * @returns the complete decoded text when it fits.\n */', + }, { signature: 'abstract streamText(target: FsTarget, signal?: AbortSignal): Promise>', jsDoc: '/**\n * Stream the whole regular text file as decoded text chunks (same text\n * semantics as {@link readText}, for large files). The backend owns\n * cross-chunk UTF-8 decoding and binary rejection so the policy layer never\n * touches raw bytes.\n * @param target - the resolved target to read.\n * @param signal - aborts the stream, including between chunks.\n * @returns the chunk iterable, decoded and validated like {@link readText}.\n */', @@ -974,10 +990,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ key: 'subprocess', summary: 'Abstract subprocess service.', methods: [ + { + signature: 'abstract resolveExecutable( command: string, env?: Readonly>, signal?: AbortSignal, ): Promise', + jsDoc: '/**\n * Resolve one configured executable in this provider\'s execution world.\n * Absolute paths are verified; bare names use the provider\'s scrubbed PATH\n * plus explicit environment overrides.\n * @param command - absolute executable path or bare PATH name.\n * @param env - explicit environment entries used for lookup.\n * @param signal - aborts remote or local lookup.\n * @returns a canonical executable path.\n */', + }, { signature: 'abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle', jsDoc: '/**\n * Start one managed child process from a fully-specified spec; this seam\n * applies no defaults.\n * @param spec - argv, directory, stdio dispositions, grace, cancellation, and environment.\n * @returns the live process handle (streams/readers, signalling, outcome promise).\n */', }, + { + signature: 'abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise', + jsDoc: '/**\n * Allocate a real terminal and start one owned process session. This is the\n * only non-pipe process primitive: implementations own terminal byte I/O,\n * foreground groups, signals, and complete session-tree cleanup.\n * @param spec - fully specified argv, cwd, environment, dimensions, grace, and cancellation.\n * @returns the live terminal handle after allocation succeeds.\n */', + }, ], }, { @@ -2879,6 +2903,22 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'SubprocessStdio', declaration: 'export interface SubprocessStdio {\n stdin: SubprocessStdinMode;\n stdout: SubprocessOutputMode;\n stderr: SubprocessOutputMode;\n}', }, + { + name: 'SubprocessTerminalForeground', + declaration: 'export interface SubprocessTerminalForeground {\n processGroupId: number;\n inputWaiting: boolean;\n}', + }, + { + name: 'SubprocessTerminalHandle', + declaration: 'export interface SubprocessTerminalHandle {\n readonly pid: number;\n readonly output: Readable;\n readonly done: Promise;\n write(data: Uint8Array): Promise;\n inspectForeground(): Promise;\n signalForeground(signal: SubprocessTerminalSignal): Promise;\n terminate(): void;\n waitForExit(signal?: AbortSignal): Promise;\n}', + }, + { + name: 'SubprocessTerminalSignal', + declaration: 'export type SubprocessTerminalSignal = \'SIGINT\' | \'SIGTERM\' | \'SIGKILL\' | \'SIGTSTP\' | \'SIGHUP\';', + }, + { + name: 'SubprocessTerminalSpawnSpec', + declaration: 'export interface SubprocessTerminalSpawnSpec {\n argv: readonly string[];\n cwd: string;\n env?: Record | undefined;\n rows: number;\n cols: number;\n graceMs: number;\n signal?: AbortSignal | undefined;\n}', + }, { name: 'SurfaceEvent', declaration: 'export type SurfaceEvent = SessionEvent & {\n surfaceOp: SurfaceOp;\n};', diff --git a/packages/fs/README.i18n.yaml b/packages/fs/README.i18n.yaml index e435f166ad..c1a381cb81 100644 --- a/packages/fs/README.i18n.yaml +++ b/packages/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/README.md -README.md: 108d7d862a1dc6307268e2a93fa00789c952e440 -README.zh.md: 8b037cc3192bf6ceb0f0671d62911a8e035db24c +README.md: b6adabd5744cb2b3dcee78b71815f8e95ba780f1 +README.zh.md: 0841f538932452921d2b0d7d9534f023672f241d diff --git a/packages/fs/README.md b/packages/fs/README.md index 108d7d862a..b6adabd574 100644 --- a/packages/fs/README.md +++ b/packages/fs/README.md @@ -2,16 +2,19 @@ English | [中文](README.zh.md) -The filesystem capability family: provider seam, interchangeable backends, policy, and model-facing tools. All **product** packages. +The filesystem stack: a provider seam (execution-world paths, bounded text IO, and atomic mutation with an optional version guard), a local implementation, a policy gate plugin (observed-state + read-before-edit + version-guarded write/edit), the model-facing file tools + executor, and the bash-backed discovery tools. All **product** packages. | Package | Role | ctx key | |---|---|---| -| [`fs/`](fs/README.md) | Filesystem provider seam and policy-event vocabulary | `ctx.fs` | -| [`fs-local/`](fs-local/README.md) | Local-filesystem backend | registers `ctx.fs` | -| [`fs-sandbox/`](fs-sandbox/README.md) | Sandbox-enforcing backend | registers `ctx.fs` | -| [`fs-policy/`](fs-policy/README.md) | Observed-state and mutation policy | `fs/*` listeners | -| [`tool-fs/`](tool-fs/README.md) | Model-facing file tools | registers on `ctx.tools` | -| [`tool-fs-search/`](tool-fs-search/README.md) | Process-backed discovery tools | registers on `ctx.tools` | -| [`tool-str-replace-editor/`](tool-str-replace-editor/README.md) | Model-facing string-replacement editor | registers on `ctx.tools` | +| `fs/` | Provider seam: canonical process paths/file URIs/containment, text IO, and atomic mutation primitives; owns the `fs/*` policy events | `ctx.fs` | +| `fs-local/` | Local-filesystem `FileSystem` implementation | (registers `ctx.fs`) | +| `fs-sandbox/` | Sandbox-enforcing `FileSystem`: extends `fs-local` and fences write/edit by the per-call mode + workspace root policy (read-only denies, workspace-write contains to the session workspace + temp roots), reads pass through | (registers `ctx.fs`) | +| `fs-policy/` | Policy gate plugin: observed-state + read-before-edit + version-guarded write/edit, via the `fs/*` event gate | (no service — `fs/*` listeners) | +| `tool-fs/` | Model-facing `read`/`write`/`edit` tools AND the executor (reads via `ctx.fs`, owns read windowing, dispatches `fs/*`); preserves filesystem semantics for session-cwd-relative paths and advertises sandbox escalation fields when the mounted `ctx.fs` confines | (registers on `ctx.tools`) | +| `tool-fs-search/` | Model-facing `glob`/`grep` discovery tools when `rg` is available on the bash executor `PATH`, backed by fixed ripgrep commands through `ctx.bash`, NOT by `ctx.fs` provider methods | (registers on `ctx.tools`) | -Backends replace one another behind `ctx.fs`; policy and tools consume the seam independently. Discovery remains process-backed instead of expanding the provider contract. Child READMEs own containment, mutation, schema, and timeout details. +The interface lives at `fs/fs/`. A sandboxed, remote, or project-scoped filesystem backend can replace `fs-local` without touching the seam, the policy gate, or the model-facing tool schemas — `fs-sandbox` is the first such replacement (an in-process path fence over the shared sandbox mode; see [the cross-family fs sandbox Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)). The policy (`fs-policy/`) is a plugin that participates only through the `fs/*` event gate, not a service the tool injects — so dropping it gracefully loses the policy and leaves the unconstrained bare provider rather than breaking the tool. A deployment that loads `tool-fs/` is expected to also load it. The mode fence and the read-before-edit gate are orthogonal and compose. Discovery (`tool-fs-search/`) deliberately does NOT extend the provider seam: search is a process-backed `rg` workflow on the bash executor, so filesystem backends stay free of a universal search contract; its tools register only when that executor can find `rg`, and its results are follow-up-readable when the bash workdir and the `read` root are the same workspace (the co-located deployment its README documents). + +## No timeouts on file IO + +`read`/`write`/`edit` take **no** `timeoutMs`, and the provider seam arms no deadline — unlike bash and web (which consume [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md)) and the bash-backed `glob`/`grep` (whose declared `timeoutMs` is enforced by `@deepseek-ai/dsh-timeout-policy`): those are process-backed, where a deadline can really kill the work. A local syscall is best-effort-abortable at most: a timeout could not force an in-progress `fsync`/`rename` to stop, so a deadline here would be a knob that cannot deliver on its promise. Adding one would also be an implicit default in the exact place explicit-over-implicit forbids. Both reference agents (Claude Code, Codex) leave file IO untimed for the same reason; cancellation still propagates through the tool-execution signal for best-effort abort at syscall boundaries. diff --git a/packages/fs/README.zh.md b/packages/fs/README.zh.md index 8b037cc319..0841f53893 100644 --- a/packages/fs/README.zh.md +++ b/packages/fs/README.zh.md @@ -1,17 +1,20 @@ -# fs/ - 文件系统能力家族 +# fs/：文件系统能力族 [English](README.md) | 中文 -文件系统能力家族：提供方 seam、可互换后端、策略和面向模型工具。这些全是**产品**包。 +文件系统栈包括：提供方 seam（执行世界路径、有界文本 I/O 与带可选版本防护的原子变更）、本地实现、政策门禁插件（已观察状态、编辑前读取、版本防护的写入/编辑）、面向模型的文件工具与执行器，以及基于 bash 的发现工具。全部都是**产品** 包。 -| 包 | 职责 | ctx key | +| 包 | 角色 | ctx 键 | |---|---|---| -| [`fs/`](fs/README.md) | 文件系统提供方 seam 和策略事件词汇 | `ctx.fs` | -| [`fs-local/`](fs-local/README.md) | 本地文件系统后端 | 注册 `ctx.fs` | -| [`fs-sandbox/`](fs-sandbox/README.md) | 强制执行沙箱的后端 | 注册 `ctx.fs` | -| [`fs-policy/`](fs-policy/README.md) | 已观察状态和修改策略 | `fs/*` 监听器 | -| [`tool-fs/`](tool-fs/README.md) | 面向模型的文件工具 | 注册到 `ctx.tools` | -| [`tool-fs-search/`](tool-fs-search/README.md) | 基于进程的发现工具 | 注册到 `ctx.tools` | -| [`tool-str-replace-editor/`](tool-str-replace-editor/README.md) | 面向模型的字符串替换编辑器 | 注册到 `ctx.tools` | +| `fs/` | 提供方 seam：规范化进程路径、文件 URI 与包含关系、文本 I/O 和原子变更原语；拥有 `fs/*` 政策事件 | `ctx.fs` | +| `fs-local/` | 本地文件系统 `FileSystem` 实现 | （注册 `ctx.fs`） | +| `fs-sandbox/` | 强制沙箱的 `FileSystem`：扩展 `fs-local`，并按每次调用的模式与工作区根政策约束写入/编辑（只读模式拒绝，工作区写入模式限制在会话工作区与临时根目录内）；读取直接通过 | （注册 `ctx.fs`） | +| `fs-policy/` | 政策门禁插件：通过 `fs/*` 事件门禁提供已观察状态、编辑前读取和版本防护的写入/编辑 | （无服务，仅有 `fs/*` 监听器） | +| `tool-fs/` | 面向模型的 `read`/`write`/`edit` 工具以及执行器（通过 `ctx.fs` 读取，拥有读取窗口逻辑，分派 `fs/*`）；为会话 cwd 相对路径保留文件系统语义，并在已挂载的 `ctx.fs` 实施约束时声明沙箱升级字段 | （注册到 `ctx.tools`） | +| `tool-fs-search/` | 面向模型的 `glob`/`grep` 发现工具；当 `rg` 位于 bash 执行器 `PATH` 上时注册，通过 `ctx.bash` 运行固定 ripgrep 命令，而不是使用 `ctx.fs` 提供方方法 | （注册到 `ctx.tools`） | -后端可在 `ctx.fs` 后互相替换；策略和工具独立消费该 seam。发现功能仍由进程提供，不扩展提供方契约。子 README 负责围堵、修改、schema 和超时细节。 +接口位于 `fs/fs/`。沙箱化、远程或限定项目作用域的文件系统后端可以替换 `fs-local`，而无需更改 seam、政策门禁或面向模型的工具 schema；`fs-sandbox` 是第一个这样的替代实现（基于共享沙箱模式的进程内路径围栏；见[跨能力族 fs 沙箱 Agent Note](../../.agents/notes/implemented/feature/2026-07-14-cross-family-fs-sandbox.md)）。政策（`fs-policy/`）是一个只通过 `fs/*` 事件门禁参与的插件，不是工具注入的服务；因此移除它会平稳失去政策，留下不受约束的裸提供方，而不会破坏工具。加载 `tool-fs/` 的部署也应加载该插件。模式围栏与编辑前读取门禁彼此正交，可以组合。发现（`tool-fs-search/`）有意不扩展提供方 seam：搜索是在 bash 执行器上运行 `rg`、由进程支持的工作流，因此文件系统后端无需承担通用搜索契约；只有当执行器能找到 `rg` 时，其工具才会注册。如果 bash 工作目录与 `read` 根目录是同一工作区，结果就能继续读取，这也是其 README 所述的共置部署。 + +## 文件 I/O 不设超时 + +`read`/`write`/`edit` **不** 接受 `timeoutMs`，提供方 seam 也不启动 deadline。这与 bash 和 web（两者使用 [`@deepseek-ai/dsh-timeout`](../util/timeout/README.md)）及基于 bash 的 `glob`/`grep` 不同（它们声明的 `timeoutMs` 由 `@deepseek-ai/dsh-timeout-policy` 强制执行）：这些工作由进程支持，deadline 可以实际终止工作。本地系统调用至多只能尽力中止：超时无法强制正在进行的 `fsync`/`rename` 停止，因此这里的 deadline 会成为无法兑现承诺的配置项。在此添加 deadline 还会在「显式优于隐式」明确禁止的地方引入隐式默认值。两个参考 agent（Claude Code、Codex）出于同一原因都不为文件 I/O 计时；取消仍通过工具执行信号传播，在系统调用边界尽力中止。 diff --git a/packages/fs/fs-local/README.i18n.yaml b/packages/fs/fs-local/README.i18n.yaml index cc14be584f..57d2a32f97 100644 --- a/packages/fs/fs-local/README.i18n.yaml +++ b/packages/fs/fs-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md -README.md: 6d344fa3fef7f6bda6c0daa50184661156a925a7 -README.zh.md: 4c94de64561d805684f91f02d5b0dfc375f0d04f +README.md: f5cf2441adcd62e55c6169ceb766f88382e314f5 +README.zh.md: 40f5a83626780bae8f335c80662721bae6cc0b0b diff --git a/packages/fs/fs-local/README.md b/packages/fs/fs-local/README.md index 6d344fa3fe..f5cf2441ad 100644 --- a/packages/fs/fs-local/README.md +++ b/packages/fs/fs-local/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the eight `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. +The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the twelve `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`. ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' @@ -15,8 +15,9 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## Behavior - **`resolve(path, opts?)`** — a relative `path` resolves against `opts.cwd` when the caller supplies one (the model-facing tools pass the calling agent's session cwd — see [the per-session cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)), else `config.cwd` (default `process.cwd()`); an absolute `path` ignores both. `opts.signal` is checked before and after local resolution, while a remote sibling backend may use it to abort its round-trip. The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path. +- **Execution-world coordinates** — `processPath` exposes the target's canonical host path, `fileUrl` encodes that path through Node's platform-aware URL conversion, and `contains` uses platform path semantics to test identity or descendant containment without consumers parsing `targetKey`. - **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence. -- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) decides which to call by size and owns the line windowing. +- **`readText` / `readTextBounded` / `streamText`** — UTF-8 only. `readText` reads the whole file; `readTextBounded` opens one no-follow, nonblocking handle, verifies it is regular, and retains at most `maxBytes + 1` bytes so growth cannot bypass the cap; `streamText` decodes chunks so a huge file need not be held whole in memory. All reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing; protocol consumers such as the LSP host use the stable bounded operation. - **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`. - **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). - **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`). @@ -35,7 +36,7 @@ No direct invalidation; the named consumer owns any request-prefix changes. - **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)). - **An overwrite reads the whole prior file into memory** — solely as the UI diff basis; bounding that pre-read above a size threshold is deferred (`TODO(overwrite-diff-bound)`). -- **Version tokens are `mtimeMs:size`** — an external change that preserves both within the filesystem's timestamp granularity defeats the stale guard. +- **Version tokens depend on filesystem metadata** — they combine device, inode, size, nanosecond mtime, and nanosecond ctime; a storage layer that cannot update any of those facts for a rewrite can still defeat the stale guard. - **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path. - **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits. - **The per-target mutation lock is in-process only** — a writer in another process is caught only by the optional version guard, never serialized. diff --git a/packages/fs/fs-local/README.zh.md b/packages/fs/fs-local/README.zh.md index 4c94de6456..40f5a83626 100644 --- a/packages/fs/fs-local/README.zh.md +++ b/packages/fs/fs-local/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -`ctx.fs` 提供方 seam（[`@deepseek-ai/dsh-fs`](../fs)）的**本地文件系统实现**。它使用宿主文件系统支持八个 `FileSystem` 原语；将其作为插件加载会填充 `ctx.fs`。 +`ctx.fs` 提供方 seam（[`@deepseek-ai/dsh-fs`](../fs)）的**本地文件系统实现**。它使用宿主文件系统支持十二个 `FileSystem` 原语；将其作为插件加载会填充 `ctx.fs`。 ```ts ignore-check import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' @@ -15,27 +15,28 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) ## 行为 - **`resolve(path, opts?)`**：相对 `path` 在调用方提供 `opts.cwd` 时以该值为基准解析（面向模型的工具会传入调用 agent（智能体）的会话 cwd；见[每会话 cwd Agent Note](../../../.agents/notes/implemented/architecture/2026-07-02-fs-per-session-cwd.md)），否则以 `config.cwd` 为基准（默认 `process.cwd()`）；绝对 `path` 会忽略两者。`opts.signal` 会在本地解析前后检查，远程同级后端则可以用它中止往返。`targetKey` 是文件的 `realpath`，因此经符号链接到达同一文件的两个输入路径会共享一个身份，写入/编辑落在链接目标上，同时保留链接。尚不存在的路径在父目录存在时使用 realpath 后的父目录加 basename；只有父目录无法解析时才回退到绝对路径。`displayPath` 是绝对但未经解析的路径。 -- **`stat` / `lstat`**：返回目标元数据；目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`（`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token，`type` 为 `file`/`directory`/`other`，`size` 以字节计）；路径形态的 `lstat` 不跟随最后一个符号链接，报告 `FsPathInfo`，因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消，因此异步探测进行期间发生的中止会报告 `FS_ABORTED`，而非已失效的「不存在」结果。 -- **`readText` / `streamText`**：只支持 UTF-8。`readText` 读取整个文件；`streamText` 按分片流式读取（跨分片解码），因此超大文件无需整体保存在内存中。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本（`FS_NOT_TEXT`）以及非普通文件目标。`read` 工具（`@deepseek-ai/dsh-tool-fs`）按大小决定调用哪个方法，并负责行窗口逻辑。 +- **执行世界坐标**：`processPath` 公开目标的规范化宿主路径，`fileUrl` 通过 Node 的平台感知 URL 转换对该路径编码，`contains` 则使用平台路径语义检查身份相等或后代包含关系，消费方无需解析 `targetKey`。 +- **`stat` / `lstat`**：返回目标元数据；目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`（`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token，`type` 为 `file`/`directory`/`other`，`size` 以字节计）；路径形态的 `lstat` 不跟随最后一个符号链接，报告 `FsPathInfo`，因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消，因此飞行中的中止会报告 `FS_ABORTED`，而非陈旧的不存在结果。 +- **`readText` / `readTextBounded` / `streamText`**：只支持 UTF-8。`readText` 读取整个文件；`readTextBounded` 打开一个不跟随符号链接的非阻塞句柄，确认其为普通文件，并最多保留 `maxBytes + 1` 字节，使文件增长无法绕过上限；`streamText` 按分片解码，因此超大文件无需整体保存在内存中。三者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本（`FS_NOT_TEXT`）以及非普通文件目标。`read` 工具（`@deepseek-ai/dsh-tool-fs`）拥有行窗口逻辑；LSP 主机等协议消费方使用稳定的有界操作。 - **`listDir`**：按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标（`displayPath` 位于所列目录下，`targetKey` 是 realpath 身份）和低成本 stat 元数据（`version`，普通文件另有 `size`）。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`，文件/特殊文件目标报告 `FS_NOT_DIRECTORY`，已中止调用报告 `FS_ABORTED`，权限失败报告 `FS_PERMISSION_DENIED`，其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回，但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。 -- **`writeText`**：原子写入。它会向排他打开的临时文件（`wx`、`0o600`）写入；该文件位于目标旁随机命名的私有暂存目录（`0o700`）内。完成写入和 fsync 后，以 rename 覆盖目标。现有文件的 mode 会保留，新文件默认为 `0o600`；Windows 上的新文件继承目标目录的 DACL，而替换会在写入前把目标 DACL 复制到空临时文件，并通过 `ReplaceFileW` 发布，使原访问策略得以保留（见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)）。`expected` 防护是可选（OPTIONAL）的：省略时无条件创建或覆盖；`createIfAbsent` 创建缺失目标并拒绝现有目标（`FS_NOT_OBSERVED`）；`replaceIfVersion` 只在观察到的版本上替换（目标缺失或版本不匹配均为 `FS_STALE_VERSION`）。 -- **`editText`**：在同一原语之上执行原子式的字面量读取-修改-写入，并通过变更锁按目标串行化。`expected` 防护是可选（OPTIONAL）的：提供时，会在字面量匹配之前校验版本（陈旧编辑报告 `FS_STALE_VERSION`，绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`）；省略时，无条件编辑当前内容。无论哪种情况，目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF，随后恢复文件主要的 CRLF/LF 风格；空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`，未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 +- **`writeText`**：原子写入。它会向排他打开的临时文件（`wx`、`0o600`）写入；该文件位于目标旁随机命名的私有暂存目录（`0o700`）内。完成写入和 fsync 后，以 rename 覆盖目标。现有文件的 mode 会保留，新文件默认为 `0o600`；Windows 上的新文件继承目标目录的 DACL，而替换会在写入前把目标 DACL 复制到空临时文件，并通过 `ReplaceFileW` 发布，使原访问政策得以保留（见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)）。`expected` 防护是可选的：省略时无条件创建或覆盖；`createIfAbsent` 创建缺失目标并拒绝现有目标（`FS_NOT_OBSERVED`）；`replaceIfVersion` 只在观察到的版本上替换（目标缺失或版本不匹配均为 `FS_STALE_VERSION`）。 +- **`editText`**：在同一原语之上依次执行原子的字面量读取、修改和写入，并通过变更锁按目标串行化。`expected` 防护是可选的：提供时，会在字面量匹配之前校验版本（陈旧编辑报告 `FS_STALE_VERSION`，绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`）；省略时，无条件编辑当前内容。无论哪种情况，目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF，随后恢复文件主要的 CRLF/LF 风格；空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`，未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。 -包根目录的 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`（不依赖 Cordis，单独进行单元测试）；`src/index.ts` 是轻量服务接线。 +包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`（不依赖 Cordis，单独进行单元测试）；`src/index.ts` 是轻量服务接线。 ## 模型体验 -通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接产生影响；该消费方在有上限的保留结果中渲染本提供方带行窗口的 UTF-8 内容、变更确认和精确提供方消息，而版本、原子写入机制和目录元数据仍属内部实现。 +通过 [`dsh-tool-fs`](../tool-fs/README.md) 间接产生影响；该消费方把本提供方带行窗口的 UTF-8 内容、变更确认和精确提供方消息渲染为有上限且保留的结果，而版本、原子写入机制和目录元数据保持内部可见。 #### KV Cache 影响 不会直接使缓存失效；具名消费方负责请求前缀的任何变化。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 - **`config.cwd` 不是沙箱**：它是解析默认值，而非约束；绝对路径和 `..` 可以逃逸。请使用更严格的 `ctx.fs` 后端或 `tools/execute` waterfall（瀑布式事件）上的权限插件实施约束（见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)）。 - **覆盖会把整个旧文件读入内存**：只用于 UI diff；在大小阈值之上限制这次预读取的工作延期处理（`TODO(overwrite-diff-bound)`）。 -- **版本 token 是 `mtimeMs:size`**：如果外部变更在文件系统时间戳粒度内保持两者不变，就能绕过陈旧防护。 +- **版本 token 依赖文件系统元数据**：它们组合设备、inode、大小、纳秒级 mtime 和纳秒级 ctime；如果存储层在重写时无法更新其中任何一项事实，仍可能绕过陈旧防护。 - **`editText` 会把整个文件及编辑后的副本保存在内存中**：只有读取路径支持流式处理。 - **二进制检测不对称**：读取只对前 8192 字节执行 NUL 采样，编辑则扫描整个 buffer，因此 NUL 出现在后部的文件可以读取，但编辑会被拒绝。 - **每目标变更锁仅限进程内**：其他进程中的写入方只会被可选版本防护发现，绝不会被串行化。 diff --git a/packages/fs/fs-local/src/fsio.ts b/packages/fs/fs-local/src/fsio.ts index c93e4ddaeb..e95cb14cca 100644 --- a/packages/fs/fs-local/src/fsio.ts +++ b/packages/fs/fs-local/src/fsio.ts @@ -6,7 +6,7 @@ */ import { randomUUID } from 'node:crypto' -import { createReadStream } from 'node:fs' +import { constants, createReadStream } from 'node:fs' import { chmod, lstat, mkdir, open, readFile, realpath, readdir, rename, rm, stat } from 'node:fs/promises' import type { BigIntStats, Dirent, Stats } from 'node:fs' import { basename, dirname, join, resolve } from 'node:path' @@ -369,6 +369,77 @@ export async function readWholeText(target: LocalTarget, signal?: AbortSignal): return decodeUtf8(raw, 'read', target.displayPath) } +/** + * Read one regular UTF-8 file through a single no-follow handle, retaining at + * most `maxBytes + 1` bytes so a concurrent grow cannot bypass the bound. + * @param target - the resolved file to read. + * @param maxBytes - positive safe-integer byte ceiling. + * @param signal - aborts between handle operations. + * @returns the complete decoded text when it fits. + */ +export async function readWholeTextBounded( + target: LocalTarget, + maxBytes: number, + signal?: AbortSignal, +): Promise { + if (!Number.isSafeInteger(maxBytes) || maxBytes <= 0) { + throw new Error('bounded read maxBytes must be a positive safe integer') + } + throwIfAborted(signal, 'read') + let handle: Awaited> + try { + handle = await open( + target.targetKey, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK, + ) + } catch (error: unknown) { + if (isENOENT(error)) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND', { cause: error }) + if (isPermissionError(error)) throw new FsError(`cannot read "${target.displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error }) + throw new FsError(`cannot read "${target.displayPath}" safely: ${errorMessage(error)}`, 'FS_IO_ERROR', { cause: error }) + } + try { + throwIfAborted(signal, 'read') + const info = await handle.stat() + if (!info.isFile()) { + throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE') + } + if (info.size > maxBytes) { + throw new FsError( + `cannot read "${target.displayPath}": ${info.size} bytes exceeds the ${maxBytes}-byte limit`, + 'FS_IO_ERROR', + ) + } + const chunks: Buffer[] = [] + let total = 0 + for (;;) { + throwIfAborted(signal, 'read') + // Allocate in fixed internal chunks so a permissive deployment cap does + // not reserve that entire cap for a small file. Once exactly at the + // bound, one final byte detects concurrent growth without retaining it. + const remaining = total === maxBytes ? 1 : Math.min(64 * 1024, maxBytes - total) + const chunk = Buffer.allocUnsafe(remaining) + const { bytesRead } = await handle.read(chunk, 0, chunk.length, total) + if (bytesRead === 0) break + total += bytesRead + if (total > maxBytes) { + throw new FsError( + `cannot read "${target.displayPath}": file grew past the ${maxBytes}-byte limit while reading`, + 'FS_IO_ERROR', + ) + } + chunks.push(chunk.subarray(0, bytesRead)) + } + throwIfAborted(signal, 'read') + const bytes = chunks.length === 1 ? chunks[0] as Buffer : Buffer.concat(chunks, total) + if (bytes.subarray(0, BINARY_SAMPLE_BYTES).includes(0)) { + throw new FsError(`cannot read "${target.displayPath}": binary file`, 'FS_NOT_TEXT') + } + return decodeUtf8(bytes, 'read', target.displayPath) + } finally { + await handle.close() + } +} + /** * Stream a whole regular UTF-8 text file as decoded text chunks. Same text * semantics as {@link readWholeText} (regular-file check, binary/NUL rejection, diff --git a/packages/fs/fs-local/src/index.ts b/packages/fs/fs-local/src/index.ts index 18433f3f7c..1a266301db 100644 --- a/packages/fs/fs-local/src/index.ts +++ b/packages/fs/fs-local/src/index.ts @@ -5,7 +5,8 @@ */ import { Context } from 'cordis' -import { resolve } from 'node:path' +import { isAbsolute, relative, resolve, sep } from 'node:path' +import { pathToFileURL } from 'node:url' import z from 'schemastery' import { FileSystem, FsError, FsVersion } from '@deepseek-ai/dsh-fs' import type { @@ -27,6 +28,7 @@ import { readForEdit, readTextForDiff, readWholeText, + readWholeTextBounded, resolveLocalTarget, restoreLineEndings, streamWholeText, @@ -90,6 +92,19 @@ export class LocalFileSystem extends FileSystem { return { targetKey: local.targetKey, displayPath: local.displayPath } } + override processPath(target: FsTarget): string { + return String(target.targetKey) + } + + override fileUrl(target: FsTarget): string { + return pathToFileURL(this.processPath(target)).href + } + + override contains(parent: FsTarget, child: FsTarget): boolean { + const path = relative(this.processPath(parent), this.processPath(child)) + return path === '' || (path !== '..' && !path.startsWith(`..${sep}`) && !isAbsolute(path)) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { if (signal?.aborted) throw new FsError('stat aborted', 'FS_ABORTED') const info = await probe(target.targetKey) @@ -111,6 +126,10 @@ export class LocalFileSystem extends FileSystem { return readWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal) } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + return readWholeTextBounded({ displayPath: target.displayPath, targetKey: target.targetKey }, maxBytes, signal) + } + override streamText(target: FsTarget, signal?: AbortSignal): Promise> { return Promise.resolve(streamWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal)) } diff --git a/packages/fs/fs-local/tests/filesystem.spec.ts b/packages/fs/fs-local/tests/filesystem.spec.ts index 61e2c0e999..02beab7323 100644 --- a/packages/fs/fs-local/tests/filesystem.spec.ts +++ b/packages/fs/fs-local/tests/filesystem.spec.ts @@ -10,6 +10,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { pathToFileURL } from 'node:url' import { Context } from 'cordis' import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local' import { FsVersion } from '@deepseek-ai/dsh-fs' @@ -85,6 +86,20 @@ describe('resolve', () => { await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' }) }) + + it('projects process paths, file URLs, and canonical containment', async () => { + await mkdir(join(dir, 'nested')) + await writeFile(join(dir, 'nested', 'file.txt'), 'text') + const root = await fs.resolve('.') + const child = await fs.resolve('nested/file.txt') + const outside = await fs.resolve('..') + + expect(fs.processPath(child)).toBe(await realpath(join(dir, 'nested', 'file.txt'))) + expect(fs.fileUrl(child)).toBe(pathToFileURL(await realpath(join(dir, 'nested', 'file.txt'))).href) + expect(fs.contains(root, root)).toBe(true) + expect(fs.contains(root, child)).toBe(true) + expect(fs.contains(root, outside)).toBe(false) + }) }) describe('stat', () => { @@ -203,6 +218,15 @@ describe('readText / streamText', () => { expect(await fs.readText(await fs.resolve('a.txt'))).toBe('one\ntwo\nthree') }) + it('reads complete text through the stable byte bound', async () => { + await writeFile(join(dir, 'bounded.txt'), '€abc') + const target = await fs.resolve('bounded.txt') + expect(await fs.readTextBounded(target, 6)).toBe('€abc') + await expect(fs.readTextBounded(target, 5)).rejects.toThrow('exceeds the 5-byte limit') + await expect(fs.readTextBounded(target, 0)).rejects.toThrow('positive safe integer') + await expect(fs.readTextBounded(target, 6, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' }) + }) + it('streams the same text', async () => { await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree') const target = await fs.resolve('a.txt') diff --git a/packages/fs/fs-local/tests/fsio.spec.ts b/packages/fs/fs-local/tests/fsio.spec.ts index 15588e40b9..2c5fd7a8f4 100644 --- a/packages/fs/fs-local/tests/fsio.spec.ts +++ b/packages/fs/fs-local/tests/fsio.spec.ts @@ -5,7 +5,7 @@ * policy and lives in `dsh-fs-policy`, so it is not tested here. */ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { chmod, mkdtemp, readFile, rename, rm, stat, symlink, unlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -17,6 +17,7 @@ import { probeNoFollow, readForEdit, readWholeText, + readWholeTextBounded, resolveLocalTarget, restoreLineEndings, streamWholeText, @@ -316,6 +317,66 @@ describe('readWholeText', () => { }) }) +describe('readWholeTextBounded', () => { + it('rejects non-files, binary text, and initial oversize without a full read', async () => { + await expect(readWholeTextBounded(localTarget(dir), 10)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' }) + await writeFile(join(dir, 'large'), '12345') + await expect(readWholeTextBounded(localTarget(join(dir, 'large')), 4)).rejects.toThrow('exceeds the 4-byte limit') + await writeFile(join(dir, 'binary'), Buffer.from([0x61, 0x00, 0x62])) + await expect(readWholeTextBounded(localTarget(join(dir, 'binary')), 3)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' }) + }) + + it('detects growth past the bound on the same open handle', async () => { + const close = vi.fn(async () => {}) + const read = vi.fn(async (buffer: Buffer, offset: number, length: number, position: number) => { + const bytes = position === 0 ? Buffer.from('abc') : Buffer.from('d') + bytes.copy(buffer, offset, 0, Math.min(length, bytes.length)) + return { bytesRead: Math.min(length, bytes.length), buffer } + }) + vi.resetModules() + vi.doMock('node:fs/promises', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + open: async () => ({ + stat: async () => ({ isFile: () => true, size: 3 }), + read, + close, + }), + } + }) + try { + const isolated = await import('../src/fsio.ts') + await expect(isolated.readWholeTextBounded(localTarget('/virtual/growing'), 3)) + .rejects.toThrow('grew past the 3-byte limit') + expect(close).toHaveBeenCalledOnce() + } finally { + vi.doUnmock('node:fs/promises') + vi.resetModules() + } + }) + + it('translates permission and generic open failures', async () => { + const failure: { current: Error & { code?: string } } = { current: Object.assign(new Error('denied'), { code: 'EACCES' }) } + vi.resetModules() + vi.doMock('node:fs/promises', async (importOriginal) => { + const actual = await importOriginal() + return { ...actual, open: async () => { throw failure.current } } + }) + try { + const isolated = await import('../src/fsio.ts') + await expect(isolated.readWholeTextBounded(localTarget('/virtual/denied'), 3)) + .rejects.toMatchObject({ code: 'FS_PERMISSION_DENIED' }) + failure.current = new Error('open broke') + await expect(isolated.readWholeTextBounded(localTarget('/virtual/broken'), 3)) + .rejects.toMatchObject({ code: 'FS_IO_ERROR' }) + } finally { + vi.doUnmock('node:fs/promises') + vi.resetModules() + } + }) +}) + describe('streamWholeText', () => { it('streams the whole file as decoded text', async () => { const file = join(dir, 'a.txt') diff --git a/packages/fs/fs/README.i18n.yaml b/packages/fs/fs/README.i18n.yaml index ddf57a98d0..4a0e675c0c 100644 --- a/packages/fs/fs/README.i18n.yaml +++ b/packages/fs/fs/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/fs/fs/README.md -README.md: 6c80cc22f6f28e792c3458df3390b241b51d8202 -README.zh.md: 4772d799221efbfff9667cbc8b9e1df6af07dcff +README.md: 67079f795c705ab4c9cfa476e0458be04a48c6c8 +README.zh.md: d812a94fab9f4b7e9d15ff78bd1fea3bcc00c0d9 diff --git a/packages/fs/fs/README.md b/packages/fs/fs/README.md index 6c80cc22f6..67079f795c 100644 --- a/packages/fs/fs/README.md +++ b/packages/fs/fs/README.md @@ -2,20 +2,33 @@ English | [中文](README.zh.md) -The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives a backend provides — resolve a path, stat metadata, no-follow path metadata, read/stream text, list directories, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. +The **filesystem provider seam**: an abstract `FileSystem` service (`ctx.fs`) defining the storage primitives in one execution world — resolve paths, expose canonical process paths and file URIs, test containment, read bounded or streaming text, inspect/list metadata, write atomically, and apply a literal edit — without saying HOW. Both mutations take their version guard **optionally**, so `ctx.fs` on its own is a complete, unconstrained text-storage seam. This package also owns the `fs/*` policy event vocabulary the tool dispatches and the policy plugin listens for. -This package is the provider-seam layer of the [filesystem family](../README.md). The [tool](../tool-fs/README.md), [policy](../fs-policy/README.md), and [local](../fs-local/README.md) and [sandboxed](../fs-sandbox/README.md) backends remain separate consumers and implementations; the capability-seam decisions own the split ([foundation](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [filesystem seam](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [provider split](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), [event gate](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)). +This package is the provider-seam layer of the four-layer filesystem stack, split so each concern can evolve (and be swapped) independently (see [the capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md), [the filesystem capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md), [the split-the-filesystem-seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md), and [the file-context event-gate Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)): + +| Layer | Package | Role | +|---|---|---| +| tool / executor | `@deepseek-ai/dsh-tool-fs` | model-facing `read`/`write`/`edit` schemas + read windowing + text rendering; reads/writes/edits via `ctx.fs`, dispatches the `fs/*` events | +| policy | `@deepseek-ai/dsh-fs-policy` | observed-state + read-before-edit + version-guarded write/edit, contributed through the `fs/*` event gate (no service) | +| provider seam | `@deepseek-ai/dsh-fs` (this) | `ctx.fs`: execution-world paths, text IO, and atomic mutation primitives (optional version guard); owns the `fs/*` event vocabulary | +| provider | `@deepseek-ai/dsh-fs-local` | the host-filesystem implementation | + +A future sandboxed, virtual, or remote backend implements this interface and the policy/tool layers don't change. ## Service API (`ctx.fs`) -A backend subclasses `FileSystem` and implements eight primitives. +A backend subclasses `FileSystem` and implements twelve primitives. | Member | Semantics | |---|---| | `resolve(path, opts?)` | Resolve a path into a stable `FsTarget` (opaque `targetKey`, `displayPath`). `opts.cwd` is the base a relative `path` resolves against (a caller supplies its session workspace; absolute paths ignore it; omitted ⇒ the backend default), while `opts.signal` aborts a backend round-trip. Async — a remote backend may need I/O. The same file via different paths must yield the same `targetKey`. | +| `processPath(target)` | Return the canonical absolute path that a subprocess in this provider's execution world can open. This is intentionally distinct from opaque `targetKey`. | +| `fileUrl(target)` | Return the canonical `file:` URI in the execution world's platform syntax. The backend, not the host process, owns encoding. | +| `contains(parent, child)` | Test canonical identity/descendant containment without exposing or parsing target keys. Both targets come from this provider. | | `stat(target, signal?)` | Return `FsInfo` metadata (`version`, `type`, optional `size`), or `undefined` when the target is absent. Never content. | | `lstat(path, opts?, signal?)` | Return `FsPathInfo` metadata without following the final path component when it is a symlink. This is path-shaped so consumers can reject repository-owned symlinks before `resolve` follows them into a target. | | `readText(target, signal?)` | Read the whole regular text file as one decoded string. Owns regular-file checks, UTF-8 decoding, binary/NUL rejection (`FS_NOT_TEXT`). | +| `readTextBounded(target, maxBytes, signal?)` | Read one complete regular UTF-8 file through a backend-owned stable operation, rejecting before retaining more than `maxBytes`. Consumers must not emulate this with `stat` then `readText`, which admits growth and replacement races. | | `streamText(target, signal?)` | Stream the same text as decoded chunks for large files (cross-chunk UTF-8 decoding stays here). | | `listDir(target, signal?)` | List direct directory children in stable name order. Returns entry names, entry types, resolved child targets, and cheap metadata (`version`/file `size` when available); never reads file contents. Missing targets throw `FS_NOT_FOUND`, non-directories throw `FS_NOT_DIRECTORY`, permission failures throw `FS_PERMISSION_DENIED`, and other backend I/O failures throw `FS_IO_ERROR`. Broken/disappeared children may be returned as `other` without metadata; child permission/IO failures fail the whole listing with the same structured codes. | | `writeText(target, content, expected?, signal?)` | Atomic create/replace. `expected` is OPTIONAL: omit ⇒ unconditional create-or-overwrite; supply an `FsWriteIntent` (`createIfAbsent`/`replaceIfVersion`) to guard. | @@ -37,10 +50,6 @@ This package declares three events (see the generated [events catalog](../../../ `FsTargetKey` / `FsVersion` are branded opaque ids ([the branded-ids Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-branded-ids.md)) — consumers must not parse `targetKey` or interpret `version`; only `displayPath` is for model/UI output. `FsWriteIntent` is the explicit GUARDED write intent (`createIfAbsent` creates a missing target and rejects an existing one with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version, else `FS_STALE_VERSION`); omitting it from `writeText` is the third, unconditional state. `FsPathInfo` is the no-follow metadata shape that can report `symlink`, unlike target-level `FsInfo`. Failures throw `FsError` (extends `HarnessError`, [the structured error taxonomy Agent Note](../../../.agents/notes/implemented/architecture/2026-06-11-structured-error-taxonomy.md)) carrying a stable `FsErrorCode` (`FS_NOT_FOUND`, `FS_NOT_DIRECTORY`, `FS_NOT_TEXT`, `FS_NOT_REGULAR_FILE`, `FS_PERMISSION_DENIED`, `FS_IO_ERROR`, `FS_STALE_VERSION`, `FS_NOT_OBSERVED`, `FS_AMBIGUOUS_EDIT`, `FS_EDIT_NOT_FOUND`, `FS_ABORTED`); the tool registry surfaces `{ name, code }` on `isError` results. See `src/types.ts` for the full contracts. -## No IO deadline - -Filesystem primitives accept an optional `AbortSignal` but arm no deadline. Local IO is only best-effort abortable: a timeout cannot force an in-progress `fsync` or `rename` to stop, so a fixed deadline would promise control the backend cannot provide. Process-backed discovery owns its separate timeout contract. - ## Model Experience Indirectly, through `dsh-tool-fs`, which renders provider text and errors as bounded, retained filesystem tool results. @@ -52,6 +61,6 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work - **Text-only by contract** — backends reject binary/non-UTF-8 content with `FS_NOT_TEXT`; binary-safe operations are a deliberate deferral of [the tool-schemas Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md). -- **Eight primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). -- **No IO deadline** — cancellation is best-effort at primitive boundaries. +- **Twelve primitives only** — no delete, rename/move, copy, or watch; `listDir` is single-level, with recursion, globbing, pagination, and search out of scope per [the directory-listing Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md). +- **No IO deadline** — the seam arms no timeout; cancellation is a best-effort optional `AbortSignal` per primitive (the deliberate [fs-family stance](../README.md)). - **Resolve-then-operate costs a remote backend two round-trips per tool call** — folding or caching resolution is left to such a backend. diff --git a/packages/fs/fs/README.zh.md b/packages/fs/fs/README.zh.md index 4772d79922..d812a94fab 100644 --- a/packages/fs/fs/README.zh.md +++ b/packages/fs/fs/README.zh.md @@ -2,56 +2,65 @@ [English](README.md) | 中文 -**文件系统提供方 seam**：抽象 `FileSystem` 服务（`ctx.fs`），定义后端提供的存储原语，包括路径解析、stat 元数据、不跟随链接的路径元数据、读取/流式读取文本、列出目录、原子写入和应用字面量编辑，但不规定实现方式。两个变更操作都**可选**接收版本防护，因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、策略插件监听的 `fs/*` 策略事件词汇。 +**文件系统提供方 seam**：抽象 `FileSystem` 服务（`ctx.fs`），定义同一个执行世界中的存储原语，包括解析路径、公开规范化进程路径与文件 URI、检查包含关系、有界或流式读取文本、检查／列出元数据、原子写入和应用字面量编辑，但不规定实现方式。两个变更操作都**可选** 接收版本防护，因此 `ctx.fs` 本身就是完整且不受约束的文本存储 seam。本包还拥有由工具分派、政策插件监听的 `fs/*` 政策事件词汇。 -本包是[文件系统家族](../README.md)中的提供方 seam 层。[工具](../tool-fs/README.md)、[策略](../fs-policy/README.md)、[本地](../fs-local/README.md)与[沙箱化](../fs-sandbox/README.md)后端分别作为消费方与实现保持独立；能力 seam 决策负责该拆分（[基础](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统 seam](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[提供方拆分](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)、[事件门禁](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)）。 +本包是四层文件系统栈中的提供方 seam 层；该拆分使每个关注点可以独立演进和替换（见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)、[文件系统能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md)、[拆分文件系统 seam Agent Note](../../../.agents/notes/implemented/simplification/2026-06-26-fsspec-style-fs-seam.md)和[文件上下文事件门禁 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-26-file-context-as-event-gate.md)）： + +| 层 | 包 | 角色 | +|---|---|---| +| 工具/执行器 | `@deepseek-ai/dsh-tool-fs` | 面向模型的 `read`/`write`/`edit` schema、读取窗口和文本渲染；通过 `ctx.fs` 读取/写入/编辑，并分派 `fs/*` 事件 | +| 政策 | `@deepseek-ai/dsh-fs-policy` | 已观察状态、编辑前读取和版本防护的写入/编辑，通过 `fs/*` 事件门禁贡献（无服务） | +| 提供方 seam | `@deepseek-ai/dsh-fs`（本包） | `ctx.fs`：执行世界路径、文本 I/O 与原子变更原语（可选版本防护）；拥有 `fs/*` 事件词汇 | +| 提供方 | `@deepseek-ai/dsh-fs-local` | 宿主文件系统实现 | + +未来的沙箱化、虚拟或远程后端只需实现该接口，政策层和工具层无需改变。 ## 服务 API（`ctx.fs`） -后端继承 `FileSystem` 并实现八个原语。 +后端继承 `FileSystem` 并实现十二个原语。 | 成员 | 语义 | |---|---| | `resolve(path, opts?)` | 把路径解析为稳定的 `FsTarget`（不透明 `targetKey`、`displayPath`）。`opts.cwd` 是相对 `path` 解析所依据的基准（调用方提供其会话工作区；绝对路径忽略该值；省略时使用后端默认值），`opts.signal` 则中止后端往返。该方法是异步的，因为远程后端可能需要 I/O。经不同路径到达的同一文件必须产生相同 `targetKey`。 | +| `processPath(target)` | 返回该提供方执行世界中的子进程可以打开的规范化绝对路径。该路径有意与不透明的 `targetKey` 分离。 | +| `fileUrl(target)` | 返回采用执行世界平台语法的规范化 `file:` URI。编码由后端而非宿主进程负责。 | +| `contains(parent, child)` | 在不公开或解析目标 key 的情况下，检查规范化身份相等或后代包含关系。两个目标都来自该提供方。 | | `stat(target, signal?)` | 返回 `FsInfo` 元数据（`version`、`type`、可选 `size`）；目标不存在时返回 `undefined`。绝不返回内容。 | -| `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时，不跟随该组件，返回 `FsPathInfo` 元数据。该方法采用路径形态，使消费方能在 `resolve` 跟随仓库自有的符号链接进入目标前拒绝它。 | +| `lstat(path, opts?, signal?)` | 当最后一个路径组件是符号链接时，不跟随该组件，返回 `FsPathInfo` 元数据。该方法采用路径形态，使消费方能在 `resolve` 跟随仓库所有的符号链接进入目标前拒绝它。 | | `readText(target, signal?)` | 把整个普通文本文件读取为一个解码后的字符串。负责普通文件检查、UTF-8 解码和二进制/NUL 拒绝（`FS_NOT_TEXT`）。 | +| `readTextBounded(target, maxBytes, signal?)` | 通过后端自有的稳定操作读取一个完整的普通 UTF-8 文件，在保留超过 `maxBytes` 前拒绝。消费方不得以先 `stat` 再 `readText` 模拟此操作，因为那会容许文件增长与替换竞态。 | | `streamText(target, signal?)` | 为大文件按解码后的分片流式读取相同文本（跨分片 UTF-8 解码仍由此处负责）。 | -| `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据（若可用则包括 `version`/文件 `size`）；绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`，非目录抛出 `FS_NOT_DIRECTORY`，权限失败抛出 `FS_PERMISSION_DENIED`，其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回；子项权限/I/O 失败会使用相同结构化代码使整个列出操作失败。 | +| `listDir(target, signal?)` | 按稳定名称顺序列出直接子项。返回条目名称、条目类型、解析后的子目标和低成本元数据（若可用则包括 `version`/文件 `size`）；绝不读取文件内容。缺失目标抛出 `FS_NOT_FOUND`，非目录抛出 `FS_NOT_DIRECTORY`，权限失败抛出 `FS_PERMISSION_DENIED`，其他后端 I/O 失败抛出 `FS_IO_ERROR`。损坏/消失的子项可以作为无元数据的 `other` 返回；子项权限/I/O 失败会使用相同结构化代码使整个列表失败。 | | `writeText(target, content, expected?, signal?)` | 原子创建/替换。`expected` 是可选的：省略 ⇒ 无条件创建或覆盖；提供 `FsWriteIntent`（`createIfAbsent`/`replaceIfVersion`）⇒ 添加防护。 | | `editText(target, edit, expected?, signal?)` | 字面量编辑。`expected` 是可选的：省略 ⇒ 无条件编辑当前内容；提供 `{ version }` ⇒ 添加防护，并在匹配之前校验。无论哪种情况，目标缺失都报告 `FS_STALE_VERSION`。应用和写入以原子方式完成，使用同一个变更临界区。 | 无论是否有版本防护，变更都在后端的每目标锁内运行，因此无条件写入/编辑仍是原子的；「无条件」只移除*版本*前置条件，不移除原子性。 -## `fs/*` 策略事件 +## `fs/*` 政策事件 -本包声明三个事件（见已生成的[事件目录](../../../docs/cordis-catalog/events.md)），使发出方（`@deepseek-ai/dsh-tool-fs`）和策略监听器（`@deepseek-ai/dsh-fs-policy`）共享词汇，而无需让发出方依赖策略插件。`fs/write-intent` 和 `fs/edit-intent` 是单槽决策 waterfall（瀑布式事件）（监听器完整决策，绝不调用 `next()`）；`fs/observed` 是发后即忘的记录事件。它们只携带 `dsh-fs` 词汇和一个不透明 `object` 参与者，不含面向模型的概念或 agent（智能体）/会话所有者结构。 +本包声明三个事件（见已生成的[事件目录](../../../docs/cordis-catalog/events.md)），使发出方（`@deepseek-ai/dsh-tool-fs`）和政策监听器（`@deepseek-ai/dsh-fs-policy`）共享词汇，而无需让发出方依赖政策插件。`fs/write-intent` 和 `fs/edit-intent` 是单槽决策 waterfall（监听器完整决策，绝不调用 `next()`）；`fs/observed` 是发后即忘的记录事件。它们只携带 `dsh-fs` 词汇和一个不透明 `object` 参与者，不含面向模型的概念或 agent（智能体）/会话所有者结构。 -## 提供方 seam，不是策略层 +## 提供方 seam，不是政策层 -`ctx.fs` 有意接近 fsspec 风格的存储原语，比字节级 `cat`/`open` 高半层，因为它会解码文本并拒绝二进制，使策略层绝不接触原始字节。它负责 UTF-8 解码、二进制拒绝、原子写入和字面量编辑临界区。它**不**负责行窗口、编号行、渲染 footer 或已观察状态。已观察状态、编辑前读取和版本防护的写入/编辑属于插件（`@deepseek-ai/dsh-fs-policy`）通过提供可选防护而添加的策略，并非提供方行为，因此沙箱化/远程后端不会继承任何面向模型的观察策略。 +`ctx.fs` 有意接近 fsspec 风格的存储原语，比字节级 `cat`/`open` 高半层，因为它会解码文本并拒绝二进制，使政策层绝不接触原始字节。它负责 UTF-8 解码、二进制拒绝、原子写入和字面量编辑临界区。它**不** 负责行窗口、编号行、渲染 footer 或已观察状态。已观察状态、编辑前读取和版本防护的写入/编辑属于插件（`@deepseek-ai/dsh-fs-policy`）通过提供可选防护而添加的政策，并非提供方行为，因此沙箱化/远程后端不会继承任何面向模型的观察政策。 -`editText` 留在该 seam 上，不由策略层通过读取加写入组合，因为版本防护、字面量匹配和原子重写必须处于同一临界区内，才能正确归因错误并实现一方胜出/一方陈旧的并发；远程后端也可以将其实现为原生比较并编辑操作。 +`editText` 留在该 seam 上，不由政策层通过读取加写入组合，因为版本防护、字面量匹配和原子重写必须处于同一临界区内，才能正确归因错误并实现一方胜出/一方陈旧的并发；远程后端也可以将其实现为原生比较并编辑操作。 ## 词汇 `FsTargetKey` / `FsVersion` 是带品牌的不透明 id（见[品牌 id Agent Note](../../../.agents/notes/implemented/architecture/2026-06-20-branded-ids.md)）；消费方不得解析 `targetKey` 或解释 `version`，只有 `displayPath` 用于模型/UI 输出。`FsWriteIntent` 是显式的防护写入意图（`createIfAbsent` 创建缺失目标，并以 `FS_NOT_OBSERVED` 拒绝现有目标；`replaceIfVersion` 只在观察版本上替换，否则为 `FS_STALE_VERSION`）；从 `writeText` 中省略该值就是第三种无条件状态。`FsPathInfo` 是可报告 `symlink` 的不跟随链接元数据形态，区别于目标级 `FsInfo`。失败会抛出 `FsError`（继承 `HarnessError`；见[结构化错误分类 Agent Note](../../../.agents/notes/implemented/architecture/2026-06-11-structured-error-taxonomy.md)），并携带稳定的 `FsErrorCode`（`FS_NOT_FOUND`、`FS_NOT_DIRECTORY`、`FS_NOT_TEXT`、`FS_NOT_REGULAR_FILE`、`FS_PERMISSION_DENIED`、`FS_IO_ERROR`、`FS_STALE_VERSION`、`FS_NOT_OBSERVED`、`FS_AMBIGUOUS_EDIT`、`FS_EDIT_NOT_FOUND`、`FS_ABORTED`）；工具注册表公开 `{ name, code }`，并将其附在 `isError` 结果上。完整契约见 `src/types.ts`。 -## 无 I/O deadline - -文件系统原语接受可选 `AbortSignal`，但不会启动 deadline。本地 I/O 只能尽力取消：超时无法强制进行中的 `fsync` 或 `rename` 停止，因此固定 deadline 会承诺后端无法提供的控制能力。基于进程的发现功能拥有独立的超时契约。 - ## 模型体验 通过 `dsh-tool-fs` 间接产生影响；该消费方把提供方文本和错误渲染为有界且保留的文件系统工具结果。 #### KV Cache 影响 -不会直接使缓存失效；上述消费方负责请求前缀的任何变化。 +不会直接使缓存失效；具名消费方负责请求前缀的任何变化。 -## 已知限制与暂缓事项 +## 已知限制与延期工作 - **契约只支持文本**：后端以 `FS_NOT_TEXT` 拒绝二进制/非 UTF-8 内容；二进制安全操作是[工具 schema Agent Note](../../../.agents/notes/implemented/feature/2026-06-17-filesystem-tool-schemas.md)有意延期的工作。 -- **只有八个原语**：没有删除、重命名/移动、复制或监视；`listDir` 只支持一层，递归、glob、分页和搜索不在范围内，见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 -- **没有 I/O deadline**：取消只能在原语边界尽力执行。 +- **只有十二个原语**：没有删除、重命名/移动、复制或监视；`listDir` 只支持一层，递归、glob、分页和搜索不在范围内，见[目录列出 Agent Note](../../../.agents/notes/archived/architecture/2026-07-03-filesystem-directory-listing-seam.md)。 +- **没有 I/O deadline**：该 seam 不启动超时；取消只是每个原语上尽力而为的可选 `AbortSignal`（见有意采用的 [fs 能力族立场](../README.md)）。 - **先解析后操作使远程后端每次工具调用需要两次往返**：折叠或缓存解析由这种后端自行决定。 diff --git a/packages/fs/fs/src/index.ts b/packages/fs/fs/src/index.ts index b43fa48c4b..8bb20ece18 100644 --- a/packages/fs/fs/src/index.ts +++ b/packages/fs/fs/src/index.ts @@ -1,8 +1,10 @@ /** - * Filesystem text-storage provider seam. Backends own stable target identity, - * text decoding, binary rejection, and atomic mutations. Read windows and - * observed-state policy stay in consumer and policy plugins; `editText` remains - * here so version check, literal match, and rewrite share one critical section. + * Filesystem provider seam for one execution world. Backends own stable target + * identity, process paths and file URIs, containment, stable bounded text + * reads, decoding, binary rejection, and atomic mutations. Read windows and + * observed-state policy stay in consumer and policy plugins; `editText` + * remains here so version check, literal match, and rewrite share one critical + * section. * @module @deepseek-ai/dsh-fs */ @@ -83,7 +85,6 @@ export abstract class FileSystem extends Service { super(ctx, 'fs') } - /** /** * The sandbox mode this backend enforces on mutations BY DEFAULT, or * `undefined` when it does not confine at all — the capability fact the tool @@ -111,6 +112,34 @@ export abstract class FileSystem extends Service { */ abstract resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise + /** + * Return the canonical absolute path a subprocess in this filesystem's + * execution world can open. The path is deliberately separate from + * {@link FsTarget.targetKey}: consumers may pass this value to another OS + * capability, but must continue treating the target key as opaque. + * @param target - the resolved target whose process path is required. + * @returns an absolute path in the backend's execution world. + */ + abstract processPath(target: FsTarget): string + + /** + * Return the canonical `file:` URI for a target in this filesystem's + * execution world. Backends own URI encoding because the host platform may + * differ from the execution platform. + * @param target - the resolved target to encode. + * @returns the target's canonical file URI. + */ + abstract fileUrl(target: FsTarget): string + + /** + * Test canonical containment without exposing or parsing backend target + * keys. Both targets must come from this provider. + * @param parent - canonical directory target. + * @param child - canonical candidate target. + * @returns true when `child` is `parent` or a descendant of it. + */ + abstract contains(parent: FsTarget, child: FsTarget): boolean + /** * Return target metadata, or `undefined` when the target does not exist. * @param target - the resolved target to stat. @@ -143,6 +172,19 @@ export abstract class FileSystem extends Service { */ abstract readText(target: FsTarget, signal?: AbortSignal): Promise + /** + * Read one regular UTF-8 text file through a backend-owned stable handle, + * rejecting before more than `maxBytes` are retained. The size check and + * bytes read are one operation: a caller must not emulate this with + * {@link stat} followed by {@link readText}, which admits growth and path + * replacement races between the two calls. + * @param target - the resolved target to read. + * @param maxBytes - positive safe-integer byte ceiling. + * @param signal - aborts the open/read operation. + * @returns the complete decoded text when it fits. + */ + abstract readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise + /** * Stream the whole regular text file as decoded text chunks (same text * semantics as {@link readText}, for large files). The backend owns diff --git a/packages/fs/fs/tests/service.spec.ts b/packages/fs/fs/tests/service.spec.ts index 19ee033cce..92f1cfc37f 100644 --- a/packages/fs/fs/tests/service.spec.ts +++ b/packages/fs/fs/tests/service.spec.ts @@ -19,13 +19,18 @@ import type { FsWriteOutcome, } from '@deepseek-ai/dsh-fs' -/** A minimal in-memory fake implementing the eight provider primitives. */ +/** A minimal in-memory fake implementing the provider primitives. */ class FakeFileSystem extends FileSystem { files = new Map() override async resolve(path: string): Promise { return { targetKey: FsTargetKey(path), displayPath: path } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + override fileUrl(target: FsTarget): string { return `file:///${encodeURIComponent(String(target.targetKey))}` } + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } override async stat(target: FsTarget): Promise { const content = this.files.get(target.targetKey) if (content === undefined) return undefined @@ -41,6 +46,11 @@ class FakeFileSystem extends FileSystem { if (content === undefined) throw new FsError(`not found: ${target.displayPath}`, 'FS_NOT_FOUND') return content } + override async readTextBounded(target: FsTarget, maxBytes: number): Promise { + const content = await this.readText(target) + if (Buffer.byteLength(content) > maxBytes) throw new FsError('too large', 'FS_IO_ERROR') + return content + } override async streamText(target: FsTarget): Promise> { const content = await this.readText(target) return (async function* () { yield content })() @@ -75,6 +85,7 @@ describe('FileSystem provider seam', () => { const ctx = new Context() await ctx.plugin(FakeFileSystem) const fs = ctx.fs as FakeFileSystem + expect(fs.sandboxMode).toBeUndefined() fs.files.set('a.txt', 'hi') const target = await fs.resolve('a.txt') expect((await fs.stat(target))?.type).toBe('file') diff --git a/packages/fs/tool-fs/tests/tools.spec.ts b/packages/fs/tool-fs/tests/tools.spec.ts index ad01237c2b..f54701246a 100644 --- a/packages/fs/tool-fs/tests/tools.spec.ts +++ b/packages/fs/tool-fs/tests/tools.spec.ts @@ -48,6 +48,11 @@ class FakeFs extends FileSystem { override async resolve(path: string): Promise { return { targetKey: FsTargetKey(`key:${path}`), displayPath: `/abs/${path}` } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } override async stat(target: FsTarget): Promise { this.throwIfArmed() const content = this.files.get(target.targetKey) @@ -62,6 +67,11 @@ class FakeFs extends FileSystem { override async readText(target: FsTarget): Promise { return this.files.get(target.targetKey) ?? '' } + override async readTextBounded(target: FsTarget, maxBytes: number): Promise { + const content = await this.readText(target) + if (Buffer.byteLength(content) > maxBytes) throw new Error('too large') + return content + } override async streamText(target: FsTarget): Promise> { const content = this.files.get(target.targetKey) ?? '' return (async function* () { yield content })() diff --git a/packages/lsp/README.i18n.yaml b/packages/lsp/README.i18n.yaml index a41b05f148..9919cdaba5 100644 --- a/packages/lsp/README.i18n.yaml +++ b/packages/lsp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/README.md -README.md: 4964d78f1096d1a4bc78fa80c6b5febaf24a5661 -README.zh.md: 93b872002cf1f53cdbb96ff42402bfeb9557575f +README.md: 7fbdf071735673fb0158f6fa66148be1c644a433 +README.zh.md: e059dbd80b7e38c0e447e54178162316dfd127c7 diff --git a/packages/lsp/README.md b/packages/lsp/README.md index 4964d78f10..7fbdf07173 100644 --- a/packages/lsp/README.md +++ b/packages/lsp/README.md @@ -6,8 +6,10 @@ The language-server capability seam: an abstract LSP interface, a generic stdio | Package | Role | ctx key | |---|---|---| -| [`lsp/`](lsp/README.md) | LSP provider seam and shared vocabulary | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | Local stdio language-server backend | registers providers on `ctx.lsp` | -| [`tool-lsp/`](tool-lsp/README.md) | Model-facing semantic-navigation tool | registers on `ctx.tools` | +| `lsp/` | Abstract LSP seam (provider registry by branded id + extension mapping, per-query selection, vocabulary, `LspError`) | `ctx.lsp` | +| `lsp-local/` | Generic multi-server stdio backend over `ctx.fs` and `ctx.subprocess` (JSON-RPC, transient-open queries) | (registers providers on `ctx.lsp`) | +| `tool-lsp/` | Model-facing `lsp` tool (four operations, one-based UTF-16 cursor coordinates) | (registers on `ctx.tools`) | -Providers register semantic capabilities; the tool owns the model-facing contract. The child READMEs document operation, protocol, and presentation details, while the [LSP capability-seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) owns the rationale. +The interface lives at `lsp/lsp/`. The seam exposes exactly four semantic operations — `goToDefinition`, `findReferences`, `goToImplementation`, `hover` — and no generic JSON-RPC escape hatch, so a provider swap does not change how the model asks for navigation and no protocol payload or unreviewed mutation reaches the model contract. Providers register **capabilities**, not tools; `tool-lsp` is the only owner of the model-facing name, schema, prompt guidance, and presentation. + +See the [LSP capability seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md) for the design rationale, including why documents open transiently per query, why the stdio host consumes the shared filesystem/subprocess execution world, and why extension ownership is exclusive within one runtime. diff --git a/packages/lsp/README.zh.md b/packages/lsp/README.zh.md index 93b872002c..e059dbd80b 100644 --- a/packages/lsp/README.zh.md +++ b/packages/lsp/README.zh.md @@ -2,12 +2,14 @@ [English](README.md) | 中文 -语言服务器能力 seam：抽象 LSP 接口、通用 stdio 提供方和面向模型的 `lsp` 工具。这些全是**产品**包。 +语言服务器能力 seam：抽象 LSP 接口、通用 stdio 提供方，以及面向模型的 `lsp` 工具。这些全是**产品** 包。 | 包 | 职责 | ctx key | |---|---|---| -| [`lsp/`](lsp/README.md) | LSP 提供方 seam 和共享词汇 | `ctx.lsp` | -| [`lsp-local/`](lsp-local/README.md) | 本地 stdio 语言服务器后端 | 在 `ctx.lsp` 上注册提供方 | -| [`tool-lsp/`](tool-lsp/README.md) | 面向模型的语义导航工具 | 注册到 `ctx.tools` | +| `lsp/` | 抽象 LSP seam（按品牌化 id + 扩展名映射组织的提供方注册表、逐查询选择、词汇、`LspError`） | `ctx.lsp` | +| `lsp-local/` | 基于 `ctx.fs` 与 `ctx.subprocess` 的通用多服务器 stdio 后端（JSON-RPC、临时打开查询） | （在 `ctx.lsp` 上注册提供方） | +| `tool-lsp/` | 面向模型的 `lsp` 工具（四种操作、从 1 开始的 UTF-16 光标坐标） | （注册到 `ctx.tools`） | -提供方注册语义能力；工具负责面向模型的契约。子 README 记录操作、协议和呈现细节，[LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)负责设计原理。 +接口位于 `lsp/lsp/`。该 seam 恰好公开四种语义操作：`goToDefinition`、`findReferences`、`goToImplementation`、`hover`，且不提供通用 JSON-RPC 逃生口；因此，替换提供方不会改变模型请求导航的方式，也不会让协议载荷或未经评审的修改进入模型契约。提供方注册的是**能力** 而非工具；`tool-lsp` 是面向模型名称、schema、提示词指引和呈现的唯一 owner。 + +设计原理见 [LSP 能力 seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)，其中也解释了文档为何在每次查询时临时打开、stdio 主机为何使用共享的文件系统／子进程执行环境，以及扩展名归属为何在同一运行时内互斥。 diff --git a/packages/lsp/lsp-local/README.i18n.yaml b/packages/lsp/lsp-local/README.i18n.yaml index b786ec1a60..1504e6f726 100644 --- a/packages/lsp/lsp-local/README.i18n.yaml +++ b/packages/lsp/lsp-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/lsp/lsp-local/README.md -README.md: 37676a82fb5d45b40ca86507259aca9509d25a43 -README.zh.md: 9e8b7f4f4395985bdbc29c1d911520b3559d7e0c +README.md: 2c5ab309f3557ad77696a881b41d164e65bd24fe +README.zh.md: d4493832964a5ac0d804602c18774bb106ddcbd9 diff --git a/packages/lsp/lsp-local/README.md b/packages/lsp/lsp-local/README.md index 37676a82fb..2c5ab309f3 100644 --- a/packages/lsp/lsp-local/README.md +++ b/packages/lsp/lsp-local/README.md @@ -2,18 +2,19 @@ English | [中文](README.zh.md) -A **generic local stdio language-server backend** for `ctx.lsp`. One plugin instance accepts a named server table and registers one isolated provider per entry. This is a generic host, not a language-server catalog or installer — deployments configure commands and mappings explicitly; presets belong in `cordis.yml` overlays. +A **generic stdio language-server backend** for `ctx.lsp`. One plugin instance accepts a named server table and registers one isolated provider per entry. It reads through `ctx.fs` and launches through `ctx.subprocess`, so the server and source always inhabit the mounted execution world. This is a generic host, not a language-server catalog or installer — deployments configure commands and mappings explicitly; presets belong in `cordis.yml` overlays. Namespace plugin (`name` / `inject` / `Config` / `apply`, no default export). ## What it does - Resolves every server-local setting before registration; an invalid mapping or registration conflict rolls back earlier entries, so a failed load leaves no provider routes. -- Lazily single-flights one server process per `(server id, canonical workspace realpath)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. -- Uses a compatibility-first **transient-open** sequence per query: canonicalize and read the source with Node APIs, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. +- Lazily single-flights one server process per `(server id, canonical workspace target)`. A live server error is not replayed; if the selected pooled transport fails before or during a read-only query, the provider awaits its disposal and retries that query once on a fresh process. +- Uses a compatibility-first **transient-open** sequence per query: resolve and boundedly read the source through `ctx.fs`, `textDocument/didOpen` (version 1, full text), the requested request, then `textDocument/didClose` in `finally`. A failed or canceled `didOpen` write terminates the instance before the pool can reuse it. Documents close after each call, so the first version needs no `didChange`, content cache, or document LRU. - Serializes each source-read/open/query/close lifecycle through one abortable per-workspace queue so queued calls read current source only when their turn starts; distinct workspaces run in parallel. - After protocol shutdown fails, terminates the server's descendant tree through the subprocess seam (POSIX process-group signaling; Windows `taskkill /T /F`). Tree-kill delivery is contained like every group signal — it races server exit — and quiescence is confirmed by the handle's tree-liveness wait rather than by the kill's own outcome. -- Reads sources through Node filesystem APIs in the subprocess's host namespace — NOT `ctx.fs`, and emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. +- Resolves the server executable, cwd, process, and protocol streams through `ctx.subprocess`; `initialize.processId` is `null` because another machine or PID namespace must not monitor the harness process. +- Uses `ctx.fs` canonical containment, file URIs, and stable bounded reads, but emits no `fs/observed`: only the LSP result is model-visible, so a query does not satisfy read-before-write policy. ## Configuration @@ -23,7 +24,7 @@ The `servers` record key is the stable provider id reserved on `ctx.lsp`; each v |---|---|---| | `command` | (required) | Executable to spawn — absolute, or resolved on the child PATH at load. Launch uses no shell. | | `args` | `[]` | Arguments passed to the executable. | -| `env` | `{}` | Extra env merged on top of the credential-scrubbed ambient env (vars matching `KEY`/`PASSWORD`/`SECRET`/`TOKEN` are not forwarded); an explicit `DSH_*` entry merges after the seam's scrub of ambient ones. | +| `env` | `{}` | Extra env merged on top of the credential-scrubbed ambient env (vars matching `KEY`/`SECRET`/`TOKEN` are not forwarded); an explicit `DSH_*` entry merges after the seam's scrub of ambient ones. | | `extensionToLanguage` | (required) | Lowercase leading-dot extension → LSP language id (e.g. `{ '.ts': 'typescript' }`). | | `initializationOptions` | `null` | Static `initialize` options forwarded to the server. | | `configuration` | `null` | Static answer to every `workspace/configuration` item. | @@ -41,7 +42,7 @@ Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { ## Security boundary -The provider trusts its configured server and claims no sandbox confinement. It canonicalizes and reads source through Node APIs, rejecting a source that is missing, non-regular, non-UTF-8, oversized, or whose canonical path resolves outside the canonical workspace (symlink aliases share one instance). Result locations may be external, but an external path cannot become a query source. The first implementation therefore requires trusted host-local deployment; restricted, remote, or virtual workspaces require another provider. +The provider trusts its configured server and claims no sandbox confinement. It delegates canonical identity, containment, no-follow/stable bounded reads, UTF-8 validation, and file-URI encoding to `ctx.fs`; it rejects missing, non-regular, non-UTF-8, oversized, or canonically out-of-workspace query sources before server startup. Result locations may be external, but an external path cannot become a query source. A deployment must mount filesystem and subprocess providers for the same execution world; split-world composition is invalid. ## Model Experience @@ -53,6 +54,7 @@ No direct invalidation; `dsh-tool-lsp` owns request-prefix changes. ## Known Limitations and Deferred Work -- **Trusted host-local only** — no sandbox confinement, no private cache/temp write contract; supporting untrusted binaries or restricted/remote/virtual workspaces requires a later process/filesystem contract and a different provider ([seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)). Containment resolves `realpath`, then opens the source through one handle with `O_NOFOLLOW | O_NONBLOCK` (final-component symlink guard plus nonblocking rejection of FIFOs) and a bounded read; a concurrent mutator that swaps an *ancestor* directory for a symlink between the resolve and the open is an accepted residual TOCTOU under this trusted-deployment model, not closed with non-portable `openat` segment walks. -- **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; compatibility with one TypeScript server does not imply cross-language support. +- **No confinement policy** — this package trusts the configured server and does not sandbox its process; a restricted deployment must supply appropriate process/filesystem providers or a same-world sandbox wrapper. +- **Execution-world URI rendering** — the stdio host produces provider-owned `file:` URIs. The current model tool renders them with the harness host's path library, so a Windows harness paired with a POSIX remote execution world may show remote locations as URIs or non-native paths; protocol queries remain correct. +- **Transient-open compatibility floor** — servers whose synchronization omits open/close (or advertise `None`) are unsupported even if closed-document queries would work; the pinned TypeScript e2e establishes one compatibility floor, not a cross-language claim. - **Per-server/workspace serialization latency** — parallel agents sharing one server and workspace queue behind one process; long-lived workspace processes consume memory until disposal. diff --git a/packages/lsp/lsp-local/README.zh.md b/packages/lsp/lsp-local/README.zh.md index 9e8b7f4f43..d449383296 100644 --- a/packages/lsp/lsp-local/README.zh.md +++ b/packages/lsp/lsp-local/README.zh.md @@ -2,18 +2,19 @@ [English](README.md) | 中文 -`ctx.lsp` 的**通用本地 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表，并逐配置项注册一个隔离的提供方。这是通用主机，而不是语言服务器目录或安装器：部署需要显式配置命令与映射，预设应放在 `cordis.yml` overlay 中。 +`ctx.lsp` 的**通用 stdio 语言服务器后端**。一个插件实例接受一张命名服务器表，并逐配置项注册一个隔离的提供方。它通过 `ctx.fs` 读取，并通过 `ctx.subprocess` 启动，因此服务器与源文件始终位于所挂载的同一执行环境。这是通用主机，而不是语言服务器目录或安装器：部署需要显式配置命令与映射，preset 应放在 `cordis.yml` overlay 中。 Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出）。 ## 功能 - 在注册前解析每项服务器局部设置；无效映射或注册冲突会回滚较早配置项，因此加载失败不会留下提供方路由。 -- 每个 `(server id, canonical workspace realpath)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放；如果选中的池化传输在只读查询之前或期间失败，提供方会等待其 dispose（资源释放）完成，并在新进程上重试该查询一次。 -- 每次查询都使用兼容性优先的**临时打开**序列：通过 Node API 规范化并读取源文件、`textDocument/didOpen`（版本 1、完整文本）、所请求操作，然后执行 `textDocument/didClose`，该操作位于 `finally` 中。写入 `didOpen` 失败或取消时，会在池复用该实例前将其终止。文档在每次调用后关闭，因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 +- 每个 `(server id, canonical workspace target)` 惰性 single-flight 一个服务器进程。存活服务器错误不会回放；如果选中的池化传输在只读查询之前或期间失败，提供方会等待其释放，并在新进程上重试该查询一次。 +- 每次查询都使用兼容性优先的**临时打开** 序列：通过 `ctx.fs` 解析源文件并进行有界读取、`textDocument/didOpen`（版本 1、完整文本）、所请求操作，然后执行 `textDocument/didClose`，该操作位于 `finally` 中。写入 `didOpen` 失败或取消时，会先终止实例再允许池复用。文档在每次调用后关闭，因此第一版不需要 `didChange`、内容 cache 或文档 LRU。 - 通过一条逐 Workspace、可中止的队列，串行执行每个源读取／打开／查询／关闭生命周期，因此排队调用只会在轮到自身时读取当前源；不同 Workspace 并行运行。 -- 协议 shutdown 失败后，经由子进程 seam 终止服务器后代树（POSIX 进程组信号；Windows `taskkill /T /F`）。树终止的投递结果与所有进程组信号一样被就地吸收，不向外抛出（投递与服务器退出存在竞态）；服务器是否完全停稳，由句柄的进程树存活等待确认，而非由这次终止自身的结果确认。 -- 通过子进程 host namespace 中的 Node 文件系统 API 读取源文件，绝不使用 `ctx.fs`，也不发出 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足先读后写策略。 +- 协议 shutdown 失败后，经由进程管理器 seam 终止服务器后代树（POSIX 进程组信号；Windows `taskkill /T /F`）。树终止的投递结果与所有进程组信号一样被就地吸收，不向外抛出（投递与服务器退出存在竞态）；服务器是否完全停稳，由句柄的进程树存活等待确认，而非由这次终止自身的结果确认。 +- 通过 `ctx.subprocess` 解析服务器可执行文件、cwd、进程与协议流；`initialize.processId` 为 `null`，因为另一台机器或 PID 命名空间不得监控 harness 进程。 +- 使用 `ctx.fs` 提供的规范 containment、文件 URI 与稳定有界读取，但不发出 `fs/observed`：只有 LSP 结果对模型可见，因此查询不满足先读后写策略。 ## 配置 @@ -23,13 +24,13 @@ Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出） |---|---|---| | `command` | （必填） | 要 spawn 的可执行文件：绝对路径，或在加载时从子进程 PATH 解析。不使用 shell 启动。 | | `args` | `[]` | 传给可执行文件的参数。 | -| `env` | `{}` | 合并到已清理 credential 的环境之上的额外 env（匹配 `KEY`／`PASSWORD`／`SECRET`／`TOKEN` 的变量不会转发）；显式 `DSH_*` 条目在 seam 清除环境中同名值之后合并。 | +| `env` | `{}` | 合并到已清理 credential 的环境之上的额外 env（匹配 `KEY`／`SECRET`／`TOKEN` 的变量不会转发）；显式 `DSH_*` 条目在 seam 清除环境中同名值之后合并。 | | `extensionToLanguage` | （必填） | 小写、以点开头的扩展名 → LSP language id（例如 `{ '.ts': 'typescript' }`）。 | | `initializationOptions` | `null` | 转发给服务器的静态 `initialize` 选项。 | | `configuration` | `null` | 每个 `workspace/configuration` 配置项的静态答案。 | | `maxMessageBytes` | `16000000` | 从服务器接受的单条 framed 消息最大大小。 | | `maxStderrBytes` | `1000000` | 为诊断保留的 stderr 尾部最大大小。 | -| `maxDocumentBytes` | `4000000` | 该主机可打开的源文件大小上限。 | +| `maxDocumentBytes` | `4000000` | 该主机可打开的最大源文件。 | | `shutdownTimeoutMs` | `5000` | 升级前用于优雅 `shutdown`／`exit` 的预算。 | | `killGraceMs` | `2000` | 请求取消及 SIGTERM→SIGKILL 升级的宽限期。 | @@ -37,11 +38,11 @@ Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出） ## 协议行为 -初始化会声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`，以及定义与实现使用的 `linkSupport: true`，且不进行动态注册。服务器返回的能力具有最终决定权：不受支持的操作，或缺少临时打开／关闭的同步方式，会使查询失败。服务器省略 `positionEncoding` 时默认为 `utf-16`；其他值都属于协议错误。客户端通过静态配置回答 `workspace/configuration`，接受生命周期记账请求，并拒绝 `workspace/applyEdit`：它绝不应用编辑或运行命令。导航直接映射 `Location`，并从 `LocationLink` 的 `targetUri` + `targetSelectionRange` 映射；hover 规范化会取得有效的 `MarkupContent.value`，保留 string `MarkedString`，把带 language tag 的值渲染为围栏代码，并用一个空行连接数组。缺失结果、格式错误的范围或位置，以及格式错误的 hover 编码，都会以结构化 `LSP_MALFORMED_RESPONSE` 错误的形式失败。 +初始化会声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`，以及定义与实现使用的 `linkSupport: true`，且不进行动态注册。服务器返回的能力具有最终决定权：不受支持的操作，或缺少临时打开／关闭的同步方式，会使查询失败。服务器省略 `positionEncoding` 时默认为 `utf-16`；其他值都属于协议错误。客户端通过静态配置回答 `workspace/configuration`，接受生命周期记账请求，并拒绝 `workspace/applyEdit`：它绝不应用编辑或运行命令。导航直接映射 `Location`，并从 `LocationLink` 的 `targetUri` + `targetSelectionRange` 映射；hover 规范化会取得有效的 `MarkupContent.value`，保留 string `MarkedString`，把带 language tag 的值渲染为围栏代码，并用一个空行连接数组。缺失结果、格式错误的范围或位置，以及格式错误的 hover 编码，都会作为结构化 `LSP_MALFORMED_RESPONSE` 错误失败。 ## 安全边界 -提供方信任其配置的服务器，不提供任何沙箱隔离。它通过 Node API 规范化并读取源文件，拒绝缺失、非普通文件、非 UTF-8、过大，或规范路径位于规范 Workspace 外部的源文件（符号链接别名共享一个实例）。结果位置可以在外部，但外部路径不能成为查询源。因此，第一版要求可信的主机本地部署；受限、远程或虚拟 Workspace 需要另一个提供方。 +提供方信任其配置的服务器，不声明任何沙箱限制。它把规范身份、containment、不跟随符号链接的稳定有界读取、UTF-8 校验与文件 URI 编码委托给 `ctx.fs`；服务器启动前，系统会拒绝缺失、非普通文件、非 UTF-8、过大或规范路径位于工作区外的查询源。结果位置可以在外部，但外部路径不能成为查询源。部署必须为同一执行环境挂载文件系统与子进程提供方；分裂执行环境的组合无效。 ## 模型体验 @@ -53,6 +54,7 @@ Namespace 插件（`name`／`inject`／`Config`／`apply`，无默认导出） ## 已知限制与暂缓事项 -- **仅限可信主机本地环境**：没有沙箱隔离，也没有私有 cache／temp 写入契约；支持不受信任 binary 或受限／远程／虚拟 Workspace，需要后续的进程／文件系统契约及不同提供方（见 [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-15-lsp-capability-seam.md)）。限制逻辑先解析 `realpath`，再通过一个带 `O_NOFOLLOW | O_NONBLOCK` 的 handle 打开源文件（最终组件符号链接防护，并以非阻塞方式拒绝 FIFO），同时进行有界读取；并发修改方如果在解析与打开之间把*祖先*目录替换为符号链接，会造成残余 TOCTOU。在该可信部署模型下接受此风险，不使用不可移植的 `openat` 逐 segment 遍历来封闭。 -- **临时打开兼容性下限**：同步能力省略打开／关闭（或声明 `None`）的服务器不受支持，即使关闭文档查询能够工作；与一个 TypeScript 服务器兼容，并不表示支持其他语言。 -- **逐服务器／Workspace 串行化延迟**：共享同一个服务器与 Workspace 的并行 agent（智能体）会在一个进程后排队；长生命周期 Workspace 进程会占用内存直到 dispose。 +- **不提供隔离策略**：这个包（package）信任配置的服务器，不会对其进程执行沙箱化；受限部署必须提供适当的进程／文件系统提供方，或包装同一执行环境的沙箱。 +- **执行环境 URI 渲染**：stdio 主机生成归提供方所有的 `file:` URI。当前面向模型的工具使用 harness 宿主的路径库渲染这些 URI，因此 Windows harness 与 POSIX 远程执行环境配对时，可能把远程位置显示为 URI 或非本机路径；协议查询仍然正确。 +- **临时打开兼容性下限**：同步能力省略打开／关闭（或声明 `None`）的服务器不受支持，即使关闭文档查询能够工作；固定的 TypeScript e2e 只建立一项兼容性下限，不代表跨语言承诺。 +- **逐服务器／Workspace 串行化延迟**：共享同一个服务器与 Workspace 的并行 agent 会在一个进程后排队；长生命周期 Workspace 进程会占用内存直到释放。 diff --git a/packages/lsp/lsp-local/package.json b/packages/lsp/lsp-local/package.json index b68745d617..a5411bab55 100644 --- a/packages/lsp/lsp-local/package.json +++ b/packages/lsp/lsp-local/package.json @@ -26,6 +26,7 @@ "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-brand": "^0.0.1", + "@deepseek-ai/dsh-fs": "^0.0.1", "@deepseek-ai/dsh-invariants": "^0.0.1", "@deepseek-ai/dsh-llm": "^0.0.1", "@deepseek-ai/dsh-lsp": "^0.0.1", @@ -38,6 +39,8 @@ }, "devDependencies": { "@deepseek-ai/dsh-brand": "workspace:^", + "@deepseek-ai/dsh-fs": "workspace:^", + "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-lsp": "workspace:^", diff --git a/packages/lsp/lsp-local/src/connection.ts b/packages/lsp/lsp-local/src/connection.ts index e19718eed6..4be75cbd33 100644 --- a/packages/lsp/lsp-local/src/connection.ts +++ b/packages/lsp/lsp-local/src/connection.ts @@ -22,7 +22,7 @@ export interface ConnectionSpec { readonly args: readonly string[] /** The child's working directory (the canonical workspace). */ readonly cwd: string - /** The child's environment (credential-scrubbed, with overrides applied). */ + /** Explicit child environment overrides; the subprocess provider owns its ambient scrub. */ readonly env: Record /** Largest single framed message accepted from the server. */ readonly maxMessageBytes: number @@ -98,9 +98,8 @@ export class LspConnection { stderr: { maxBytes: spec.maxStderrBytes }, }, graceMs: spec.killGraceMs, - // spec.env mixes the scrubbed base with explicit config entries; the - // seam merges the whole map after its own ambient scrub, so a - // configured DSH_* fact reaches the child. + // The seam merges explicit config entries after its ambient scrub, so a + // configured credential or DSH_* fact reaches the child deliberately. env: spec.env, }) /* v8 ignore start -- 'pipe' dispositions expose both streams by the seam contract; defensive. */ diff --git a/packages/lsp/lsp-local/src/host.ts b/packages/lsp/lsp-local/src/host.ts index 11996908ac..3a2b9fbd7b 100644 --- a/packages/lsp/lsp-local/src/host.ts +++ b/packages/lsp/lsp-local/src/host.ts @@ -1,154 +1,106 @@ -/** - * Host-filesystem source access for the local provider, using Node APIs directly in the - * subprocess's namespace (never `ctx.fs`): only the LSP result is model-visible, so a query does not - * satisfy read-before-write policy and emits no `fs/observed`. Canonicalization derives target - * identity from `realpath`, so symlink aliases share a workspace; a source is rejected before server - * startup when it is missing, non-regular, non-UTF-8, oversized, or canonically outside the - * workspace. External result locations are allowed, but an external path can never become a query - * source. - * @module @deepseek-ai/dsh-lsp-local/host - */ +/** Filesystem-seam source access for the generic stdio LSP provider. */ -import { constants } from 'node:fs' -import { open, realpath, stat } from 'node:fs/promises' -import type { FileHandle } from 'node:fs/promises' -import { isAbsolute, resolve as resolvePath, sep } from 'node:path' +import type { FileSystem, FsTarget } from '@deepseek-ai/dsh-fs' import { throwIfAborted } from './abort.ts' -/** A validated source: its canonical absolute path and current UTF-8 text. */ -export interface HostSource { - /** The canonical (realpath-resolved) absolute path, inside the canonical workspace. */ +/** A canonical workspace in the filesystem/subprocess execution world. */ +export interface HostWorkspace { + /** Stable filesystem identity used for provider pooling. */ + readonly target: FsTarget + /** Canonical absolute path accepted as a subprocess cwd. */ readonly canonicalPath: string - /** The file's current text, read as UTF-8. */ + /** Canonical file URI sent during LSP initialization. */ + readonly fileUrl: string +} + +/** A validated source and the exact URI sent to the language server. */ +export interface HostSource { + /** Canonical file URI in the execution world's platform syntax. */ + readonly fileUrl: string + /** Current complete UTF-8 text. */ readonly text: string } /** - * Canonicalize a workspace root: it must exist and be a directory. The returned realpath supplies - * process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity, so symlinked roots - * collapse to one instance. - * @param workspaceRoot - the caller's workspace root (absolute). - * @param signal - optional cancellation observed around each filesystem operation. - * @returns the canonical directory path. - * @throws Error when the path is missing or not a directory. + * Resolve and validate one workspace through `ctx.fs`. + * @param fs - filesystem provider sharing the language server's execution world. + * @param workspaceRoot - caller-supplied workspace path. + * @param signal - optional cancellation around provider operations. + * @returns stable identity plus process path and file URI. */ -export async function canonicalizeWorkspace(workspaceRoot: string, signal?: AbortSignal): Promise { +export async function canonicalizeWorkspace( + fs: FileSystem, + workspaceRoot: string, + signal?: AbortSignal, +): Promise { throwIfAborted(signal) - let canonical: string + let target: FsTarget try { - canonical = await realpath(workspaceRoot) - } catch (error) { - throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`) + target = await fs.resolve(workspaceRoot, signal === undefined ? {} : { signal }) + } catch (error: unknown) { + throwIfAborted(signal) + throw new Error(`workspace root "${workspaceRoot}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - const info = await stat(canonical) + const info = await fs.stat(target, signal) throwIfAborted(signal) - if (!info.isDirectory()) { + if (info?.type !== 'directory') { throw new Error(`workspace root "${workspaceRoot}" is not a directory`) } - return canonical + return { + target, + canonicalPath: fs.processPath(target), + fileUrl: fs.fileUrl(target), + } } /** - * Resolve, canonicalize, validate, and read a query source in one pass. A relative `filePath` - * resolves against `canonicalWorkspace`; an absolute one is taken directly. The canonical target - * must be a regular UTF-8 file no larger than `maxDocumentBytes`, and must lie inside the canonical - * workspace. - * @param filePath - the model-supplied source path (relative or absolute). - * @param canonicalWorkspace - the already-canonicalized workspace root. - * @param maxDocumentBytes - the largest source this host will open. - * @param signal - optional cancellation observed throughout resolution, validation, and reading. - * @returns the canonical path and current UTF-8 text. - * @throws Error when the source is missing, non-regular, oversized, non-UTF-8, or out of workspace. + * Resolve, contain, and atomically read one bounded query source through + * `ctx.fs`. The provider's bounded read owns stable-handle and no-follow + * mechanics; this layer owns only LSP-facing validation and messages. + * @param fs - filesystem provider sharing the server's execution world. + * @param filePath - absolute source path or path relative to `workspace`. + * @param workspace - already-canonical workspace. + * @param maxDocumentBytes - largest complete source accepted by this host. + * @param signal - optional cancellation. + * @returns canonical file URI and current text. */ export async function readHostSource( + fs: FileSystem, filePath: string, - canonicalWorkspace: string, + workspace: HostWorkspace, maxDocumentBytes: number, signal?: AbortSignal, ): Promise { throwIfAborted(signal) - const requested = isAbsolute(filePath) ? filePath : resolvePath(canonicalWorkspace, filePath) - let canonicalPath: string + let target: FsTarget try { - canonicalPath = await realpath(requested) - } catch (error) { - throw new Error(`source "${filePath}" cannot be resolved: ${messageOf(error)}`) + target = await fs.resolve(filePath, { + cwd: workspace.canonicalPath, + ...signal === undefined ? {} : { signal }, + }) + } catch (error: unknown) { + throwIfAborted(signal) + throw new Error(`source "${filePath}" cannot be resolved: ${messageOf(error)}`, { cause: error }) } throwIfAborted(signal) - if (!isInside(canonicalWorkspace, canonicalPath)) { + if (!fs.contains(workspace.target, target)) { throw new Error(`source "${filePath}" resolves outside the workspace`) } - // Open ONE handle after containment, then stat and read through it: a concurrent replace between - // realpath and read cannot swap the target, so the regular-file and size checks bind the bytes we - // actually read (no path-based TOCTOU). O_NOFOLLOW rejects the final component being swapped for a - // symlink between realpath and open (which would otherwise escape the workspace). - // O_NONBLOCK prevents a FIFO with no writer from hanging before fstat can reject it as nonregular. - const handle = await open(canonicalPath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK) + let text: string try { + text = await fs.readTextBounded(target, maxDocumentBytes, signal) + } catch (error: unknown) { throwIfAborted(signal) - const info = await handle.stat() - throwIfAborted(signal) - if (!info.isFile()) { - throw new Error(`source "${filePath}" is not a regular file`) - } - if (info.size > maxDocumentBytes) { - throw new Error(`source "${filePath}" is ${info.size} bytes, over the ${maxDocumentBytes}-byte limit`) - } - // Bound the read to the cap even if the file grew after stat: read one extra byte and reject on - // overflow, so a concurrent grow cannot defeat the memory bound. - const buffer = await readCapped(handle, maxDocumentBytes, filePath, signal) - const text = decodeUtf8Strict(buffer, filePath) - throwIfAborted(signal) - return { canonicalPath, text } - } finally { - await handle.close() + throw new Error(`source "${filePath}" could not be opened safely: ${messageOf(error)}`, { cause: error }) + } + throwIfAborted(signal) + return { + fileUrl: fs.fileUrl(target), + text, } } -/** Read at most `maxBytes` from the handle, rejecting when the source overflows the cap. */ -async function readCapped( - handle: FileHandle, - maxBytes: number, - filePath: string, - signal?: AbortSignal, -): Promise { - const limit = maxBytes + 1 - const chunk = Buffer.allocUnsafe(limit) - let total = 0 - for (;;) { - throwIfAborted(signal) - const { bytesRead } = await handle.read(chunk, total, limit - total, total) - throwIfAborted(signal) - if (bytesRead === 0) break - total += bytesRead - /* v8 ignore next 3 -- overflow requires the file to grow past the cap between stat and read (a concurrent mutation); defensive. */ - if (total > maxBytes) { - throw new Error(`source "${filePath}" grew past the ${maxBytes}-byte limit while reading`) - } - } - return chunk.subarray(0, total) -} - -/** Whether `child` is the workspace itself or a descendant of it (both already canonical). */ -function isInside(workspace: string, child: string): boolean { - if (child === workspace) return true - /* v8 ignore next -- a canonical non-root workspace never ends with a separator; the guard covers the filesystem root. */ - const base = workspace.endsWith(sep) ? workspace : workspace + sep - return child.startsWith(base) -} - -/** Decode strictly as UTF-8: a fatal decoder rejects only malformed bytes, keeping a legitimate U+FFFD. */ -function decodeUtf8Strict(buffer: Buffer, filePath: string): string { - try { - return new TextDecoder('utf-8', { fatal: true }).decode(buffer) - } catch { - throw new Error(`source "${filePath}" is not valid UTF-8 text`) - } -} - -/** Extract a message from an unknown thrown value without leaking `any`. */ function messageOf(error: unknown): string { - /* v8 ignore next -- Node fs rejections are always Error instances; the String() fallback is defensive. */ return error instanceof Error ? error.message : String(error) } diff --git a/packages/lsp/lsp-local/src/index.ts b/packages/lsp/lsp-local/src/index.ts index d926ae9428..d12e8986fc 100644 --- a/packages/lsp/lsp-local/src/index.ts +++ b/packages/lsp/lsp-local/src/index.ts @@ -1,18 +1,16 @@ /** * Generic stdio language-server backend for `ctx.lsp`. One plugin instance configures a named table * of server commands and registers one isolated provider for each entry. Every provider lazily - * single-flights one server process per canonical workspace realpath, serves transient-open queries + * single-flights one server process per canonical workspace target, serves transient-open queries * through it, and replaces a selected transport that fails before or during the next read-only - * query. Providers read sources through Node APIs in the host namespace (not `ctx.fs`) - * and trust their configured servers — no sandbox confinement. + * query. Providers read sources through `ctx.fs` and launch servers through + * `ctx.subprocess`, so both local and remote implementations share one host. * * Namespace plugin (named exports, no default export). Lifecycle is effect-scoped: disposal * unregisters from `ctx.lsp` and tears down every live server. * @module @deepseek-ai/dsh-lsp-local */ -import { accessSync, constants, statSync } from 'node:fs' -import { delimiter, isAbsolute, join } from 'node:path' import type { Context } from 'cordis' import z from 'schemastery' import { LspError, LspProviderId } from '@deepseek-ai/dsh-lsp' @@ -24,9 +22,9 @@ import type { import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { abortable, abortError } from './abort.ts' import { canonicalizeWorkspace, readHostSource } from './host.ts' +import type { HostWorkspace } from './host.ts' import { LspInstance } from './instance.ts' import type { ConnectionSpawner } from './connection.ts' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { InstanceSpec } from './instance.ts' export { canonicalizeWorkspace, readHostSource } from './host.ts' @@ -46,10 +44,7 @@ export { LspConnection } from './connection.ts' export const name = 'lsp-local' /** Services required by this plugin. */ -export const inject = ['lsp', 'subprocess'] - -/** Credential-shaped ambient env vars are NOT forwarded to the child by default. */ - +export const inject = ['fs', 'lsp', 'subprocess'] const DEFAULT_MAX_MESSAGE_BYTES = 16_000_000 const DEFAULT_MAX_STDERR_BYTES = 1_000_000 @@ -91,6 +86,7 @@ export interface Config { /** One server config after schemastery fills every default. */ type ResolvedServerConfig = Required +type WorkspaceKey = HostWorkspace['target']['targetKey'] const LspLocalServerConfig: z = z.object({ command: z.string().required(), @@ -114,23 +110,28 @@ export const Config: z = z.object({ * Register the configured stdio LSP providers. Resolves every executable at load (after credential * scrubbing) before publishing any provider; each process launches lazily on its first matching * query. - * @param ctx - the plugin context (must inject `lsp`). + * @param ctx - the plugin context carrying `fs`, `lsp`, and `subprocess`. * @param config - the resolved plugin configuration (schemastery has filled every default). */ -export function apply(ctx: Context, config: Config): void { +export async function apply(ctx: Context, config: Config): Promise { const entries = Object.entries(config.servers) if (entries.length === 0) throw new Error('lsp-local: servers must contain at least one server') // Resolve every server-local setting before registration so a bad later command or bound cannot // publish an earlier provider. Registry-level mapping conflicts are rolled back below. - const providers = entries.map(([providerId, rawConfig]) => { + const providers = await Promise.all(entries.map(async ([providerId, rawConfig]) => { if (providerId.trim() === '') throw new Error('lsp-local: server ids must be non-empty strings') const resolved = rawConfig as ResolvedServerConfig validateServerConfig(providerId, resolved) - const childEnv = buildChildEnv(resolved.env) - const executable = resolveExecutable(resolved.command, childEnv) - return new LocalLspProvider(providerId, resolved, childEnv, executable, spec => ctx.subprocess.spawn(spec)) - }) + const executable = await ctx.subprocess.resolveExecutable(resolved.command, resolved.env) + return new LocalLspProvider( + providerId, + ctx.fs, + resolved, + executable, + spec => ctx.subprocess.spawn(spec), + ) + })) ctx.effect(() => { const disposers: Array<() => void> = [] @@ -180,16 +181,16 @@ function assertPositiveInteger(providerId: string, name: string, value: number): class LocalLspProvider implements LspProvider { readonly id: LspProviderId readonly extensionToLanguage: Readonly> - /** One live instance per canonical workspace realpath. */ - private readonly instances = new Map() + /** One live instance per stable canonical workspace identity. */ + private readonly instances = new Map() /** One complete source-read→open→query→close serialization tail per canonical workspace. */ - private readonly queues = new Map>() + private readonly queues = new Map>() private disposed = false constructor( providerId: string, + private readonly fs: Context['fs'], private readonly config: ResolvedServerConfig, - private readonly childEnv: Record, private readonly executable: string, private readonly spawner: ConnectionSpawner, ) { @@ -211,19 +212,20 @@ class LocalLspProvider implements LspProvider { } async query(request: LspProviderQuery, signal?: AbortSignal): Promise { - // Honor an already-aborted signal before host I/O so a canceled request never starts a server. + // Honor an already-aborted signal before provider I/O so a canceled request never starts a server. this.assertActive(signal) - const workspace = await canonicalizeWorkspace(request.workspaceRoot, signal) + const workspace = await canonicalizeWorkspace(this.fs, request.workspaceRoot, signal) this.assertActive(signal) - return this.enqueue(workspace, signal, async () => { + const workspaceKey = workspace.target.targetKey + return this.enqueue(workspaceKey, signal, async () => { this.assertActive(signal) // Read inside the workspace queue but before spawning: a queued query sees current bytes when // its turn starts, while an invalid source still cannot leave an idle process pooled. - const source = await readHostSource(request.filePath, workspace, this.config.maxDocumentBytes, signal) + const source = await readHostSource(this.fs, request.filePath, workspace, this.config.maxDocumentBytes, signal) // Disposal may have snapshotted the instance map while host I/O was pending. Re-check before a // synchronous get-or-create so every spawned process remains owned by teardown. this.assertActive(signal) - let instance = this.instanceFor(workspace) + let instance = this.instanceFor(workspaceKey, workspace) try { return await instance.query(request, source, signal) } catch (error) { @@ -231,22 +233,22 @@ class LocalLspProvider implements LspProvider { // read-only, so replace that transport once and retry transparently. if (!instance.isTransportFailure(error)) throw error await instance.dispose() - this.evictIfCurrent(workspace, instance) + this.evictIfCurrent(workspaceKey, instance) this.assertActive(signal) - instance = this.instanceFor(workspace) + instance = this.instanceFor(workspaceKey, workspace) return await instance.query(request, source, signal) } finally { // Reach quiescence before dropping a dead slot; a replacement must survive this ownership check. if (instance.dead) { await instance.dispose() - this.evictIfCurrent(workspace, instance) + this.evictIfCurrent(workspaceKey, instance) } } }) } /** Serialize one complete query lifecycle for a canonical workspace. */ - private enqueue(workspace: string, signal: AbortSignal | undefined, run: () => Promise): Promise { + private enqueue(workspace: WorkspaceKey, signal: AbortSignal | undefined, run: () => Promise): Promise { const previous = this.queues.get(workspace) ?? Promise.resolve() const result = abortable(previous, signal).then(run) // The tail follows the actual prior work even when this caller aborts its wait. It never rejects, @@ -260,27 +262,28 @@ class LocalLspProvider implements LspProvider { } /** Return or synchronously publish the one instance for a canonical workspace. */ - private instanceFor(workspace: string): LspInstance { + private instanceFor(workspaceKey: WorkspaceKey, workspace: HostWorkspace): LspInstance { this.assertActive() - const existing = this.instances.get(workspace) + const existing = this.instances.get(workspaceKey) if (existing !== undefined) return existing const created = this.createInstance(workspace) - this.instances.set(workspace, created) + this.instances.set(workspaceKey, created) return created } /** Drop the slot iff it still contains this instance. */ - private evictIfCurrent(workspace: string, instance: LspInstance): void { + private evictIfCurrent(workspace: WorkspaceKey, instance: LspInstance): void { /* v8 ignore next -- mismatch requires another query to replace the slot before this finally runs. */ if (this.instances.get(workspace) === instance) this.instances.delete(workspace) } - private createInstance(workspace: string): LspInstance { + private createInstance(workspace: HostWorkspace): LspInstance { const spec: InstanceSpec = { command: this.executable, args: this.config.args, - cwd: workspace, - env: this.childEnv, + cwd: workspace.canonicalPath, + workspaceUri: workspace.fileUrl, + env: this.config.env, configuration: this.config.configuration, initializationOptions: this.config.initializationOptions, maxMessageBytes: this.config.maxMessageBytes, @@ -304,41 +307,3 @@ class LocalLspProvider implements LspProvider { this.queues.clear() } } - -/** The seam's scrubbed parent env (credential-shaped and DSH_* names dropped), plus the config's explicit env. */ -function buildChildEnv(extra: Record): Record { - return { ...scrubbedParentEnv(), ...extra } -} - -/** - * Resolve the server executable to an absolute path: an absolute command is verified directly; a - * bare command is looked up on the child's PATH. Fails loudly when nothing is executable. - */ -function resolveExecutable(command: string, childEnv: Record): string { - if (isAbsolute(command)) { - // Verify an absolute command too, so an unavailable one fails at load, not on the first query. - if (!isExecutableFileSync(command)) { - throw new Error(`lsp-local: command "${command}" is not an executable file`) - } - return command - } - /* v8 ignore next -- buildChildEnv always sets PATH from the ambient env; the further fallbacks are defensive. */ - const pathValue = childEnv.PATH ?? process.env.PATH ?? '' - for (const dir of pathValue.split(delimiter)) { - if (dir === '') continue - const candidate = join(dir, command) - if (isExecutableFileSync(candidate)) return candidate - } - throw new Error(`lsp-local: command "${command}" was not found on PATH`) -} - -/** Synchronous regular-file and executable check used only at load-time resolution. */ -function isExecutableFileSync(path: string): boolean { - try { - if (!statSync(path).isFile()) return false - accessSync(path, constants.X_OK) - return true - } catch { - return false - } -} diff --git a/packages/lsp/lsp-local/src/instance.ts b/packages/lsp/lsp-local/src/instance.ts index c1f78eaa38..8584f64f48 100644 --- a/packages/lsp/lsp-local/src/instance.ts +++ b/packages/lsp/lsp-local/src/instance.ts @@ -7,7 +7,6 @@ * @module @deepseek-ai/dsh-lsp-local/instance */ -import { pathToFileURL } from 'node:url' import { LspError } from '@deepseek-ai/dsh-lsp' import type { LspOperation, @@ -31,6 +30,8 @@ import { /** Everything an instance needs beyond the connection spec. */ export interface InstanceSpec extends ConnectionSpec { + /** Canonical workspace file URI supplied by the filesystem provider. */ + readonly workspaceUri: string /** Static `initialize` options forwarded to the server. */ readonly initializationOptions: unknown /** Graceful `shutdown`/`exit` budget before escalation (ms). */ @@ -108,9 +109,11 @@ export class LspInstance { private async initialize(): Promise { const initializeResult = await this.connection.request('initialize', { - processId: process.pid, - rootUri: pathToFileURL(this.spec.cwd).href, - workspaceFolders: [{ uri: pathToFileURL(this.spec.cwd).href, name: 'workspace' }], + // A subprocess provider may run in another PID namespace or machine; + // the host PID would let the server monitor an unrelated process. + processId: null, + rootUri: this.spec.workspaceUri, + workspaceFolders: [{ uri: this.spec.workspaceUri, name: 'workspace' }], capabilities: CLIENT_CAPABILITIES, initializationOptions: this.spec.initializationOptions, }) as WireInitializeResult @@ -147,7 +150,7 @@ export class LspInstance { throw new LspError('server does not support the transient textDocument/didOpen this host requires', 'LSP_UNSUPPORTED_OPERATION') } - const uri = pathToFileURL(source.canonicalPath).href + const uri = source.fileUrl let opened = false try { /* v8 ignore next -- guards an abort landing between the ready wait and didOpen; not deterministically reproducible. */ diff --git a/packages/lsp/lsp-local/tests/built-lib.e2e.ts b/packages/lsp/lsp-local/tests/built-lib.e2e.ts index bdd0ec1067..75f9f227c3 100644 --- a/packages/lsp/lsp-local/tests/built-lib.e2e.ts +++ b/packages/lsp/lsp-local/tests/built-lib.e2e.ts @@ -16,8 +16,9 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest' const pkgDir = fileURLToPath(new URL('..', import.meta.url)) const seamLib = join(pkgDir, '../lsp/lib/index.js') +const fsLib = join(pkgDir, '../../fs/fs-local/lib/index.js') const subprocessLib = join(pkgDir, '../../subprocess/subprocess-local/lib/index.js') -const built = existsSync(join(pkgDir, 'lib/index.js')) && existsSync(seamLib) && existsSync(subprocessLib) +const built = existsSync(join(pkgDir, 'lib/index.js')) && existsSync(seamLib) && existsSync(fsLib) && existsSync(subprocessLib) const fixtureServer = fileURLToPath(new URL('./fixture-server.ts', import.meta.url)) @@ -42,10 +43,12 @@ describe.skipIf(!built)('built lib real load path (plain node)', () => { const { Context } = await import('cordis') const { default: Lsp } = await import('@deepseek-ai/dsh-lsp') const LspLocal = await import('@deepseek-ai/dsh-lsp-local') + const { default: LocalFileSystem } = await import('@deepseek-ai/dsh-fs-local') const { default: LocalSubprocessService } = await import('@deepseek-ai/dsh-subprocess-local') const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { fake: { diff --git a/packages/lsp/lsp-local/tests/host.spec.ts b/packages/lsp/lsp-local/tests/host.spec.ts index 26aacdc1f4..cd7ed87e17 100644 --- a/packages/lsp/lsp-local/tests/host.spec.ts +++ b/packages/lsp/lsp-local/tests/host.spec.ts @@ -4,7 +4,10 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { realpath } from 'node:fs/promises' import { execFile } from 'node:child_process' +import { pathToFileURL } from 'node:url' import { promisify } from 'node:util' +import { Context } from 'cordis' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import { deadline } from '@deepseek-ai/dsh-timeout' import { canonicalizeWorkspace, readHostSource } from '@deepseek-ai/dsh-lsp-local' @@ -12,84 +15,105 @@ const execFileAsync = promisify(execFile) let root: string let ws: string +let ctx: Context +let fs: LocalFileSystem beforeEach(async () => { root = await realpath(await mkdtemp(join(tmpdir(), 'lsp-host-'))) ws = join(root, 'ws') await mkdir(ws) + ctx = new Context() + await ctx.plugin(LocalFileSystem, { cwd: root }) + fs = ctx.fs as LocalFileSystem }) afterEach(async () => { + await ctx.fiber.dispose() await rm(root, { recursive: true, force: true }) }) const BIG = 1_000_000 +async function workspace() { + return await canonicalizeWorkspace(fs, ws) +} + +async function readSource(filePath: string, maxBytes = BIG, signal?: AbortSignal) { + return await readHostSource(fs, filePath, await workspace(), maxBytes, signal) +} + describe('canonicalizeWorkspace', () => { it('returns the realpath of a directory', async () => { - expect(await canonicalizeWorkspace(ws)).toBe(ws) + expect((await workspace()).canonicalPath).toBe(ws) }) it('resolves a symlinked workspace to its target so aliases share identity', async () => { const link = join(root, 'ws-link') await symlink(ws, link) - expect(await canonicalizeWorkspace(link)).toBe(ws) + expect((await canonicalizeWorkspace(fs, link)).canonicalPath).toBe(ws) }) it('rejects a missing workspace', async () => { - await expect(canonicalizeWorkspace(join(root, 'nope'))).rejects.toThrow(/cannot be resolved/) + await expect(canonicalizeWorkspace(fs, join(root, 'nope'))).rejects.toThrow(/not a directory/) }) it('rejects a non-directory workspace', async () => { const file = join(root, 'file.txt') await writeFile(file, 'x') - await expect(canonicalizeWorkspace(file)).rejects.toThrow(/not a directory/) + await expect(canonicalizeWorkspace(fs, file)).rejects.toThrow(/not a directory/) }) }) describe('readHostSource', () => { it('reads a relative path against the workspace', async () => { await writeFile(join(ws, 'a.ts'), 'const x = 1\n') - const source = await readHostSource('a.ts', ws, BIG) - expect(source.canonicalPath).toBe(join(ws, 'a.ts')) + const source = await readSource('a.ts') + expect(source.fileUrl).toBe(pathToFileURL(join(ws, 'a.ts')).href) expect(source.text).toBe('const x = 1\n') }) it('reads an absolute path inside the workspace', async () => { const abs = join(ws, 'b.ts') await writeFile(abs, 'b') - const source = await readHostSource(abs, ws, BIG) - expect(source.canonicalPath).toBe(abs) + const source = await readSource(abs) + expect(source.fileUrl).toBe(pathToFileURL(abs).href) }) it('accepts a source reached through a symlink that stays inside the workspace', async () => { await mkdir(join(ws, 'real')) await writeFile(join(ws, 'real', 'c.ts'), 'c') await symlink(join(ws, 'real'), join(ws, 'linked')) - const source = await readHostSource('linked/c.ts', ws, BIG) - expect(source.canonicalPath).toBe(join(ws, 'real', 'c.ts')) + const source = await readSource('linked/c.ts') + expect(source.fileUrl).toBe(pathToFileURL(join(ws, 'real', 'c.ts')).href) }) it('rejects a source whose canonical path escapes the workspace via symlink', async () => { const outside = join(root, 'outside.ts') await writeFile(outside, 'secret') await symlink(outside, join(ws, 'escape.ts')) - await expect(readHostSource('escape.ts', ws, BIG)).rejects.toThrow(/outside the workspace/) + await expect(readSource('escape.ts')).rejects.toThrow(/outside the workspace/) }) it('rejects an absolute source outside the workspace', async () => { const outside = join(root, 'out.ts') await writeFile(outside, 'x') - await expect(readHostSource(outside, ws, BIG)).rejects.toThrow(/outside the workspace/) + await expect(readSource(outside)).rejects.toThrow(/outside the workspace/) }) it('rejects a missing source', async () => { - await expect(readHostSource('nope.ts', ws, BIG)).rejects.toThrow(/cannot be resolved/) + await expect(readSource('nope.ts')).rejects.toThrow(/not found/) + }) + + it('wraps a provider failure while resolving the source', async () => { + const canonical = await workspace() + fs.resolve = async () => { throw 'raw resolve failure' } + await expect(readHostSource(fs, 'broken.ts', canonical, BIG)) + .rejects.toThrow('source "broken.ts" cannot be resolved: raw resolve failure') }) it('rejects a non-regular source (directory)', async () => { await mkdir(join(ws, 'dir')) - await expect(readHostSource('dir', ws, BIG)).rejects.toThrow(/not a regular file/) + await expect(readSource('dir')).rejects.toThrow(/not a regular file/) }) // Windows has no filesystem FIFO; the directory case above pins non-regular rejection there. @@ -97,36 +121,36 @@ describe('readHostSource', () => { const fifo = join(ws, 'pipe.ts') await execFileAsync('mkfifo', [fifo]) using d = deadline(undefined, 1000, 'FIFO_READ_TIMEOUT') - await expect(readHostSource('pipe.ts', ws, BIG, d.signal)).rejects.toThrow(/not a regular file/) + await expect(readSource('pipe.ts', BIG, d.signal)).rejects.toThrow(/not a regular file/) }) it('honors a pre-aborted source read before filesystem work', async () => { const controller = new AbortController() controller.abort(new Error('source read cancelled')) - await expect(readHostSource('missing.ts', ws, BIG, controller.signal)).rejects.toThrow(/source read cancelled/) + await expect(readSource('missing.ts', BIG, controller.signal)).rejects.toThrow(/source read cancelled/) }) it('treats the workspace root itself as inside, then rejects it as non-regular', async () => { - // filePath '.' canonicalizes to the workspace dir: isInside's identity branch is taken, and the - // directory then fails the regular-file check. - await expect(readHostSource('.', ws, BIG)).rejects.toThrow(/not a regular file/) + // The filesystem containment primitive accepts the workspace itself; the + // bounded read then rejects the directory as non-regular. + await expect(readSource('.')).rejects.toThrow(/not a regular file/) }) it('rejects an oversized source', async () => { await writeFile(join(ws, 'big.ts'), 'x'.repeat(100)) - await expect(readHostSource('big.ts', ws, 10)).rejects.toThrow(/over the 10-byte limit/) + await expect(readSource('big.ts', 10)).rejects.toThrow(/10-byte limit/) }) it('rejects a non-UTF-8 source', async () => { await writeFile(join(ws, 'bin.ts'), Buffer.from([0xff, 0xfe, 0x00])) - await expect(readHostSource('bin.ts', ws, BIG)).rejects.toThrow(/not valid UTF-8/) + await expect(readSource('bin.ts')).rejects.toThrow(/invalid UTF-8|binary file/) }) it('keeps a valid U+FFFD replacement character in otherwise-valid UTF-8', async () => { // The literal replacement char is valid UTF-8; a fatal decoder must accept it (only malformed // byte sequences are rejected). await writeFile(join(ws, 'repl.ts'), 'const s = "�"\n') - const source = await readHostSource('repl.ts', ws, BIG) + const source = await readSource('repl.ts') expect(source.text).toBe('const s = "�"\n') }) }) diff --git a/packages/lsp/lsp-local/tests/instance.spec.ts b/packages/lsp/lsp-local/tests/instance.spec.ts index 75b9e541fc..3027cbbd29 100644 --- a/packages/lsp/lsp-local/tests/instance.spec.ts +++ b/packages/lsp/lsp-local/tests/instance.spec.ts @@ -3,6 +3,8 @@ import { mkdtemp, mkdir, readFile, rm, writeFile, realpath } from 'node:fs/promi import { tmpdir } from 'node:os' import { join } from 'node:path' import { pathToFileURL, fileURLToPath } from 'node:url' +import { Context } from 'cordis' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import { LspInstance, readHostSource } from '@deepseek-ai/dsh-lsp-local' import { encodeMessage } from '@deepseek-ai/dsh-lsp-local' import type { ConnectionWriter } from '@deepseek-ai/dsh-lsp-local/src/connection.ts' @@ -15,6 +17,8 @@ const fixtureServer = fileURLToPath(new URL('./fixture-server.ts', import.meta.u let root: string let ws: string +let ctx: Context +let fs: LocalFileSystem let live: LspInstance[] = [] beforeEach(async () => { @@ -22,11 +26,15 @@ beforeEach(async () => { ws = join(root, 'ws') await mkdir(ws) await writeFile(join(ws, 'a.ts'), 'const x = 1\n') + ctx = new Context() + await ctx.plugin(LocalFileSystem, { cwd: root }) + fs = ctx.fs as LocalFileSystem }) afterEach(async () => { for (const instance of live) await instance.dispose() live = [] + await ctx.fiber.dispose() await rm(root, { recursive: true, force: true }) }) @@ -39,6 +47,7 @@ function makeInstance( command: process.execPath, args: [fixtureServer], cwd: ws, + workspaceUri: pathToFileURL(ws).href, env: { ...scrubbedParentEnv(), ...env }, configuration: { setting: 42 }, initializationOptions: { init: true }, @@ -58,7 +67,12 @@ function query(operation: LspProviderQuery['operation'] = 'goToDefinition'): Lsp /** Run a query against an instance, reading the source first the way the provider does. */ async function run(instance: LspInstance, operation: LspProviderQuery['operation'] = 'goToDefinition', signal?: AbortSignal): Promise { - const source = await readHostSource('a.ts', ws, 4_000_000) + const workspace = { + target: await fs.resolve(ws), + canonicalPath: ws, + fileUrl: pathToFileURL(ws).href, + } + const source = await readHostSource(fs, 'a.ts', workspace, 4_000_000) return instance.query(query(operation), source, signal) } @@ -68,6 +82,7 @@ function scriptInstance(script: string, overrides: Partial = {}): command: process.execPath, args: ['-e', script], cwd: ws, + workspaceUri: pathToFileURL(ws).href, env: scrubbedParentEnv(), configuration: null, initializationOptions: null, diff --git a/packages/lsp/lsp-local/tests/lifecycle.spec.ts b/packages/lsp/lsp-local/tests/lifecycle.spec.ts index ccd85a1e14..a7a006c06e 100644 --- a/packages/lsp/lsp-local/tests/lifecycle.spec.ts +++ b/packages/lsp/lsp-local/tests/lifecycle.spec.ts @@ -8,6 +8,7 @@ import { Context } from 'cordis' import Lsp, { type LspProvider, type LspQueryRequest, type LspQueryResult } from '@deepseek-ai/dsh-lsp' import { deadline } from '@deepseek-ai/dsh-timeout' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' import type { LspLocalServerConfig } from '@deepseek-ai/dsh-lsp-local' @@ -47,6 +48,7 @@ async function mount( const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) const register = ctx.lsp.registerProvider.bind(ctx.lsp) const registrationSpy = captureProvider === undefined ? undefined @@ -79,6 +81,7 @@ describe('lsp-local end to end over a fake server', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { typescript: fakeServer({ LSP_FAKE_HOVER: JSON.stringify({ contents: 'ts' }) }), @@ -322,6 +325,7 @@ describe('lsp-local end to end over a fake server', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { missing: { diff --git a/packages/lsp/lsp-local/tests/provider.spec.ts b/packages/lsp/lsp-local/tests/provider.spec.ts index 77fcbe7851..b0209e09b8 100644 --- a/packages/lsp/lsp-local/tests/provider.spec.ts +++ b/packages/lsp/lsp-local/tests/provider.spec.ts @@ -4,6 +4,7 @@ import { tmpdir } from 'node:os' import { delimiter, join } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import Lsp, { type LspQueryRequest } from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' import type { Config, LspLocalServerConfig } from '@deepseek-ai/dsh-lsp-local' @@ -44,6 +45,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('onpath', { command: 'fake-lsp', args: [], @@ -57,6 +59,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('nope', { command: 'fake-lsp', args: [], @@ -71,6 +74,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) // Grab the provider instance by registering, then dispose the whole plugin fiber. const lsp = ctx.lsp const fiber = await ctx.plugin(LspLocal, config('disp', { @@ -88,6 +92,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-budget', { command: process.execPath, args: ['-e', ''], @@ -101,6 +106,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-cap', { command: process.execPath, args: ['-e', ''], @@ -114,6 +120,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('bad-timer', { command: process.execPath, args: ['-e', ''], @@ -130,6 +137,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('abs-bad', { command: notExe, args: [], @@ -142,6 +150,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('abs-directory', { command: ws, args: [], @@ -154,6 +163,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: {} })).rejects.toThrow(/servers must contain at least one server/) await ctx.fiber.dispose() }) @@ -162,6 +172,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, config('', { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' }, @@ -173,6 +184,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { valid: { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' } }, @@ -187,6 +199,7 @@ describe('lsp-local provider resolution', () => { const ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await expect(ctx.plugin(LspLocal, { servers: { first: { command: process.execPath, extensionToLanguage: { '.ts': 'typescript' } }, diff --git a/packages/lsp/lsp-local/tests/typescript-server.e2e.ts b/packages/lsp/lsp-local/tests/typescript-server.e2e.ts index fe9230ce14..9bdb7d1fd4 100644 --- a/packages/lsp/lsp-local/tests/typescript-server.e2e.ts +++ b/packages/lsp/lsp-local/tests/typescript-server.e2e.ts @@ -11,6 +11,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import Lsp, { type LspQueryRequest, type LspQueryResult } from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' @@ -54,6 +55,7 @@ beforeAll(async () => { ctx = new Context() await ctx.plugin(Lsp) await ctx.plugin(LocalSubprocessService) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LspLocal, { servers: { typescript: { diff --git a/packages/lsp/lsp-local/tsconfig.json b/packages/lsp/lsp-local/tsconfig.json index 2b106ddc81..04fdeb9a84 100644 --- a/packages/lsp/lsp-local/tsconfig.json +++ b/packages/lsp/lsp-local/tsconfig.json @@ -26,6 +26,9 @@ { "path": "../../llm/llm" }, + { + "path": "../../fs/fs" + }, { "path": "../lsp" }, diff --git a/packages/lsp/tool-lsp/package.json b/packages/lsp/tool-lsp/package.json index 28b5a34bdb..f757df35fd 100644 --- a/packages/lsp/tool-lsp/package.json +++ b/packages/lsp/tool-lsp/package.json @@ -38,11 +38,13 @@ }, "devDependencies": { "@deepseek-ai/dsh-agent": "workspace:^", + "@deepseek-ai/dsh-fs-local": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-lsp": "workspace:^", "@deepseek-ai/dsh-lsp-local": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", "@deepseek-ai/dsh-timeout": "workspace:^", "@deepseek-ai/dsh-timeout-policy": "workspace:^", diff --git a/packages/lsp/tool-lsp/tests/integration.spec.ts b/packages/lsp/tool-lsp/tests/integration.spec.ts index 4d790cbd6b..8312a6e3ab 100644 --- a/packages/lsp/tool-lsp/tests/integration.spec.ts +++ b/packages/lsp/tool-lsp/tests/integration.spec.ts @@ -6,6 +6,7 @@ import { pathToFileURL } from 'node:url' import { Context } from 'cordis' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRegistry from '@deepseek-ai/dsh-tools' +import LocalFileSystem from '@deepseek-ai/dsh-fs-local' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import Lsp from '@deepseek-ai/dsh-lsp' import * as LspLocal from '@deepseek-ai/dsh-lsp-local' @@ -50,6 +51,7 @@ async function mount(hang: boolean, timeoutMs?: number): Promise { await ctx.plugin(SystemPrompt) await ctx.plugin(ToolRegistry) await ctx.plugin(Lsp) + await ctx.plugin(LocalFileSystem, { cwd: process.cwd() }) await ctx.plugin(LocalSubprocessService) await ctx.plugin(LspLocal, { servers: { diff --git a/packages/pty/README.i18n.yaml b/packages/pty/README.i18n.yaml index 083a144b55..65d541a7c1 100644 --- a/packages/pty/README.i18n.yaml +++ b/packages/pty/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/README.md -README.md: 9c8206464d45b1be1d6ee3861c57c128e77686c5 -README.zh.md: 70d081e60a7db61443ed616b64586a93c119a640 +README.md: a4f743056b4a524be9623b0f700f37e0534b463f +README.zh.md: c84ad3f1b59afcdbbd111f1b82c57c56aa24fdcf diff --git a/packages/pty/README.md b/packages/pty/README.md index 9c8206464d..a4f743056b 100644 --- a/packages/pty/README.md +++ b/packages/pty/README.md @@ -2,13 +2,12 @@ English | [中文](README.zh.md) -This family provides persistent, owner-scoped pseudo-terminal sessions for interactive or stateful terminal work. It complements one-shot bash execution. +`PTY` stands for **Pseudo-Terminal**（伪终端）. This capability provides persistent, owner-scoped terminal sessions for workflows that require state across tool calls or interactive stdin. PTY complements the one-shot bash and filesystem tools; it does not replace their stronger per-operation contracts. | Package | Role | ctx key | |---|---|---| -| [`pty/`](pty/README.md) | Defines the PTY service and session lifecycle | `ctx.pty` | -| [`pty-local/`](pty-local/README.md) | Provides local persistent terminal sessions | registers on `ctx.pty` | -| [`tool-pty/`](tool-pty/README.md) | Exposes PTY session operations to the model | registers on `ctx.tools` | -| [`tool-bash-persistent/`](tool-bash-persistent/README.md) | Exposes a reusable PTY-backed bash tool | registers on `ctx.tools` | +| [`pty`](pty/README.md) (`@deepseek-ai/dsh-pty`) | Backend registry, branded ids, exact-Agent ownership, session operations, and awaited cleanup | `ctx.pty` | +| `pty-local` (`@deepseek-ai/dsh-pty-local`) | Shell backend over `ctx.subprocess.spawnTerminal`: readiness detection, bounded terminal state, sandbox policy, and session operations | registers on `ctx.pty` | +| `tool-pty` (`@deepseek-ai/dsh-tool-pty`) | Six model-facing tools and generic task integration for background sends | registers on `ctx.tools` | -The [persistent PTY decision](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) records the family boundary. +The design and deferred boundaries live in the [persistent PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md). diff --git a/packages/pty/README.zh.md b/packages/pty/README.zh.md index 70d081e60a..c84ad3f1b5 100644 --- a/packages/pty/README.zh.md +++ b/packages/pty/README.zh.md @@ -2,13 +2,12 @@ [English](README.md) | 中文 -本家族为交互式或有状态的终端工作提供持久且限定所有者范围的伪终端会话，是单次 bash 执行的补充。 +`PTY` 的全称是 **Pseudo-Terminal（伪终端）**。这项能力提供持久且限定所有者范围的终端会话，适用于需要跨工具调用保留状态或使用交互式 stdin 的工作流。PTY 是单次 bash 与文件系统工具的补充，不会取代后两者更严格的逐操作契约。 | 包 | 职责 | ctx 键 | |---|---|---| -| [`pty/`](pty/README.md) | 定义 PTY 服务和会话生命周期 | `ctx.pty` | -| [`pty-local/`](pty-local/README.md) | 提供本地持久终端会话 | 注册到 `ctx.pty` | -| [`tool-pty/`](tool-pty/README.md) | 向模型公开 PTY 会话操作 | 注册到 `ctx.tools` | -| [`tool-bash-persistent/`](tool-bash-persistent/README.md) | 公开可复用的 PTY 后端 bash 工具 | 注册到 `ctx.tools` | +| [`pty`](pty/README.md)（`@deepseek-ai/dsh-pty`） | 后端注册表、品牌化 id、精确的 Agent 所有权、会话操作与等待完成的清理 | `ctx.pty` | +| `pty-local`（`@deepseek-ai/dsh-pty-local`） | `ctx.subprocess.spawnTerminal` 之上的 shell 后端：就绪检测、有界终端状态、沙箱策略与会话操作 | 注册到 `ctx.pty` | +| `tool-pty`（`@deepseek-ai/dsh-tool-pty`） | 6 个面向模型的工具，并为后台发送集成通用任务 | 注册到 `ctx.tools` | -[持久 PTY 决策](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md)记录了该家族的边界。 +设计与暂缓边界记录在[持久 PTY Agent Note](../../.agents/notes/implemented/feature/2026-07-16-persistent-pty-sessions.md) 中。 diff --git a/packages/pty/pty-local/README.i18n.yaml b/packages/pty/pty-local/README.i18n.yaml index 48ae675425..d8f5c2e2d0 100644 --- a/packages/pty/pty-local/README.i18n.yaml +++ b/packages/pty/pty-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/pty/pty-local/README.md -README.md: ba05495318127b63b3d2a6a60ec743e1ff1c5821 -README.zh.md: 81987ea0685d761507b535b7ed6eefa0888fbd54 +README.md: 8a0a60139e27d98c4f506245a204723da997cd4a +README.zh.md: 3df845931dac33abecbcd5030a1b204c68acc638 diff --git a/packages/pty/pty-local/README.md b/packages/pty/pty-local/README.md index ba05495318..8a0a60139e 100644 --- a/packages/pty/pty-local/README.md +++ b/packages/pty/pty-local/README.md @@ -2,35 +2,35 @@ English | [中文](README.zh.md) -Local Linux/macOS `node-pty` backend for `ctx.pty`; loading it on another platform fails as unsupported. It starts an interactive shell under the shared `ctx.sandboxPolicy`, strips credential-shaped ambient environment variables, retains bounded line-oriented output, detects readiness, and tears down the captured process tree rooted at the `node-pty` child. +Persistent shell backend for `ctx.pty` over `ctx.subprocess.spawnTerminal`. It starts an interactive shell under the shared `ctx.sandboxPolicy`, retains bounded line-oriented output, and detects readiness while the subprocess provider owns PTY allocation, environment scrubbing, foreground process groups, signalling, and complete terminal-session cleanup. The same PTY backend therefore composes with local or remote execution-world providers. ## Plugin (`pty-local`) -The plugin injects `pty`, `sandbox`, and `sandboxPolicy`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. At spawn, one `ctx.sandboxPolicy.resolve({ session })` call supplies both the effective mode and the session workspace root; the same root is the default shell cwd when the caller omits one. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a local-provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. +The plugin injects `pty`, `sandbox`, `sandboxPolicy`, and `subprocess`, then registers the configured backend type (`shell`). `danger-full-access` starts the shell directly; confined modes wrap the exact shell argv through `ctx.sandbox`. The effective session mode is resolved at spawn. A change to a different effective mode is rejected before its `sandbox/mode` event commits while that owner has an open PTY or a spawn in progress; the fence is attached to the exact owner and therefore outlives a provider reload that retains existing sessions. Wait for creation to settle and close the sessions before changing modes, so a terminal opened with wider access cannot survive a downgrade. -Linux readiness combines a foreground-verified private bash prompt marker, foreground-process-group syscall inspection, silence fallback, and absolute timeout. macOS uses the verified prompt marker plus silence/timeout because it has no `/proc` syscall surface. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the kernel publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win; that grace must cover at least one `pollIntervalMs` and is rejected at load otherwise. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unrecognized or unreadable process state is never a positive exact-idle signal. A foreground group's stdin wait that already existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason even when its foreground process group is not observable yet; if that close fails, `PtyBackendCleanupError` separately preserves the cleanup failure for registry disposal. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. +Readiness combines a foreground-verified private bash prompt marker, provider-reported foreground stdin-wait facts, silence fallback, and absolute timeout. A marker is not ready until printable prompt text arrives, including when the OSC marker and `PS1` are split across data callbacks; when bash prints the marker before the terminal provider publishes its return to the foreground process group, polling retains the candidate for `handoffGraceMs` past the ordinary silence bound so a coincident handoff can win. An interactive child that inherits `PROMPT_COMMAND` therefore cannot suppress inferred-idle readiness until the absolute timeout. Unknown foreground state is never a positive exact-idle signal. A foreground group's stdin wait that existed before a send is likewise not post-write readiness: the same group must be observed outside that wait before a later wait can settle the send, while a changed foreground group is new evidence. During unpublished startup, a fallback requires observed output; zero-output silence cannot publish an empty session, and timeout rejects the spawn. Cancellation closes the unpublished shell and rejects with the caller's exact abort reason; `PtyBackendCleanupError` separately preserves a cleanup failure. Incomplete terminal-control sequences are bounded by `maxReadBytes` and discarded through their terminator after crossing that limit; a trailing carriage return is carried across callbacks so split CRLF becomes one newline. -Send cancellation resolves the current foreground process group and delivers a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close sends `SIGTERM` to descendants, waits, then sends `SIGKILL` to the union of captured survivors and newly scanned descendants so reparenting cannot hide a process from teardown. It verifies that every retained identity is gone or, on Linux, a non-executing zombie before stopping the shell; zombie entries are quiescent and are reaped as the shell exits. A survivor failure does not cache a permanently rejected close; a later close retries the teardown. +Send cancellation asks the terminal handle to signal the current foreground process group with a real `SIGINT`; it never emulates interruption by writing `\x03`, so raw-mode programs remain cancellable. Close starts provider-owned TERM-to-KILL whole-session cleanup and awaits quiescence after the terminal outcome. A cleanup failure does not cache a permanently rejected close; a later close retries the provider operation. ## Model Experience -### Current file policy and indirect consumer +### Indirect consumer #### What the model sees -The policy owner contributes capability-neutral `sandbox:policy` context. Through `@deepseek-ai/dsh-tool-pty` or another PTY consumer, the model may also receive bounded MOTD, send deltas, scrollback pages, readiness reasons, and cleanup errors. +Nothing directly. Through `@deepseek-ai/dsh-tool-pty`, the model may receive bounded MOTD, send deltas, scrollback pages, readiness reasons, and cleanup errors. #### Token effect -The current-policy clause is present while this backend is mounted. Retained PTY scrollback is not placed in model history until a consumer returns bounded output. +None until a consumer returns bounded backend output. Retained PTY scrollback is not placed in model history by this package. #### KV Cache effect -A standing-policy change appends an owner-rendered superseding runtime-context snapshot after retained history; consumer results remain append-only. +No direct invalidation; the consumer owns prompts, schemas, and appended results. ## Known Limitations and Deferred Work - Line-oriented output is normalized; full-screen alternate-buffer interaction is unsupported. -- Linux exact probes support x64 and arm64 UAPI tables; other architectures use prompt-marker and silence/timeout readiness. -- A descendant that daemonizes and reparents before teardown leaves the captured tree; cleanup never broadens to the launcher PID's POSIX session because that can include unrelated processes. +- Exact stdin-wait detection depends on the mounted subprocess provider; providers that cannot prove it use prompt-marker and silence/timeout readiness. +- Cleanup guarantees are those of `SubprocessTerminalHandle`; provider-specific gaps belong to that implementation's contract rather than this PTY consumer. - Sessions do not survive harness process exit. diff --git a/packages/pty/pty-local/README.zh.md b/packages/pty/pty-local/README.zh.md index 81987ea068..3df845931d 100644 --- a/packages/pty/pty-local/README.zh.md +++ b/packages/pty/pty-local/README.zh.md @@ -2,35 +2,35 @@ [English](README.md) | 中文 -这个本地 Linux／macOS `node-pty` 后端实现 `ctx.pty`；在其他平台加载时会以不支持为由失败。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell，移除形似凭据的环境变量，保留有界的逐行输出，检测就绪状态，并清理以 `node-pty` 子进程为根的已捕获进程树。 +这是基于 `ctx.subprocess.spawnTerminal` 的 `ctx.pty` 持久 shell 后端。它在共享 `ctx.sandboxPolicy` 下启动交互式 shell，保留有界的逐行输出并检测就绪状态；进程管理提供方负责 PTY 分配、环境清理、前台进程组、信号发送和完整终端会话清理。因此，同一个 PTY 后端可以与本地或远程执行世界提供方组合。 ## 插件（`pty-local`） -该插件注入 `pty`、`sandbox` 和 `sandboxPolicy`，然后注册所配置的后端类型（`shell`）。`danger-full-access` 会直接启动 shell；受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。Spawn 时，一次 `ctx.sandboxPolicy.resolve({ session })` 调用会同时给出实际模式与会话工作区根目录；调用方省略 cwd 时，同一根目录也是 shell 的默认 cwd。当某个所有者存在开放的 PTY 或正在进行 spawn 时，如果配置变更会得到不同的实际模式，系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者，因此即使本地提供方重新加载并保留现有会话，它仍然有效。更改模式前，请等待创建完成并关闭会话，避免以更宽权限打开的终端在权限降级后继续存在。 +该插件注入 `pty`、`sandbox`、`sandboxPolicy` 和 `subprocess`，然后注册所配置的后端类型（`shell`）。`danger-full-access` 会直接启动 shell；受限模式则通过 `ctx.sandbox` 包装确切的 shell argv。系统在 spawn 时解析会话的实际模式。当某个所有者存在开放的 PTY 或正在进行 spawn 时，如果配置变更会得到不同的实际模式，系统会在对应 `sandbox/mode` 事件提交前拒绝该变更。该限制绑定到确切所有者，因此即使提供方重新加载并保留现有会话，它仍然有效。更改模式前，请等待创建结算并关闭会话，避免以更宽权限打开的终端在权限降级后继续存在。 -Linux 的就绪检测结合以下机制：由前台状态验证的私有 bash 提示符标记、前台进程组 syscall 检查、静默回退和绝对超时。macOS 没有 `/proc` syscall 接口，因此使用经过验证的提示符标记以及静默／超时。当可打印的提示符文本尚未到达时，即使 OSC 标记和 `PS1` 被拆到多个数据回调中，系统也不会把标记视为就绪。如果 bash 在内核发布其重新取得前台进程组的状态前打印标记，轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`，使恰好同时发生的前台交接有机会胜出；该宽限至少要覆盖一个 `pollIntervalMs`，否则加载时即被拒绝。因此，继承 `PROMPT_COMMAND` 的交互式子进程无法一直抑制推断空闲就绪直至绝对超时。无法识别或读取的进程状态绝不会作为精确空闲的正向信号。同样，一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪：必须先观察到同一进程组脱离该等待，之后再次进入等待才能使该次 send 完成；前台进程组发生变化则构成新的证据。尚未发布的启动过程中，回退路径要求已经观察到输出；零输出静默不能发布空会话，超时则拒绝 spawn。取消操作会关闭尚未发布的 shell，并以调用方提供的确切中止原因拒绝，即使当时还无法观察其前台进程组。如果关闭失败，`PtyBackendCleanupError` 会单独保留清理失败，供注册表 dispose（资源释放）时处理。未完成的终端控制序列受 `maxReadBytes` 限制；超过上限后，系统会丢弃内容直到其终止符。末尾的回车会跨回调保留，使拆分的 CRLF 合并为一个换行。 +就绪检测结合以下机制：由前台状态验证的私有 bash 提示符标记、提供方报告的前台 stdin 等待事实、静默回退和绝对超时。可打印的提示符文本尚未到达时，即使 OSC 标记和 `PS1` 被拆到多个数据回调中，系统也不会把标记视为就绪。如果 bash 在终端提供方发布其重新取得前台进程组的状态前打印标记，轮询会在普通静默上限之后再保留该候选状态 `handoffGraceMs`，使恰好同时发生的前台交接有机会胜出。因此，继承 `PROMPT_COMMAND` 的交互式子进程无法持续压制推断空闲就绪，最多只能延续到绝对超时。未知的前台状态绝不会作为精确空闲的正向信号。同样，一次 send 之前就已存在的前台进程组 stdin 等待并不代表写入后就绪：必须先观察到同一进程组脱离该等待，之后再次进入等待才能使该次 send 完成；前台进程组发生变化则构成新的证据。尚未发布的启动过程中，回退路径要求已经观察到输出；零输出静默不能发布空会话，超时则拒绝 spawn。取消操作会关闭尚未发布的 shell，并以调用方提供的确切中止原因拒绝；`PtyBackendCleanupError` 会单独保留清理失败。未完成的终端控制序列受 `maxReadBytes` 限制；超过上限后，系统会丢弃内容直到其终止符。末尾的回车会跨回调保留，使拆分的 CRLF 合并为一个换行。 -取消发送时，系统会解析当前前台进程组并发送真正的 `SIGINT`；它绝不会通过写入 `\x03` 模拟中断，因此，即使程序运行在 raw 模式下，也仍可取消。关闭操作先向后代发送 `SIGTERM` 并等待，再向已捕获的存活进程与新扫描到的后代之并集发送 `SIGKILL`，防止进程通过重新设定父进程而逃避清理。在停止 shell 前，系统会确认每个保留的进程身份都已消失，或者在 Linux 上已成为不再执行的僵尸进程；僵尸进程条目视为完全停稳，并会随 shell 退出而回收。如果仍有进程存活，失败结果不会缓存成永久拒绝的关闭操作；后续关闭仍会重试清理。 +取消发送时，系统会请求终端句柄向当前前台进程组发送真正的 `SIGINT`；它绝不会通过写入 `\x03` 模拟中断，因此原始模式程序仍可取消。关闭操作启动由提供方负责的 TERM→KILL 全会话清理，并在终端结果之后等待完全停稳。清理失败不会缓存成永久拒绝的关闭操作；后续关闭会重试提供方操作。 ## 模型体验 -### 当前文件策略与间接消费方 +### 间接消费方 #### 模型看到的内容 -策略归属方会贡献与具体能力无关的 `sandbox:policy` 上下文。模型通过 `@deepseek-ai/dsh-tool-pty` 或其他 PTY 消费方还可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因和清理错误。 +没有直接可见内容。模型通过 `@deepseek-ai/dsh-tool-pty` 可能收到有界的 MOTD、发送增量、scrollback 页、就绪原因和清理错误。 #### Token 影响 -装载该后端期间，当前策略子句会一直存在。消费方返回有界输出前，保留的 PTY scrollback 不会进入模型历史。 +消费方返回有界的后端输出前没有影响。此包不会把保留的 PTY scrollback 放入模型历史。 #### KV Cache 影响 -常驻策略发生变化时，会在保留的历史之后追加一份由归属方渲染、取代先前状态的运行时上下文快照；消费方结果保持仅追加。 +不会直接失效；提示词、schema 与追加结果由消费方负责。 -## 已知限制与暂缓事项 +## 已知限制与暂缓工作 - 输出按行规范化；不支持全屏备用缓冲区交互。 -- Linux 精确探针支持 x64 与 arm64 UAPI 表；其他架构使用提示符标记和静默／超时就绪机制。 -- 如果后代进程在清理前守护化并重新设定父进程，它会脱离已捕获的进程树；清理绝不会扩大到启动器 PID 所属的整个 POSIX 会话，因为其中可能包含无关进程。 -- harness 进程退出后，会话无法继续存在。 +- 精确 stdin 等待检测取决于挂载的进程管理提供方；无法证明该事实的提供方使用提示符标记和静默／超时就绪机制。 +- 清理保证遵循 `SubprocessTerminalHandle`；提供方特有缺口属于该实现的契约，而非此 PTY 消费方。 +- 会话无法跨 harness 进程退出保留。 diff --git a/packages/pty/pty-local/package.json b/packages/pty/pty-local/package.json index a859f482fa..94f98c02d5 100644 --- a/packages/pty/pty-local/package.json +++ b/packages/pty/pty-local/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-pty-local", - "description": "Local node-pty backend for persistent DeepSeek Harness PTY sessions", + "description": "Persistent shell PTY backend over the DeepSeek Harness subprocess terminal primitive", "version": "0.0.1", "private": true, "type": "module", @@ -21,12 +21,8 @@ "files": [ "lib/index.js", "lib/invariant.js", - "scripts/ensure-spawn-helper.mjs", "lib/types/**/*.d.ts" ], - "scripts": { - "postinstall": "node scripts/ensure-spawn-helper.mjs" - }, "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-agent": "^0.0.1", @@ -39,7 +35,6 @@ "cordis": "^4.0.0-rc.7" }, "dependencies": { - "node-pty": "^1.1.0", "schemastery": "^3.18.0" }, "devDependencies": { @@ -50,6 +45,7 @@ "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "cordis": "^4.0.0-rc.7" } } diff --git a/packages/pty/pty-local/src/index.ts b/packages/pty/pty-local/src/index.ts index e59d53446a..0a54a0b804 100644 --- a/packages/pty/pty-local/src/index.ts +++ b/packages/pty/pty-local/src/index.ts @@ -1,22 +1,18 @@ /** - * Local persistent PTY backend using public `node-pty` APIs, shared sandbox - * policy, bounded output, platform readiness probes, and process-session cleanup. + * Persistent shell PTY backend over the subprocess terminal primitive, shared + * sandbox policy, bounded output, and provider-owned session cleanup. * @module @deepseek-ai/dsh-pty-local */ import { Context } from 'cordis' -import * as nodePty from 'node-pty' -import type { IPtyForkOptions } from 'node-pty' import type { Agent } from '@deepseek-ai/dsh-agent' import type { Session, SessionEvent } from '@deepseek-ai/dsh-session' import { PtyBackendCleanupError } from '@deepseek-ai/dsh-pty' -import { scrubbedParentEnv } from '@deepseek-ai/dsh-subprocess' import type { PtyBackend, PtyBackendSpawnSpec } from '@deepseek-ai/dsh-pty' -import type { SandboxExecutionPolicy } from '@deepseek-ai/dsh-sandbox' +import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { SandboxMode } from '@deepseek-ai/dsh-sandbox' import { effectiveSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import { type Config, type ResolvedConfig, validateConfig } from './config.ts' -import { createProcessInspector } from './process-inspector.ts' -import type { ProcessInspector } from './process-inspector.ts' import { LocalPtySession } from './session.ts' export { Config } from './config.ts' @@ -24,8 +20,8 @@ export type { Config as PtyLocalConfig } from './config.ts' /** Cordis plugin name. */ export const name = 'pty-local' -/** Required services: PTY registry plus the one shared confinement policy. */ -export const inject = ['pty', 'sandbox', 'sandboxPolicy'] +/** Required services: PTY registry, shared confinement policy, and process substrate. */ +export const inject = ['pty', 'sandbox', 'sandboxPolicy', 'subprocess'] interface SandboxModeFenceState { pty: Context['pty'] @@ -55,10 +51,10 @@ function ensureSandboxModeFence(ctx: Context, owner: Agent): void { }, { global: true }) } -function childEnvironment(spec: PtyBackendSpawnSpec): NodeJS.ProcessEnv { - // node-pty owns the spawn; the base env shares the subprocess seam's scrub. +function childEnvironment(spec: PtyBackendSpawnSpec): Record { + // The subprocess provider supplies its own scrubbed ambient base; these are + // deliberate terminal-specific overrides layered after it. return { - ...scrubbedParentEnv(), TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', @@ -71,11 +67,14 @@ function childEnvironment(spec: PtyBackendSpawnSpec): NodeJS.ProcessEnv { } } -function spawnArgv(ctx: Context, config: ResolvedConfig, policy: SandboxExecutionPolicy): string[] { +function spawnArgv(ctx: Context, config: ResolvedConfig, spec: PtyBackendSpawnSpec): string[] { const argv = [config.shellPath, ...config.shellArgs] - if (policy.mode === 'danger-full-access') return argv - // Re-state the discriminant because object spread does not preserve its narrowed type. - return ctx.sandbox.confine(argv, { ...policy, mode: policy.mode }).argv + const mode: SandboxMode = effectiveSandboxMode(spec.owner.session.events) ?? ctx.sandboxPolicy.defaultMode + if (mode === 'danger-full-access') return argv + return ctx.sandbox.confine(argv, { + mode: mode, + workspaceRoot: ctx.sandboxPolicy.workspaceRoot, + }).argv } /** Local shell backend registered under the configured type. */ @@ -85,13 +84,13 @@ export class LocalPtyBackend implements PtyBackend { constructor( private readonly ctx: Context, private readonly config: ResolvedConfig, - private readonly inspector: ProcessInspector, - private readonly spawnTerminal: typeof nodePty.spawn = nodePty.spawn, + private readonly spawnTerminal: ( + spec: SubprocessTerminalSpawnSpec, + ) => Promise = spec => ctx.subprocess.spawnTerminal(spec), private readonly createSession: ( - terminal: ReturnType, - inspector: ProcessInspector, + terminal: SubprocessTerminalHandle, config: ResolvedConfig, - ) => LocalPtySession = (terminal, inspector, config) => new LocalPtySession(terminal, inspector, config), + ) => LocalPtySession = (terminal, config) => new LocalPtySession(terminal, config), ) { this.type = config.backendType } @@ -99,19 +98,18 @@ export class LocalPtyBackend implements PtyBackend { async spawn(spec: PtyBackendSpawnSpec): Promise { spec.signal?.throwIfAborted() ensureSandboxModeFence(this.ctx, spec.owner) - const policy = this.ctx.sandboxPolicy.resolve({ session: spec.owner.session }) - const argv = spawnArgv(this.ctx, this.config, policy) - const file = argv[0] - if (file === undefined) throw new Error('pty-local: sandbox returned empty argv') - const options: IPtyForkOptions = { - name: 'dumb', - cols: this.config.cols, - rows: this.config.rows, - cwd: spec.cwd ?? policy.workspaceRoot, + const argv = spawnArgv(this.ctx, this.config, spec) + if (argv[0] === undefined) throw new Error('pty-local: sandbox returned empty argv') + const terminal = await this.spawnTerminal({ + argv, + cwd: spec.cwd ?? this.ctx.sandboxPolicy.workspaceRoot, env: childEnvironment(spec), - } - const terminal = this.spawnTerminal(file, argv.slice(1), options) - const session = this.createSession(terminal, this.inspector, this.config) + rows: this.config.rows, + cols: this.config.cols, + graceMs: this.config.disposeGraceMs, + signal: spec.signal, + }) + const session = this.createSession(terminal, this.config) try { await session.initialize(spec.signal) return session @@ -129,6 +127,5 @@ export class LocalPtyBackend implements PtyBackend { /** Register the local PTY backend. */ export function apply(ctx: Context, config: Config): void { validateConfig(config) - const inspector = createProcessInspector() - ctx.pty.registerBackend(new LocalPtyBackend(ctx, config, inspector)) + ctx.pty.registerBackend(new LocalPtyBackend(ctx, config)) } diff --git a/packages/pty/pty-local/src/session.ts b/packages/pty/pty-local/src/session.ts index 46f3be2ca8..a6abaf7785 100644 --- a/packages/pty/pty-local/src/session.ts +++ b/packages/pty/pty-local/src/session.ts @@ -1,8 +1,11 @@ -/** Local `node-pty` session: bounded output, readiness, signals, and teardown. */ +/** Persistent PTY session over the subprocess seam's terminal primitive. */ -import { constants } from 'node:os' import { Buffer } from 'node:buffer' -import type { IDisposable, IPty } from 'node-pty' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, +} from '@deepseek-ai/dsh-subprocess' import type { PtyBackendSession, PtyReadRequest, @@ -17,13 +20,8 @@ import type { PtyWaitReason, } from '@deepseek-ai/dsh-pty' import type { ResolvedConfig } from './config.ts' -import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' import { TerminalSanitizer } from './sanitize.ts' -function delay(ms: number): Promise { - return new Promise(resolve => setTimeout(resolve, ms)) -} - function utf8Tail(text: string, maxBytes: number): { text: string; truncated: boolean } { if (Buffer.byteLength(text) <= maxBytes) return { text, truncated: false } const chars = Array.from(text) @@ -80,17 +78,16 @@ class LocalSendOperation implements PtySendOperation { private readonly promise: PromiseWithResolvers private finished = false private initialForegroundLeftWait: boolean + private initialForegroundPgid: number | undefined constructor( maxBytes: number, readonly startedAt: number, - private readonly initialForegroundPgid: number | undefined, - initialForegroundWasWaiting: boolean, private readonly onCancel: () => void, ) { this.output = new BoundedTextBuffer(maxBytes) this.promise = Promise.withResolvers() - this.initialForegroundLeftWait = !initialForegroundWasWaiting + this.initialForegroundLeftWait = true } get done(): Promise { @@ -123,6 +120,11 @@ class LocalSendOperation implements PtySendOperation { return this.output.consume() } + setInitialForeground(foreground: SubprocessTerminalForeground | undefined): void { + this.initialForegroundPgid = foreground?.processGroupId + this.initialForegroundLeftWait = foreground?.inputWaiting !== true + } + acceptsStdinWait(pgid: number, waiting: boolean): boolean { // The same group may still expose the wait that existed before terminal.write. // Observe every poll so a departure before the exact-settlement threshold @@ -139,27 +141,21 @@ class LocalSendOperation implements PtySendOperation { } } -function signalName(number: number | undefined): NodeJS.Signals | null { - if (number === undefined || number === 0) return null - for (const [name, value] of Object.entries(constants.signals)) { - if (value === number) return name as NodeJS.Signals - } - return null -} - -/** Backend session wrapping one `node-pty` process and its captured process tree. */ +/** Backend session wrapping one provider-owned terminal process. */ export class LocalPtySession implements PtyBackendSession { motd = '' readonly pid: number + private readonly decoder = new TextDecoder('utf-8', { fatal: true }) private readonly sanitizer: TerminalSanitizer private readonly scrollback: BoundedTextBuffer - private readonly exitPromise: PromiseWithResolvers = Promise.withResolvers() - private readonly dataDisposable: IDisposable - private readonly exitDisposable: IDisposable + private readonly outputEnded = Promise.withResolvers() + private readonly completion: Promise private statusValue: PtySessionStatus = { kind: 'running' } private active: LocalSendOperation | undefined private activeTimer: NodeJS.Timeout | undefined + private activeDeadlineTimer: NodeJS.Timeout | undefined private activeAbort: (() => void) | undefined + private polling = false private promptSeen = false private promptTextSeen = false private shellPgid: number | undefined @@ -167,23 +163,22 @@ export class LocalPtySession implements PtyBackendSession { private lastOutputAt = Date.now() private closing = false private closePromise: Promise | undefined + private transportFailure: Error | undefined constructor( - private readonly terminal: IPty, - private readonly inspector: ProcessInspector, + private readonly terminal: SubprocessTerminalHandle, private readonly config: ResolvedConfig, ) { this.pid = terminal.pid this.sanitizer = new TerminalSanitizer(config.maxReadBytes) this.scrollback = new BoundedTextBuffer(config.scrollbackMaxBytes, config.scrollbackLines) - this.dataDisposable = terminal.onData((data) => { this.onData(data) }) - this.exitDisposable = terminal.onExit(({ exitCode, signal }) => { - const tail = this.sanitizer.flush() - this.appendOutput(tail) - this.statusValue = { kind: 'exited', exitCode, signal: signalName(signal) } - this.settleActive('session_exit') - this.exitPromise.resolve() - }) + terminal.output.on('data', this.onTerminalData) + terminal.output.once('end', this.onTerminalEnd) + terminal.output.once('error', this.onTerminalError) + this.completion = terminal.done.then( + outcome => this.onExit(outcome), + (error: unknown) => { this.onTransportFailure(error) }, + ) } /** @@ -213,14 +208,9 @@ export class LocalPtySession implements PtyBackendSession { if (this.active !== undefined) throw new Error('PTY session already has an active send') if (request.signal?.aborted === true) throw new Error('PTY send aborted before write') - const initialForegroundPgid = this.inspector.foregroundPgid(this.pid) - const initialForegroundWasWaiting = initialForegroundPgid !== undefined - && this.inspector.isStdinWaiting(initialForegroundPgid) const operation = new LocalSendOperation( this.config.maxReadBytes, Date.now(), - initialForegroundPgid, - initialForegroundWasWaiting, () => { this.interrupt(operation) }, ) this.active = operation @@ -233,20 +223,28 @@ export class LocalPtySession implements PtyBackendSession { request.signal.addEventListener('abort', onAbort, { once: true }) this.activeAbort = () => request.signal?.removeEventListener('abort', onAbort) } - - try { - if (request.text.length > 0) this.terminal.write(request.text) - if (request.submit) this.terminal.write('\r') - } catch (error: unknown) { - this.clearActive() - operation.fail(error) - return operation - } - - this.activeTimer = setInterval(() => { this.pollReadiness(operation) }, this.config.pollIntervalMs) + this.activeDeadlineTimer = setTimeout(() => { + if (this.active === operation) this.settleActive('timeout') + }, this.config.timeoutMs) + void this.beginSend(operation, request) return operation } + private async beginSend(operation: LocalSendOperation, request: PtySendRequest): Promise { + try { + const foreground = await this.terminal.inspectForeground() + if (this.active !== operation || this.closing) return + operation.setInitialForeground(foreground) + const input = `${request.text}${request.submit ? '\r' : ''}` + if (input.length > 0) await this.terminal.write(Buffer.from(input, 'utf8')) + // Closing can race the awaited provider write even though static analysis sees only local assignments. + // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition + if (this.active === operation && !this.closing) this.schedulePoll(operation, 0) + } catch (error: unknown) { + if (this.active === operation) this.failActive(error) + } + } + read(request: PtyReadRequest): PtyReadResult { const snapshot = this.scrollback.snapshot() const lines = snapshot.text.split('\n') @@ -272,16 +270,9 @@ export class LocalPtySession implements PtyBackendSession { } } - signal(signal: PtySignal): Promise { - return Promise.resolve().then(() => { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - if (signal === 'SIGKILL' && pgid === this.pid) { - throw new Error('refusing to SIGKILL the PTY shell; use terminal_close') - } - this.inspector.signalGroup(pgid, signal) - return { delivered: true, targetPgid: pgid } - }) + async signal(signal: PtySignal): Promise { + const targetPgid = await this.terminal.signalForeground(signal) + return { delivered: true, targetPgid } } status(): PtySessionStatus { @@ -300,12 +291,35 @@ export class LocalPtySession implements PtyBackendSession { return closing } + private readonly onTerminalData = (chunk: Buffer | Uint8Array | string): void => { + try { + const bytes = typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk + this.onData(this.decoder.decode(bytes, { stream: true })) + } catch (error: unknown) { + this.onTransportFailure(new Error('PTY emitted invalid UTF-8', { cause: error })) + } + } + + private readonly onTerminalEnd = (): void => { + try { + this.onData(this.decoder.decode()) + this.appendOutput(this.sanitizer.flush()) + } catch (error: unknown) { + this.onTransportFailure(new Error('PTY ended with invalid UTF-8', { cause: error })) + } finally { + this.outputEnded.resolve() + } + } + + private readonly onTerminalError = (error: Error): void => { + this.onTransportFailure(error) + this.outputEnded.resolve() + } + private onData(data: string): void { const sanitized = this.sanitizer.push(data) this.appendOutput(sanitized.text) if (sanitized.prompt) { - const foregroundPgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid === undefined) this.shellPgid = foregroundPgid // Bash can print PROMPT_COMMAND before the kernel publishes its return // to the foreground process group. Retain the marker; polling below is // the authority that accepts it only after bash owns the foreground. @@ -317,6 +331,21 @@ export class LocalPtySession implements PtyBackendSession { } } + private async onExit(outcome: SubprocessOutcome): Promise { + await this.outputEnded.promise + if (this.transportFailure !== undefined) return + this.statusValue = { kind: 'exited', exitCode: outcome.exitCode, signal: outcome.signal } + this.settleActive('session_exit') + } + + private onTransportFailure(error: unknown): void { + const failure = error instanceof Error ? error : new Error(String(error)) + this.transportFailure ??= failure + this.statusValue = { kind: 'exited', exitCode: null, signal: null } + this.failActive(failure) + this.terminal.terminate() + } + private appendOutput(text: string): void { if (text.length === 0) return this.lastOutputAt = Date.now() @@ -324,44 +353,56 @@ export class LocalPtySession implements PtyBackendSession { this.active?.append(text) } - private pollReadiness(operation: LocalSendOperation): void { - if (this.active !== operation) return - if (this.statusValue.kind === 'exited') { - this.settleActive('session_exit') - return - } - if (this.promptSeen && this.promptTextSeen && Date.now() - this.lastOutputAt >= this.config.pollIntervalMs) { - const pgid = this.inspector.foregroundPgid(this.pid) - if (this.shellPgid !== undefined && pgid === this.shellPgid) { + private schedulePoll(operation: LocalSendOperation, delayMs = this.config.pollIntervalMs): void { + if (this.active !== operation || this.polling) return + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) + this.activeTimer = setTimeout(() => { + this.activeTimer = undefined + void this.pollReadiness(operation) + }, delayMs) + } + + private async pollReadiness(operation: LocalSendOperation): Promise { + if (this.active !== operation || this.polling) return + this.polling = true + try { + if (this.statusValue.kind === 'exited') { + this.settleActive('session_exit') + return + } + const foreground = await this.terminal.inspectForeground() + if (this.active !== operation) return + const idleFor = Date.now() - this.lastOutputAt + if (this.promptSeen && foreground !== undefined && this.shellPgid === undefined) { + this.shellPgid = foreground.processGroupId + } + if (this.promptSeen && this.promptTextSeen && idleFor >= this.config.pollIntervalMs + && foreground?.processGroupId === this.shellPgid) { this.settleActive('stdin_read') return } + const elapsed = Date.now() - operation.startedAt + const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 + const acceptsStdinWait = startupHasOutput && foreground !== undefined + && operation.acceptsStdinWait(foreground.processGroupId, foreground.inputWaiting) + if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { + this.settleActive('stdin_read') + return + } + // A prompt candidate can race bash's foreground handoff, but an interactive + // child also inherits PROMPT_COMMAND. Silence therefore remains the bound + // on waiting for shell ownership instead of letting a child marker suppress + // readiness until the absolute timeout. + const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 + if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { + this.settleActive('inferred_idle') + } + } catch (error: unknown) { + if (this.active === operation) this.failActive(error) + } finally { + this.polling = false + if (this.active === operation) this.schedulePoll(operation) } - const elapsed = Date.now() - operation.startedAt - const startupHasOutput = !this.initializing || this.scrollback.snapshot().text.length > 0 - let acceptsStdinWait = false - if (startupHasOutput) { - const pgid = this.inspector.foregroundPgid(this.pid) - acceptsStdinWait = pgid !== undefined - && operation.acceptsStdinWait(pgid, this.inspector.isStdinWaiting(pgid)) - } - if (elapsed >= this.config.exactProbeAfterMs && acceptsStdinWait) { - this.settleActive('stdin_read') - return - } - // A prompt candidate can race bash's foreground handoff, but an interactive - // child also inherits PROMPT_COMMAND. Silence therefore remains the bound - // on waiting for shell ownership instead of letting a child marker suppress - // readiness until the absolute timeout. When a prompt marker was seen, the - // configured grace holds the fallback past the silence bound so polls in - // that window can observe the foreground handoff and settle as stdin_read. - const idleFor = Date.now() - this.lastOutputAt - const handoffGrace = this.promptSeen ? this.config.handoffGraceMs : 0 - if (startupHasOutput && idleFor >= this.config.idleSilenceMs + handoffGrace) { - this.settleActive('inferred_idle') - return - } - if (elapsed >= this.config.timeoutMs) this.settleActive('timeout') } private settleActive(waitReason: PtyWaitReason): void { @@ -373,8 +414,10 @@ export class LocalPtySession implements PtyBackendSession { } private stopPolling(): void { - if (this.activeTimer !== undefined) clearInterval(this.activeTimer) + if (this.activeTimer !== undefined) clearTimeout(this.activeTimer) this.activeTimer = undefined + if (this.activeDeadlineTimer !== undefined) clearTimeout(this.activeDeadlineTimer) + this.activeDeadlineTimer = undefined } private clearActive(): void { @@ -393,104 +436,30 @@ export class LocalPtySession implements PtyBackendSession { private interrupt(operation: LocalSendOperation): void { if (this.active !== operation) return - try { - const pgid = this.inspector.foregroundPgid(this.pid) - if (pgid === undefined) throw new Error(`cannot resolve foreground process group for PTY ${this.pid}`) - this.inspector.signalGroup(pgid, 'SIGINT') - } catch (error: unknown) { - this.failActive(error) - } - } - - private survivors(members: ProcessIdentity[]): ProcessIdentity[] { - return members.filter(member => this.inspector.isAlive(member)) - } - - private descendants(): ProcessIdentity[] { - return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) - } - - private async waitForExit(members: ProcessIdentity[]): Promise { - const deadline = Date.now() + this.config.disposeGraceMs - let survivors = this.survivors(members) - while (survivors.length > 0 && Date.now() < deadline) { - await delay(Math.min(25, Math.max(1, deadline - Date.now()))) - survivors = this.survivors(members) - } - return survivors - } - - private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { - for (const member of members) { - try { - this.inspector.signalProcess(member, signal) - } catch (_alreadyExitedDuringSignal) { - // Identity is rechecked by the inspector; a same-tick exit is success. - } - } - } - - private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { - const members: ProcessIdentity[] = [] - const seen = new Set() - for (const group of groups) { - for (const member of group) { - const key = JSON.stringify([member.pid, member.started]) - if (seen.has(key)) continue - seen.add(key) - members.push(member) - } - } - return members - } - - private async stopDescendants(): Promise { - const captured = this.descendants() - this.signalMembers(captured, 'SIGTERM') - const capturedSurvivors = await this.waitForExit(captured) - // A TERM-handling descendant may have forked while winding down. Rescan - // while the shell can still reap every member, then kill both the fresh - // tree and captured survivors that were reparented out of that tree. - const members = this.unionMembers(capturedSurvivors, this.descendants()) - this.signalMembers(members, 'SIGKILL') - const survivors = await this.waitForExit(members) - return this.survivors(this.unionMembers(survivors, this.descendants())) - } - - private async stopShell(): Promise { - try { - this.terminal.kill('SIGTERM') - } catch (_topLevelAlreadyExitedDuringTerm) { - // The exit notification remains authoritative. - } - if (this.statusValue.kind === 'running') { - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - try { - this.terminal.kill('SIGKILL') - } catch (_topLevelAlreadyExitedDuringKill) { - // The exit notification remains authoritative. - } - await Promise.race([this.exitPromise.promise, delay(this.config.disposeGraceMs)]) - } - if (this.statusValue.kind === 'running') { - throw new Error(`PTY cleanup failed; surviving pids: ${this.pid}`) - } + void this.terminal.signalForeground('SIGINT').catch((error: unknown) => { + if (this.active === operation) this.failActive(error) + }) } private async closeOnce(reason: string): Promise { - this.dataDisposable.dispose() // Stop readiness polling but retain the active operation: teardown settles // it as session_exit below, so an in-flight send is never mis-settled as // stdin_read/inferred_idle/timeout during the grace period. this.stopPolling() - const survivors = await this.stopDescendants() - if (survivors.length > 0) { - throw new Error(`PTY cleanup failed (${reason}); surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + this.terminal.terminate() + const quiescent = await this.terminal.waitForExit() + if (!quiescent) { + throw new Error(`PTY cleanup failed (${reason}); terminal session did not reach quiescence`) } - await this.stopShell() + // Whole-session cleanup can fail before the top-level process exits. Wait + // for it first so that failure is reported instead of blocking forever on + // `done`; successful quiescence guarantees `done` can now settle status and + // drain the terminal output. + await this.completion this.settleActive('session_exit') - this.exitDisposable.dispose() + this.terminal.output.off('data', this.onTerminalData) + this.terminal.output.off('end', this.onTerminalEnd) + this.terminal.output.off('error', this.onTerminalError) + if (this.transportFailure !== undefined) throw this.transportFailure } } diff --git a/packages/pty/pty-local/tests/index.spec.ts b/packages/pty/pty-local/tests/index.spec.ts index 1ad29d7d01..a8753deaf3 100644 --- a/packages/pty/pty-local/tests/index.spec.ts +++ b/packages/pty/pty-local/tests/index.spec.ts @@ -1,9 +1,9 @@ import { describe, expect, it, vi } from 'vitest' -import type { IPty, IPtyForkOptions } from 'node-pty' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import Loader from '@cordisjs/plugin-loader' import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' -import AgentRegistry, { Inbox, type Agent } from '@deepseek-ai/dsh-agent' +import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService, { setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' @@ -11,12 +11,18 @@ import PtyService, { PtyBackendCleanupError, PtySessionId } from '@deepseek-ai/d import { LocalPtyBackend } from '@deepseek-ai/dsh-pty-local' import * as ptyLocal from '@deepseek-ai/dsh-pty-local' import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-local/src/config.ts' -import type { ProcessInspector } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' import type { LocalPtySession } from '@deepseek-ai/dsh-pty-local/src/session.ts' +import { SubprocessService } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' class EmptySandbox extends SandboxProvider { confine(_argv: readonly string[], _policy: SandboxPolicy): ConfinedArgv { - return { argv: [], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } + return { argv: [], enforcement: 'full', denialSignatures: [], runnerFailureSignatures: [] } } } @@ -25,7 +31,7 @@ class RecordingSandbox extends SandboxProvider { confine(argv: readonly string[], policy: SandboxPolicy): ConfinedArgv { this.calls.push({ argv, policy }) - return { argv: ['/sandbox', '--', ...argv], enforcement: 'full', denialSignatures: [], runnerFailureRules: [] } + return { argv: ['/sandbox', '--', ...argv], enforcement: 'full', denialSignatures: [], runnerFailureSignatures: [] } } } @@ -38,28 +44,37 @@ function config(): ResolvedConfig { } } -function agent(ctx: Context, cwd?: string): Agent { +function agent(ctx: Context): Agent { const id = SessionId('agent') - const session = Session.create(id, undefined, { version: 0, id, createdAt: 0, ...cwd === undefined ? {} : { cwd } }) return { - id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), - status: 'idle', - ctx, - send: () => {}, - followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, - runMaintenance: task => task(new AbortController().signal), - whenIdle: () => Promise.resolve(), + id, options: {}, session: new Session(id), status: 'idle', acceptsNextStep: false, ctx, + followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), } } -const inspector = { - foregroundPgid: () => undefined, - isStdinWaiting: () => false, - processTree: () => [], - isAlive: () => false, - signalGroup() {}, - signalProcess() {}, -} satisfies ProcessInspector +function terminalHandle(): SubprocessTerminalHandle { + const output = new PassThrough() + return { + pid: 123, + output, + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), + signalForeground: async () => 123, + terminate: () => { output.end() }, + waitForExit: async () => true, + } +} + +class StubSubprocessService extends SubprocessService { + readonly cwd = '/tmp' + readonly runtimeRoot = '/tmp/dsh-runtime' + async resolveExecutable(command: string): Promise { return command } + spawn(_spec: SubprocessSpawnSpec): SubprocessHandle { throw new Error('unused') } + async spawnTerminal(_spec: SubprocessTerminalSpawnSpec): Promise { + return terminalHandle() + } +} function spec(owner: Agent, signal?: AbortSignal) { return { @@ -81,12 +96,11 @@ function stubLocalSession(initialize: () => Promise = () => Promise.resolv } function registerStubLocalBackend(ctx: Context, createSession: () => LocalPtySession) { - return ctx.inject(['pty', 'sandbox', 'sandboxPolicy'], (providerCtx) => { + return ctx.inject(['pty', 'sandbox', 'sandboxPolicy', 'subprocess'], (providerCtx) => { providerCtx.pty.registerBackend(new LocalPtyBackend( providerCtx, { ...config(), backendType: 'stub' }, - inspector, - (() => ({})) as never, + async () => terminalHandle(), createSession, )) }) @@ -97,7 +111,7 @@ describe('LocalPtyBackend startup rollback', () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/tmp' }) - const backend = new LocalPtyBackend(ctx, config(), inspector) + const backend = new LocalPtyBackend(ctx, config(), async () => terminalHandle()) const controller = new AbortController() const abortReason = new Error('spawn aborted') controller.abort(abortReason) @@ -109,11 +123,11 @@ describe('LocalPtyBackend startup rollback', () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) - const spawnTerminal = (() => ({} as IPty)) as never + const spawnTerminal = async (): Promise => terminalHandle() const closed = vi.fn<() => Promise>().mockResolvedValue(undefined) const failed = { initialize: () => Promise.reject(new Error('startup failed')), close: closed } as unknown as LocalPtySession - const backend = new LocalPtyBackend(ctx, config(), inspector, spawnTerminal, () => failed) + const backend = new LocalPtyBackend(ctx, config(), spawnTerminal, () => failed) await expect(backend.spawn(spec(agent(ctx)))).rejects.toThrow('startup failed') expect(closed).toHaveBeenCalledWith('PTY startup failed') @@ -123,7 +137,7 @@ describe('LocalPtyBackend startup rollback', () => { initialize: () => Promise.reject(startupFailure), close: () => Promise.reject(cleanupFailure), } as unknown as LocalPtySession - const aggregate = new LocalPtyBackend(ctx, config(), inspector, spawnTerminal, () => doublyFailed) + const aggregate = new LocalPtyBackend(ctx, config(), spawnTerminal, () => doublyFailed) await expect(aggregate.spawn(spec(agent(ctx)))).rejects.toEqual(expect.objectContaining({ name: 'PtyBackendCleanupError', spawnError: startupFailure, @@ -131,79 +145,72 @@ describe('LocalPtyBackend startup rollback', () => { } satisfies Partial)) }) - it('resolves session mode and root together before wrapping the shell', async () => { + it('wraps confined argv, scrubs the environment, and returns initialized sessions', async () => { const ctx = new Context() await ctx.plugin(RecordingSandbox) - await ctx.plugin(SandboxPolicyService, { mode: 'read-only', workspaceRoot: '/deployment-fallback' }) - const terminal = {} as IPty - let spawned: { file: string; args: string[]; options: IPtyForkOptions } | undefined - const spawnTerminal = ((file: string, args: string[], options: IPtyForkOptions) => { - spawned = { file, args, options } + await ctx.plugin(SandboxPolicyService, { mode: 'workspace-write', workspaceRoot: '/workspace' }) + const terminal = terminalHandle() + let spawned: SubprocessTerminalSpawnSpec | undefined + const spawnTerminal = async (spec: SubprocessTerminalSpawnSpec): Promise => { + spawned = spec return terminal - }) as never + } const initialized = vi.fn<() => Promise>().mockResolvedValue(undefined) const session = { initialize: initialized } as unknown as LocalPtySession const backend = new LocalPtyBackend( ctx, { ...config(), shellArgs: ['-i'] }, - inspector, spawnTerminal, () => session, ) const previous = process.env.PTY_TEST_SECRET process.env.PTY_TEST_SECRET = 'must-not-leak' - const owner = agent(ctx, '/session-workspace') - setSandboxMode(owner.session, 'workspace-write') try { - expect(await backend.spawn(spec(owner))).toBe(session) + expect(await backend.spawn({ ...spec(agent(ctx)), cwd: '/work' })).toBe(session) } finally { if (previous === undefined) delete process.env.PTY_TEST_SECRET else process.env.PTY_TEST_SECRET = previous } expect(spawned).toMatchObject({ - file: '/sandbox', - args: ['--', '/bin/bash', '-i'], - options: { - name: 'dumb', cols: 80, rows: 24, cwd: '/session-workspace', - env: { - TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', - DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', - }, + argv: ['/sandbox', '--', '/bin/bash', '-i'], + cols: 80, + rows: 24, + cwd: '/work', + graceMs: 10, + env: { + TERM: 'dumb', PAGER: 'cat', GIT_PAGER: 'cat', PS1: 'dsh> ', BASH_SILENCE_DEPRECATION_WARNING: '1', + DSH_SHELL: '1', DSH_SESSION_ID: 'agent', DSH_PTY_SESSION_ID: 'pty-1', }, }) - expect(spawned?.options.env?.PTY_TEST_SECRET).toBeUndefined() + expect(spawned?.env?.PTY_TEST_SECRET).toBeUndefined() expect(initialized).toHaveBeenCalledWith(undefined) - expect((ctx.sandbox as RecordingSandbox).calls).toEqual([{ - argv: ['/bin/bash', '-i'], - policy: { mode: 'workspace-write', workspaceRoot: '/session-workspace' }, - }]) }) it('composes the default local session around a spawned terminal', async () => { const ctx = new Context() await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/workspace' }) - let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined - const terminal = { - pid: 123, cols: 80, rows: 24, process: 'bash', handleFlowControl: false, - onData(listener: (data: string) => void) { - queueMicrotask(() => { listener('\x1b]133;D;0\x07dsh> ') }) - return { dispose() {} } + const output = new PassThrough() + const outcome = Promise.withResolvers<{ exitCode: number | null; signal: NodeJS.Signals | null }>() + const terminal: SubprocessTerminalHandle = { + pid: 123, + output, + done: outcome.promise, + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 123, inputWaiting: true }), + signalForeground: async () => 123, + terminate() { + output.end() + outcome.resolve({ exitCode: null, signal: 'SIGTERM' }) }, - onExit(listener: (event: { exitCode: number; signal?: number }) => void) { - exitListener = listener - return { dispose() {} } - }, - write() {}, - kill() { exitListener?.({ exitCode: 0, signal: 15 }) }, - resize() {}, clear() {}, pause() {}, resume() {}, - } as IPty + waitForExit: async () => true, + } + queueMicrotask(() => { output.write(Buffer.from('\x1b]133;D;0\x07dsh> ')) }) const backend = new LocalPtyBackend( ctx, config(), - { ...inspector, foregroundPgid: () => terminal.pid }, - () => terminal, + async () => terminal, ) const session = await backend.spawn(spec(agent(ctx))) expect(session.motd).toBe('dsh> ') @@ -217,7 +224,7 @@ describe('pty-local plugin shape', () => { const loader = Object.create(Loader.prototype) as Loader const unwrapped = loader.unwrapExports(ptyLocal) as Record expect(unwrapped.name).toBe('pty-local') - expect(unwrapped.inject).toEqual(['pty', 'sandbox', 'sandboxPolicy']) + expect(unwrapped.inject).toEqual(['pty', 'sandbox', 'sandboxPolicy', 'subprocess']) expect(unwrapped.Config).toBeDefined() }) @@ -227,6 +234,7 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const fiber = await ctx.plugin(ptyLocal, config()) expect(ctx.pty.listBackends()).toEqual(['shell']) await fiber.dispose() @@ -240,11 +248,12 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(EmptySandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) await ctx.plugin(ptyLocal, config()) const session = ctx.sessions.create(SessionId('unowned-mode')) expect(() => { - session.append('turn/start', { turn: 1 }) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) }).not.toThrow() expect(() => { setSandboxMode(session, 'read-only') }).not.toThrow() }) @@ -256,17 +265,13 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(RecordingSandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const session = ctx.sessions.create(SessionId('mode-owner')) const ownerFiber = await ctx.plugin(() => {}) const owner: Agent = { - id: session.id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), - status: 'idle', - ctx: ownerFiber.ctx, - send: () => {}, - followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, - runMaintenance: task => task(new AbortController().signal), - whenIdle: () => Promise.resolve(), + id: session.id, options: {}, session, status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, + followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), } ctx.agents.register(owner) const providerFiber = await registerStubLocalBackend(ctx, () => stubLocalSession()) @@ -275,7 +280,7 @@ describe('pty-local plugin shape', () => { const unrelated = ctx.sessions.create(SessionId('unrelated-mode')) expect(() => { setSandboxMode(unrelated, 'read-only') }).not.toThrow() expect(() => { - session.append('turn/start', { turn: 1 }) + session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }) }).not.toThrow() expect(() => { setSandboxMode(session, 'danger-full-access') }).not.toThrow() @@ -304,17 +309,13 @@ describe('pty-local plugin shape', () => { await ctx.plugin(PtyService) await ctx.plugin(RecordingSandbox) await ctx.plugin(SandboxPolicyService, { mode: 'danger-full-access', workspaceRoot: '/tmp' }) + await ctx.plugin(StubSubprocessService) const session = ctx.sessions.create(SessionId('pending-mode-owner')) const ownerFiber = await ctx.plugin(() => {}) const owner: Agent = { - id: session.id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), - status: 'idle', - ctx: ownerFiber.ctx, - send: () => {}, - followup: () => {}, steer: () => {}, inject: () => {}, cancel() {}, - runMaintenance: task => task(new AbortController().signal), - whenIdle: () => Promise.resolve(), + id: session.id, options: {}, session, status: 'idle', acceptsNextStep: false, ctx: ownerFiber.ctx, + followup: () => {}, steer: () => {}, inject: () => {}, send: () => {}, cancel() {}, whenIdle: () => Promise.resolve(), } ctx.agents.register(owner) const gate = Promise.withResolvers() diff --git a/packages/pty/pty-local/tests/local.spec.ts b/packages/pty/pty-local/tests/local.spec.ts index c57b0ebb8e..c02a9563ea 100644 --- a/packages/pty/pty-local/tests/local.spec.ts +++ b/packages/pty/pty-local/tests/local.spec.ts @@ -11,6 +11,7 @@ import type { PtySendOperation } from '@deepseek-ai/dsh-pty' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as ptyLocal from '@deepseek-ai/dsh-pty-local' const roots: string[] = [] @@ -57,6 +58,7 @@ async function harness( await ctx.plugin(PtyService) await ctx.plugin(PassthroughSandbox) await ctx.plugin(SandboxPolicyService, { mode, workspaceRoot: root }) + await ctx.plugin(LocalSubprocessService) const fiber = await ctx.plugin(ptyLocal, { pollIntervalMs: 10, exactProbeAfterMs: 20, diff --git a/packages/pty/pty-local/tests/session.spec.ts b/packages/pty/pty-local/tests/session.spec.ts index 53ce61eebc..7a775eb9a0 100644 --- a/packages/pty/pty-local/tests/session.spec.ts +++ b/packages/pty/pty-local/tests/session.spec.ts @@ -1,62 +1,17 @@ import { afterEach, describe, expect, it, vi } from 'vitest' -import type { IDisposable, IPty } from 'node-pty' +import { PassThrough } from 'node:stream' import { LocalPtySession } from '@deepseek-ai/dsh-pty-local/src/session.ts' import type { ResolvedConfig } from '@deepseek-ai/dsh-pty-local/src/config.ts' -import type { ProcessIdentity, ProcessInspector } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' import type { PtySendOperation, PtySessionStatus, PtySignal } from '@deepseek-ai/dsh-pty' - -class FakeTerminal { - pid = 123 - cols = 80 - rows = 24 - process = 'bash' - handleFlowControl = false - writes: string[] = [] - kills: string[] = [] - throwWrite = false - throwKill = false - autoExitOnKill = true - private dataListeners = new Set<(data: string) => void>() - private exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() - - readonly onData = (listener: (data: string) => void): IDisposable => { - this.dataListeners.add(listener) - return { dispose: () => this.dataListeners.delete(listener) } - } - - readonly onExit = (listener: (event: { exitCode: number; signal?: number }) => void): IDisposable => { - this.exitListeners.add(listener) - return { dispose: () => this.exitListeners.delete(listener) } - } - - emitData(data: string): void { - for (const listener of this.dataListeners) listener(data) - } - - emitExit(exitCode = 0, signal?: number): void { - for (const listener of this.exitListeners) listener({ exitCode, ...signal === undefined ? {} : { signal } }) - } - - write(data: string): void { - if (this.throwWrite) throw new Error('write failed') - this.writes.push(data) - } - - kill(signal?: string): void { - if (this.throwKill) throw new Error('kill failed') - this.kills.push(signal ?? 'SIGHUP') - if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) - } - - resize() {} - clear() {} - pause() {} - resume() {} - - asPty(): IPty { - return this - } -} +import type { + SubprocessOutcome, + SubprocessTerminalHandle, + SubprocessTerminalSignal, +} from '@deepseek-ai/dsh-subprocess' +import type { + ProcessIdentity, + ProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' class FakeInspector implements ProcessInspector { pgid: number | undefined = 456 @@ -84,6 +39,89 @@ class FakeInspector implements ProcessInspector { } } +class FakeTerminal implements SubprocessTerminalHandle { + pid = 123 + readonly output = new PassThrough() + readonly writes: string[] = [] + readonly kills: string[] = [] + readonly outcome = Promise.withResolvers() + readonly done = this.outcome.promise + throwWrite = false + throwKill = false + autoExitOnKill = true + quiescent = true + waitError: Error | undefined + + constructor(public inspector = new FakeInspector()) {} + + emitData(data: string): void { + this.output.write(Buffer.from(data, 'utf8')) + } + + emitBytes(data: Uint8Array): void { + this.output.write(data) + } + + emitError(error: Error): void { + this.output.emit('error', error) + } + + emitFailure(error: unknown): void { + this.output.end() + this.outcome.reject(error) + } + + emitExit(exitCode = 0, signal?: number): void { + this.output.end() + this.outcome.resolve({ + exitCode: signal === undefined || signal === 0 ? exitCode : null, + signal: signal === 9 ? 'SIGKILL' : signal === 15 ? 'SIGTERM' : null, + }) + } + + async write(data: Uint8Array): Promise { + if (this.throwWrite) throw new Error('write failed') + this.writes.push(Buffer.from(data).toString('utf8')) + } + + async inspectForeground() { + const processGroupId = this.inspector.foregroundPgid() + return processGroupId === undefined + ? undefined + : { processGroupId, inputWaiting: this.inspector.isStdinWaiting() } + } + + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) throw new Error(`cannot resolve foreground process group for terminal ${this.pid}`) + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + this.inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + + terminate(): void { + if (this.throwKill) throw new Error('kill failed') + this.kills.push('SIGTERM') + if (this.autoExitOnKill) this.emitExit(0, 15) + } + + async waitForExit(): Promise { + if (this.waitError !== undefined) throw this.waitError + return this.quiescent + } +} + +function makeSession( + terminal: FakeTerminal, + inspector: FakeInspector, + resolved: ResolvedConfig, +): LocalPtySession { + terminal.inspector = inspector + return new LocalPtySession(terminal, resolved) +} + function config(overrides: Partial = {}): ResolvedConfig { return { backendType: 'shell', shellPath: '/bin/bash', shellArgs: [], rows: 24, cols: 80, @@ -108,13 +146,15 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) expect(session.motd).toBe('dsh> ') inspector.waiting = true const operation = session.startSend({ text: 'python3', submit: true }) - expect(terminal.writes).toEqual(['python3', '\r']) + await Promise.resolve() + await Promise.resolve() + expect(terminal.writes).toEqual(['python3\r']) inspector.pgid = 789 terminal.emitData('Python\r\n>>> ') await vi.advanceTimersByTimeAsync(20) @@ -126,7 +166,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) inspector.waiting = true @@ -148,7 +188,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config({ + const session = makeSession(terminal, inspector, config({ exactProbeAfterMs: 50, idleSilenceMs: 100, timeoutMs: 200, @@ -173,7 +213,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) inspector.pgid = undefined @@ -193,7 +233,7 @@ describe('LocalPtySession readiness and output', () => { const exiting = session.startSend({ text: 'exit', submit: true }) terminal.emitExit(7, 9) - expect(await exiting.done).toMatchObject({ waitReason: 'session_exit', sessionStatus: { kind: 'exited', exitCode: 7, signal: 'SIGKILL' } }) + expect(await exiting.done).toMatchObject({ waitReason: 'session_exit', sessionStatus: { kind: 'exited', exitCode: null, signal: 'SIGKILL' } }) expect(() => session.startSend({ text: '', submit: false })).toThrow('has exited') }) @@ -201,13 +241,15 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const controller = new AbortController() const operation = session.startSend({ text: 'sleep', submit: true, signal: controller.signal }) expect(() => session.startSend({ text: 'again', submit: true })).toThrow('active send') controller.abort() + await Promise.resolve() + await Promise.resolve() expect(inspector.groups).toContainEqual([456, 'SIGINT']) expect(terminal.writes).not.toContain('\x03') terminal.emitData('\x1b]133;D;130\x07dsh> ') @@ -229,14 +271,14 @@ describe('LocalPtySession readiness and output', () => { it('handles startup exit, unknown exit signals, cancel-write failure, and stale polls', async () => { vi.useFakeTimers() const startupTerminal = new FakeTerminal() - const startup = new LocalPtySession(startupTerminal.asPty(), new FakeInspector(), config()) + const startup = new LocalPtySession(startupTerminal, config()) const initializing = startup.initialize(new AbortController().signal) startupTerminal.emitExit(1) await expect(initializing).rejects.toThrow('exited during startup') expect(startup.status()).toEqual({ kind: 'exited', exitCode: 1, signal: null }) const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) await initialize(session, terminal) const operation = session.startSend({ text: '', submit: false }) const operationInternal = operation as unknown as { @@ -247,11 +289,17 @@ describe('LocalPtySession readiness and output', () => { const sessionInternal = session as unknown as { pollReadiness(operation: PtySendOperation): void interrupt(operation: PtySendOperation): void + schedulePoll(operation: PtySendOperation): void + polling: boolean statusValue: PtySessionStatus appendOutput(text: string): void } sessionInternal.appendOutput('') sessionInternal.pollReadiness({} as PtySendOperation) + sessionInternal.schedulePoll({} as PtySendOperation) + sessionInternal.polling = true + sessionInternal.schedulePoll(operation) + sessionInternal.polling = false sessionInternal.interrupt({} as PtySendOperation) sessionInternal.statusValue = { kind: 'exited', exitCode: 2, signal: null } sessionInternal.pollReadiness(operation) @@ -259,13 +307,15 @@ describe('LocalPtySession readiness and output', () => { operationInternal.settle('timeout', { kind: 'running' }, false) const unknownTerminal = new FakeTerminal() - const unknown = new LocalPtySession(unknownTerminal.asPty(), new FakeInspector(), config()) + const unknown = new LocalPtySession(unknownTerminal, config()) unknownTerminal.emitExit(1, 999) - expect(unknown.status()).toEqual({ kind: 'exited', exitCode: 1, signal: null }) + await vi.waitFor(() => { + expect(unknown.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + }) const cancelTerminal = new FakeTerminal() const cancelInspector = new FakeInspector() - const cancel = new LocalPtySession(cancelTerminal.asPty(), cancelInspector, config()) + const cancel = makeSession(cancelTerminal, cancelInspector, config()) await initialize(cancel, cancelTerminal) const cancellable = cancel.startSend({ text: '', submit: false }) cancelInspector.throwGroup = true @@ -275,7 +325,7 @@ describe('LocalPtySession readiness and output', () => { const missingGroupTerminal = new FakeTerminal() const missingGroupInspector = new FakeInspector() - const missingGroup = new LocalPtySession(missingGroupTerminal.asPty(), missingGroupInspector, config()) + const missingGroup = makeSession(missingGroupTerminal, missingGroupInspector, config()) await initialize(missingGroup, missingGroupTerminal) missingGroupInspector.pgid = undefined const unresolved = missingGroup.startSend({ text: '', submit: false }) @@ -286,7 +336,7 @@ describe('LocalPtySession readiness and output', () => { it('does not treat zero-output startup silence as readiness and fails on startup timeout', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) let settled = false const initializing = session.initialize().then(() => { settled = true }) await vi.advanceTimersByTimeAsync(60) @@ -296,7 +346,7 @@ describe('LocalPtySession readiness and output', () => { await initializing const timeoutTerminal = new FakeTerminal() - const timeout = new LocalPtySession(timeoutTerminal.asPty(), new FakeInspector(), config()) + const timeout = new LocalPtySession(timeoutTerminal, config()) const timedOut = expect(timeout.initialize()).rejects.toThrow('startup timeout') await vi.advanceTimersByTimeAsync(100) await timedOut @@ -306,7 +356,7 @@ describe('LocalPtySession readiness and output', () => { const terminal = new FakeTerminal() const inspector = new FakeInspector() inspector.pgid = undefined - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) const controller = new AbortController() const reason = new Error('startup cancelled') @@ -320,7 +370,7 @@ describe('LocalPtySession readiness and output', () => { it('waits for printable prompt text when the startup marker is split from PS1', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config()) + const session = new LocalPtySession(terminal, config()) let settled = false const initializing = session.initialize().then(() => { settled = true }) @@ -338,7 +388,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) @@ -359,7 +409,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config({ handoffGraceMs: 40 })) + const session = makeSession(terminal, inspector, config({ handoffGraceMs: 40 })) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) @@ -380,7 +430,7 @@ describe('LocalPtySession readiness and output', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) await initialize(session, terminal) const operation = session.startSend({ text: 'bash -i', submit: true }) @@ -390,6 +440,156 @@ describe('LocalPtySession readiness and output', () => { expect((await operation.done).waitReason).toBe('inferred_idle') }) + + it('contains terminal transport failures and preserves the first failure', async () => { + const terminal = new FakeTerminal() + const session = new LocalPtySession(terminal, config()) + const operation = session.startSend({ text: '', submit: false }) + terminal.output.emit('data', 'plain text') + terminal.emitError(new Error('output transport failed')) + ;(session as unknown as { onTransportFailure(error: unknown): void }) + .onTransportFailure(new Error('later failure')) + await expect(operation.done).rejects.toThrow('output transport failed') + expect(session.status()).toEqual({ kind: 'exited', exitCode: null, signal: null }) + await expect(session.close('transport')).rejects.toThrow('output transport failed') + + const rejectedTerminal = new FakeTerminal() + const rejected = new LocalPtySession(rejectedTerminal, config()) + const rejectedOperation = rejected.startSend({ text: '', submit: false }) + rejectedTerminal.emitFailure('raw transport failure') + await expect(rejectedOperation.done).rejects.toThrow('raw transport failure') + }) + + it('rejects invalid UTF-8 in a data chunk and at stream end', async () => { + const chunkTerminal = new FakeTerminal() + const chunkSession = new LocalPtySession(chunkTerminal, config()) + const chunkOperation = chunkSession.startSend({ text: '', submit: false }) + chunkTerminal.emitBytes(Uint8Array.from([0xff])) + await expect(chunkOperation.done).rejects.toThrow('PTY emitted invalid UTF-8') + + const endTerminal = new FakeTerminal() + const endSession = new LocalPtySession(endTerminal, config()) + const endOperation = endSession.startSend({ text: '', submit: false }) + endTerminal.emitBytes(Uint8Array.from([0xe2])) + endTerminal.emitExit() + await expect(endOperation.done).rejects.toThrow('PTY ended with invalid UTF-8') + }) + + it('contains readiness inspection failure and a stale inspection result', async () => { + vi.useFakeTimers() + const failedTerminal = new FakeTerminal() + const failedSession = new LocalPtySession(failedTerminal, config()) + const failedOperation = failedSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + failedTerminal.inspectForeground = async () => { throw new Error('inspect failed') } + const failedInternal = failedSession as unknown as { + pollReadiness(operation: PtySendOperation): Promise + } + await failedInternal.pollReadiness(failedOperation) + await expect(failedOperation.done).rejects.toThrow('inspect failed') + + const staleTerminal = new FakeTerminal() + const staleSession = new LocalPtySession(staleTerminal, config()) + const staleOperation = staleSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const gate = Promise.withResolvers extends Promise ? T : never>() + staleTerminal.inspectForeground = async () => await gate.promise + const staleInternal = staleSession as unknown as { + active: PtySendOperation | undefined + pollReadiness(operation: PtySendOperation): Promise + } + const polling = staleInternal.pollReadiness(staleOperation) + staleInternal.active = undefined + gate.resolve({ processGroupId: 456, inputWaiting: false }) + await polling + ;(staleOperation as unknown as { + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + }).settle('timeout', { kind: 'running' }, false) + }) + + it('contains stale timer, write, inspection, and interrupt continuations', async () => { + vi.useFakeTimers() + const settle = (operation: PtySendOperation): void => { + ;(operation as unknown as { + settle(reason: 'timeout', status: PtySessionStatus, inherited: boolean): void + }).settle('timeout', { kind: 'running' }, false) + } + + const deadlineTerminal = new FakeTerminal() + const deadlineSession = new LocalPtySession(deadlineTerminal, config()) + const deadlineOperation = deadlineSession.startSend({ text: '', submit: false }) + ;(deadlineSession as unknown as { active: PtySendOperation | undefined }).active = undefined + await vi.advanceTimersByTimeAsync(100) + settle(deadlineOperation) + + const writeTerminal = new FakeTerminal() + const writeGate = Promise.withResolvers() + writeTerminal.write = async () => { await writeGate.promise } + const writeSession = new LocalPtySession(writeTerminal, config()) + const writeOperation = writeSession.startSend({ text: 'x', submit: false }) + await Promise.resolve() + await Promise.resolve() + ;(writeSession as unknown as { closing: boolean }).closing = true + writeGate.resolve(undefined) + await Promise.resolve() + await Promise.resolve() + settle(writeOperation) + + const beginTerminal = new FakeTerminal() + const beginGate = Promise.withResolvers() + beginTerminal.inspectForeground = async () => await beginGate.promise + const beginSession = new LocalPtySession(beginTerminal, config()) + const beginOperation = beginSession.startSend({ text: '', submit: false }) + ;(beginSession as unknown as { active: PtySendOperation | undefined }).active = undefined + beginGate.reject(new Error('stale begin failure')) + await Promise.resolve() + await Promise.resolve() + settle(beginOperation) + + const scheduledTerminal = new FakeTerminal() + const scheduledSession = new LocalPtySession(scheduledTerminal, config()) + const scheduledOperation = scheduledSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const scheduledInternal = scheduledSession as unknown as { + schedulePoll(operation: PtySendOperation, delayMs?: number): void + settleActive(reason: 'timeout'): void + } + scheduledInternal.schedulePoll(scheduledOperation, 5) + scheduledInternal.settleActive('timeout') + await scheduledOperation.done + + const pollTerminal = new FakeTerminal() + const pollSession = new LocalPtySession(pollTerminal, config()) + const pollOperation = pollSession.startSend({ text: '', submit: false }) + await Promise.resolve() + await Promise.resolve() + const pollGate = Promise.withResolvers() + pollTerminal.inspectForeground = async () => await pollGate.promise + const pollInternal = pollSession as unknown as { + active: PtySendOperation | undefined + pollReadiness(operation: PtySendOperation): Promise + } + const stalePoll = pollInternal.pollReadiness(pollOperation) + pollInternal.active = undefined + pollGate.reject(new Error('stale poll failure')) + await stalePoll + settle(pollOperation) + + const interruptTerminal = new FakeTerminal() + const interruptGate = Promise.withResolvers() + interruptTerminal.signalForeground = async () => await interruptGate.promise + const interruptSession = new LocalPtySession(interruptTerminal, config()) + const interruptOperation = interruptSession.startSend({ text: '', submit: false }) + expect(interruptOperation.cancel()).toBe(true) + ;(interruptSession as unknown as { active: PtySendOperation | undefined }).active = undefined + interruptGate.reject(new Error('stale interrupt failure')) + await Promise.resolve() + await Promise.resolve() + settle(interruptOperation) + }) }) describe('LocalPtySession bounds, signals, and teardown', () => { @@ -397,8 +597,7 @@ describe('LocalPtySession bounds, signals, and teardown', () => { vi.useFakeTimers() const terminal = new FakeTerminal() const session = new LocalPtySession( - terminal.asPty(), - new FakeInspector(), + terminal, config({ scrollbackLines: 3, scrollbackMaxBytes: 12, maxReadBytes: 6 }), ) expect(session.read({})).toMatchObject({ text: '' }) @@ -415,7 +614,7 @@ describe('LocalPtySession bounds, signals, and teardown', () => { expect(() => session.read({ count: 0 })).toThrow('count') const tinyTerminal = new FakeTerminal() - const tiny = new LocalPtySession(tinyTerminal.asPty(), new FakeInspector(), config({ maxReadBytes: 1 })) + const tiny = new LocalPtySession(tinyTerminal, config({ maxReadBytes: 1 })) await initialize(tiny, tinyTerminal) const tinyOperation = tiny.startSend({ text: '', submit: false }) tinyTerminal.emitData('一') @@ -427,32 +626,38 @@ describe('LocalPtySession bounds, signals, and teardown', () => { it('signals verified groups and refuses unresolved or shell-targeted hard kills', async () => { const terminal = new FakeTerminal() const inspector = new FakeInspector() - const session = new LocalPtySession(terminal.asPty(), inspector, config()) + const session = makeSession(terminal, inspector, config()) expect(await session.signal('SIGINT')).toEqual({ delivered: true, targetPgid: 456 }) inspector.pgid = terminal.pid - await expect(session.signal('SIGKILL')).rejects.toThrow('use terminal_close') + await expect(session.signal('SIGKILL')).rejects.toThrow('terminate the terminal session') inspector.pgid = undefined await expect(session.signal('SIGTERM')).rejects.toThrow('cannot resolve') }) it('closes idempotently, contains signal races, and reports survivors', async () => { const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 123, started: 'a' }] - inspector.alive.add(123) - inspector.throwProcess = true - terminal.throwKill = true - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 1 })) + terminal.quiescent = false + const session = new LocalPtySession(terminal, config({ disposeGraceMs: 1 })) const closing = session.close('test') expect(session.close('other')).toBe(closing) - await expect(closing).rejects.toThrow('surviving pids: 123') + await expect(closing).rejects.toThrow('did not reach quiescence') expect(() => session.startSend({ text: '', submit: false })).toThrow('closing') }) + it('reports cleanup failure without waiting for top-level exit', async () => { + const terminal = new FakeTerminal() + terminal.autoExitOnKill = false + terminal.waitError = new Error('terminal cleanup failed; surviving pids: 456') + const session = new LocalPtySession(terminal, config()) + + await expect(session.close('survivor')).rejects.toThrow('surviving pids: 456') + expect(terminal.kills).toEqual(['SIGTERM']) + }) + it('settles an active send as session_exit when closed mid-operation', async () => { vi.useFakeTimers() const terminal = new FakeTerminal() - const session = new LocalPtySession(terminal.asPty(), new FakeInspector(), config({ disposeGraceMs: 50 })) + const session = new LocalPtySession(terminal, config({ disposeGraceMs: 50 })) await initialize(session, terminal) const operation = session.startSend({ text: 'run', submit: true }) // The shell returns to its prompt while the send is active; a running @@ -467,94 +672,4 @@ describe('LocalPtySession bounds, signals, and teardown', () => { await closing }) - it('keeps the shell alive until SIGKILL recipients leave the process table', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 20 })) - - let settled = false - const closing = session.close('test').then(() => { settled = true }) - await vi.advanceTimersByTimeAsync(20) - expect(inspector.processes).toContainEqual([124, 'SIGKILL']) - expect(terminal.kills).toEqual([]) - expect(settled).toBe(false) - - inspector.alive.delete(124) - await vi.advanceTimersByTimeAsync(20) - await closing - expect(terminal.kills).toEqual(['SIGTERM']) - expect(settled).toBe(true) - }) - - it('rescans for descendants forked during TERM before stopping the shell', async () => { - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - let reads = 0 - inspector.processTree = () => { - reads += 1 - if (reads === 1) { - inspector.alive.add(124) - return [{ pid: 124, started: 'first' }] - } - if (reads === 2) { - inspector.alive.add(125) - return [{ pid: 125, started: 'late' }] - } - return [] - } - const session = new LocalPtySession(terminal.asPty(), inspector, config()) - - await session.close('test') - - expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) - expect(terminal.kills).toEqual(['SIGTERM']) - }) - - it('retains captured survivors that are reparented out of the teardown rescan', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - const captured = { pid: 124, started: 'captured' } - let reads = 0 - inspector.alive.add(captured.pid) - inspector.processTree = () => reads++ === 0 ? [captured] : [] - inspector.signalProcess = (identity, signal) => { - inspector.processes.push([identity.pid, signal]) - if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) - } - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 20 })) - - const closing = session.close('test') - await vi.advanceTimersByTimeAsync(25) - await closing - - expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) - expect(terminal.kills).toEqual(['SIGTERM']) - }) - - it('allows teardown to retry after a descendant-survivor failure', async () => { - vi.useFakeTimers() - const terminal = new FakeTerminal() - const inspector = new FakeInspector() - inspector.members = [{ pid: 124, started: 'child' }] - inspector.alive.add(124) - inspector.removeOnSignal = false - const session = new LocalPtySession(terminal.asPty(), inspector, config({ disposeGraceMs: 10 })) - - const first = session.close('first') - const rejected = expect(first).rejects.toThrow('surviving pids: 124') - await vi.advanceTimersByTimeAsync(25) - await rejected - expect(terminal.kills).toEqual([]) - - inspector.alive.delete(124) - const second = session.close('retry') - expect(second).not.toBe(first) - await second - expect(terminal.kills).toEqual(['SIGTERM']) - }) }) diff --git a/packages/pty/tool-pty/package.json b/packages/pty/tool-pty/package.json index 5512daba92..9027169941 100644 --- a/packages/pty/tool-pty/package.json +++ b/packages/pty/tool-pty/package.json @@ -51,6 +51,7 @@ "@deepseek-ai/dsh-sandbox-policy": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-system-prompt": "workspace:^", + "@deepseek-ai/dsh-subprocess-local": "workspace:^", "@deepseek-ai/dsh-tasks": "workspace:^", "@deepseek-ai/dsh-tasks-local": "workspace:^", "@deepseek-ai/dsh-tool-tasks": "workspace:^", diff --git a/packages/pty/tool-pty/tests/loader-composition.spec.ts b/packages/pty/tool-pty/tests/loader-composition.spec.ts index 835df86abc..35d0ea5d0a 100644 --- a/packages/pty/tool-pty/tests/loader-composition.spec.ts +++ b/packages/pty/tool-pty/tests/loader-composition.spec.ts @@ -16,6 +16,7 @@ import PtyService from '@deepseek-ai/dsh-pty' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' import SandboxPolicyService from '@deepseek-ai/dsh-sandbox-policy' +import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' import * as PtyLocal from '@deepseek-ai/dsh-pty-local' import * as ToolPty from '@deepseek-ai/dsh-tool-pty' @@ -72,6 +73,7 @@ suite('terminal real Loader composition through cordis.yml', () => { ' config:', ' mode: danger-full-access', ` workspaceRoot: ${JSON.stringify(root)}`, + "- name: '@deepseek-ai/dsh-subprocess-local'", "- name: '@deepseek-ai/dsh-pty-local'", ' config:', ' pollIntervalMs: 10', @@ -95,6 +97,7 @@ suite('terminal real Loader composition through cordis.yml', () => { ['@deepseek-ai/dsh-pty', PtyService], ['@deepseek-ai/dsh-test-sandbox', PassthroughSandbox], ['@deepseek-ai/dsh-sandbox-policy', SandboxPolicyService], + ['@deepseek-ai/dsh-subprocess-local', LocalSubprocessService], ['@deepseek-ai/dsh-pty-local', PtyLocal], ['@deepseek-ai/dsh-tool-pty', ToolPty], ]) diff --git a/packages/skill/skill-local/tests/skill-local.spec.ts b/packages/skill/skill-local/tests/skill-local.spec.ts index 0f92cac1d6..9dc723fb58 100644 --- a/packages/skill/skill-local/tests/skill-local.spec.ts +++ b/packages/skill/skill-local/tests/skill-local.spec.ts @@ -42,6 +42,14 @@ class TestFileSystem extends FileSystem { return { targetKey: path as never, displayPath: path } } + override processPath(target: FsTarget): string { return String(target.targetKey) } + + override fileUrl(target: FsTarget): string { return `file://${target.targetKey}` } + + override contains(parent: FsTarget, child: FsTarget): boolean { + return child.targetKey === parent.targetKey || String(child.targetKey).startsWith(`${parent.targetKey}/`) + } + override async stat(target: FsTarget, signal?: AbortSignal): Promise { this.statSignals.push(signal) if (this.failStatPaths.has(target.displayPath)) throw new FsError('stat failed', 'FS_NOT_FOUND') @@ -84,6 +92,12 @@ class TestFileSystem extends FileSystem { return text } + override async readTextBounded(target: FsTarget, maxBytes: number, signal?: AbortSignal): Promise { + const text = await this.readText(target, signal) + if (Buffer.byteLength(text) > maxBytes) throw new Error('too large') + return text + } + override async streamText(_target: FsTarget): Promise> { throw new Error('not needed in skill tests') } diff --git a/packages/subprocess/README.i18n.yaml b/packages/subprocess/README.i18n.yaml index 9c0cb0c25d..256d4ad2d9 100644 --- a/packages/subprocess/README.i18n.yaml +++ b/packages/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/README.md -README.md: 187ea5b778a4bc1f9c3c9121adb18bda11cc7b58 -README.zh.md: dd3a975daec014131877d7b1523810bd932619d8 +README.md: c4bb1da1a172b05afa63834db4e5b1fa974baabb +README.zh.md: f68bc1c720eeb5282bb9c94c951524a021e38b2f diff --git a/packages/subprocess/README.md b/packages/subprocess/README.md index 187ea5b778..c4bb1da1a1 100644 --- a/packages/subprocess/README.md +++ b/packages/subprocess/README.md @@ -2,11 +2,11 @@ English | [中文](README.zh.md) -This family runs host subprocesses behind an explicit process-lifecycle service. +The shared process substrate for one execution world: canonical cwd/runtime storage, executable lookup, fully-specified managed child-process trees with raw or collected stdio, and one deep terminal-process primitive that owns PTY allocation, foreground groups, and complete session cleanup. Command defaulting, shell semantics, deadlines, protocol framing, readiness, and presentation stay with consumers — the [bash executors](../bash/README.md), [LSP host](../lsp/README.md), [PTY shell backend](../pty/README.md), [subprocess code runtime](../code-runtime/code-runtime-subprocess/README.md), and [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). -| Package | Role | ctx key | +| Package | ctx key | Role | |---|---|---| -| [`subprocess/`](subprocess/README.md) | Defines subprocess launch, stream, termination, and disposal contracts | `ctx.subprocess` | -| [`subprocess-local/`](subprocess-local/README.md) | Implements local process-tree execution | registers on `ctx.subprocess` | +| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: execution-world coordinates and executable lookup, ordinary managed spawns, the terminal-process primitive, handle lifecycles, and shared environment/output vocabulary | +| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, bounded collection/spill, `node-pty`, foreground/session inspection, tree signalling, runtime storage, and terminate-and-join disposal | -The service owns process lifetime; each consumer owns what the process does and which defaults apply. +The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one. diff --git a/packages/subprocess/README.zh.md b/packages/subprocess/README.zh.md index dd3a975dae..f68bc1c720 100644 --- a/packages/subprocess/README.zh.md +++ b/packages/subprocess/README.zh.md @@ -1,12 +1,12 @@ -# subprocess/：子进程能力家族 +# subprocess/：进程管理能力家族 [English](README.md) | 中文 -本家族通过显式的进程生命周期服务运行宿主子进程。 +同一执行世界中的共享进程基底：规范化 cwd／运行时存储、可执行文件查找、采用原始或收集式 stdio 的完全显式受管子进程树，以及一项负责 PTY 分配、前台进程组和完整会话清理的深层终端进程原语。命令默认值补全、shell 语义、deadline、协议分帧、就绪检测与呈现留在消费方：[bash 执行器](../bash/README.md)、[LSP 主机](../lsp/README.md)、[PTY shell 后端](../pty/README.md)、[基于进程管理的 Code Runtime](../code-runtime/code-runtime-subprocess/README.md)与 [ACP（Agent Client Protocol）subagent 后端](../subagent/README.md)。参见[进程管理器 seam Agent Note（agent 决策记录）](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 -| 包 | 职责 | ctx 键 | +| 包（package） | ctx 键 | 角色 | |---|---|---| -| [`subprocess/`](subprocess/README.md) | 定义子进程启动、流、终止和 dispose（资源释放）契约 | `ctx.subprocess` | -| [`subprocess-local/`](subprocess-local/README.md) | 实现本地进程树执行 | 注册到 `ctx.subprocess` | +| [`subprocess`](subprocess/README.md)（`@deepseek-ai/dsh-subprocess`） | `ctx.subprocess` | seam 本体：执行世界坐标与可执行文件查找、普通受管 spawn、终端进程原语、句柄生命周期，以及共享的环境／输出词汇 | +| [`subprocess-local`](subprocess-local/README.md)（`@deepseek-ai/dsh-subprocess-local`） | 无 | 本地实现：detached 进程树、有界收集／spill、`node-pty`、前台／会话检查、进程树信号发送、运行时存储，以及先终止再等待退出的资源释放 | -服务负责进程生命周期；每个消费方负责进程执行的工作以及所应用的默认值。 +服务拥有跨消费方重载的进程存续期；消费方拥有一个进程的含义（一条 bash 命令、未来的非 shell 运行器）以及塑造它的每一项默认值。 diff --git a/packages/subprocess/subprocess-local/README.i18n.yaml b/packages/subprocess/subprocess-local/README.i18n.yaml index 28ebc2c110..d931d3574b 100644 --- a/packages/subprocess/subprocess-local/README.i18n.yaml +++ b/packages/subprocess/subprocess-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess-local/README.md -README.md: af9f92db714398dd52c9b5e7aeb64d5af71021da -README.zh.md: da78cbcfbff174eb5fda5b8323fbdc84b50c9997 +README.md: 6bc1003ae5903bb5640728bc79c3f9042fddbe7a +README.zh.md: 3c9ce73c7fcbeec9b17d73f212ecdcb6842ec133 diff --git a/packages/subprocess/subprocess-local/README.md b/packages/subprocess/subprocess-local/README.md index af9f92db71..6bc1003ae5 100644 --- a/packages/subprocess/subprocess-local/README.md +++ b/packages/subprocess/subprocess-local/README.md @@ -2,14 +2,16 @@ English | [中文](README.zh.md) -Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam: `LocalSubprocessService` spawns each spec's argv as a detached process tree, wires the spec's per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with optional spill files), and signals tree-scoped with SIGTERM→SIGKILL escalation. It has no config: every disposition, limit, and directory arrives on the spawn spec, so the deployment-varying knobs stay with the calling seams' configs ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)). +Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam. `LocalSubprocessService` owns a private runtime directory, resolves local executables, spawns ordinary detached process trees with explicit stdio, and implements terminal processes through `node-pty` plus platform process inspection. It has no config: every disposition, limit, terminal dimension, grace, and directory arrives from the calling seams ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-pty-local`](../../pty/pty-local/README.md), and [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md)). ## Behavior (and where it came from) -- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID /T /F`. `terminate()` — the handle's only termination verb — sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent) and is a no-op once the tree is gone; `waitForExit()` polls whole-tree liveness so consumer teardown confirms real quiescence. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools. +- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID /T /F` (injectable for tests). `terminate()` — the handle's only termination verb — sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent) and is a no-op once the tree is gone; `waitForExit()` polls whole-tree liveness so consumer teardown confirms real quiescence. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools. - **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory. -- **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). +- **Credential scrub + explicit merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; the spec's explicit `env` merges after that scrub with no namespace validation, so a deliberately supplied credential or current `DSH_*` fact wins while stale nested-harness identity cannot leak in ambiently. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md). - **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement. +- **Execution-world coordinates** — `cwd` is the host process cwd, `runtimeRoot` is an owner-private temporary directory removed on disposal, and `resolveExecutable` checks absolute files or searches the scrubbed effective PATH with platform-aware executable extensions. +- **Terminal-process ownership** — `spawnTerminal` allocates `node-pty`, bridges UTF-8 terminal bytes, inspects and signals the current foreground process group, and cleans descendants before the top-level shell. Linux `/proc`/syscall and macOS `ps` inspectors retain exact pid/start identity so pid reuse cannot redirect cleanup; the higher PTY backend owns prompt readiness, buffers, and model-facing operations. - **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement. ## Model Experience @@ -22,8 +24,10 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **Windows tree support is best-effort** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary. -- **The credential scrub is a name heuristic** — `*KEY*`/`*PASSWORD*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSPHRASE*`) pass through, and a whitelist for over-scrubbed vars is noted future work. +- **Windows tree support is best-effort and untested in CI** — termination routes through `taskkill /PID /T /F` with all outcomes contained (absent tree, races, missing binary), and liveness falls back to the direct-child boundary; the suites cover the routing through an injected runner only, and `packages/subprocess/*` is excluded from the Windows test matrix. +- **Terminal process inspection is Linux/macOS only** — the terminal primitive fails when its inspector has no supported platform implementation; Linux exact probes cover x64 and arm64, while macOS uses `ps` snapshots. +- **A daemonized terminal descendant can escape the captured tree** — a child that reparents before teardown is no longer discoverable from the `node-pty` root. The local provider accepts this gap rather than signal the root PID's POSIX session, which can include unrelated launcher processes. +- **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work. - **Completed spill files are not deleted** — bounded full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them; oversize incomplete spills are discarded and deletion is attempted immediately, but a cleanup failure can leave a bounded file behind. The raw process handling lives in `src/spawn.ts`; `src/index.ts` is the service wiring. diff --git a/packages/subprocess/subprocess-local/README.zh.md b/packages/subprocess/subprocess-local/README.zh.md index da78cbcfbf..3c9ce73c7f 100644 --- a/packages/subprocess/subprocess-local/README.zh.md +++ b/packages/subprocess/subprocess-local/README.zh.md @@ -2,15 +2,17 @@ [English](README.md) | 中文 -[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现：`LocalSubprocessService` 将每个 spec 的 argv spawn 为 detached 进程树，依照 spec 中按流划分的 stdio 处置方式（disposition）完成接线（原始管道、inherit、附带可选 spill 文件的有界尾部保留收集），并以进程树为范围发送信号，按 SIGTERM→SIGKILL 逐级升级。该实现没有任何配置：每项处置方式、限制与目录都随 spawn spec 传入，因此随部署变化的可调参数留在各调用方 seam 的配置里（[`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)）。 +[`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam 的本地实现。`LocalSubprocessService` 拥有私有运行时目录，解析本地可执行文件，以显式 stdio spawn 普通 detached 进程树，并通过 `node-pty` 与平台进程检查实现终端进程。该实现没有任何配置：每项处置方式、限制、终端尺寸、宽限期与目录都来自调用方 seam（[`dsh-bash-local`](../../bash/bash-local/README.md)、[`dsh-lsp-local`](../../lsp/lsp-local/README.md)、[`dsh-pty-local`](../../pty/pty-local/README.md)和 [`dsh-code-runtime-subprocess`](../../code-runtime/code-runtime-subprocess/README.md)）。 ## 行为（以及设计来源） -- **以适合平台的方式发送信号的 detached 进程树**：POSIX 子进程使用 `detached` spawn（拥有独立进程组），信号以负 pgid 发送并以直接子进程作为回退；Windows 通过 `taskkill /PID /T /F` 终止进程树。`terminate()`（句柄唯一的终止操作）先发送 SIGTERM，经过 spec 的宽限期后再发送 SIGKILL（沿用 OpenCode 的升级策略；流水线与子 shell 会随父进程一起结束），进程树消亡后为空操作；`waitForExit()` 轮询整棵进程树的存活状态，使消费方的拆卸能确认真正的完全停稳。组长进程退出后，仍然打开的管道也只获得同样有界的排空宽限期，因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH；重新指定父进程并脱离该组的 daemon 仍可能存活，这与所调研工具的局限相同。 -- **按流划分的处置方式**：`'pipe'` 把原始流原样交给调用方（协议分帧仍归消费方所有）；`'inherit'` 直通父进程的描述符；收集模式（collect）在输出超过上限后于内存中保留尾部（错误与结果通常聚集在末尾，沿用 pi/OpenCode 的理由），并在配置了 spill 上限时把完整流追加到一个私有临时文件；省略 `spill` 则只保留用于诊断的尾部。某条流大于 spill 上限时，会丢弃已不完整的 spill，仅返回带截断标记的尾部；spill 文件描述符在结算时封存，最终关闭失败时则不公布路径，以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机，位于按需创建、权限为 `0700` 的每进程目录之下。 -- **凭据清除 + 显式合并**：以 `process.env` 为基础，移除形似凭据的变量（`*KEY*`／`*PASSWORD*`／`*SECRET*`／`*TOKEN*`）和所有环境中已有的 `DSH_*` 名称；spec 的显式 `env` 在该清除之后合并且不做命名空间校验，因此有意提供的凭据或当前 `DSH_*` 事实会胜出，而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭；否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 -- **基于偏移量的读取**：收集模式的读取器按完整流的字节坐标返回增量；服务自身从不持有游标，因此消费方自有的游标（bash 的后台读取路径）与完整流重读可以共存，结算前后皆然。 -- **先终止再等待退出的 dispose（资源释放）**：服务保留存活句柄，只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出；已结算与 spawn 失败的句柄在结算时即离开存活集合。 +- **带平台正确信号发送的 detached 进程树**：POSIX 子进程使用 `detached` spawn（拥有独立进程组），信号以负 pgid 发送并以直接子进程作为回退；Windows 通过 `taskkill /PID /T /F` 终止进程树（可为测试注入）。`terminate()`（句柄唯一的终止动词）先发送 SIGTERM，经过 spec 的宽限期后再发送 SIGKILL（沿用 OpenCode 的升级策略；管道与子 shell 会随父进程一起结束），进程树消亡后为空操作；`waitForExit()` 轮询整棵进程树的存活状态，使消费方的拆卸能确认真正的完全停稳。组长进程退出后，仍然打开的管道也只获得同样有界的排空宽限期，因此存活的后代进程无法无限期地拖住结果不结算。系统会容忍 ESRCH；脱离该组重新挂载的 daemon 仍可能存活，这与调研工具的局限相同。 +- **按流划分的处置方式**：`'pipe'` 把原始流原样交给调用方（协议分帧仍归消费方所有）；`'inherit'` 直通父进程的描述符；收集模式（collect）在输出超过上限后于内存中保留尾部（错误与结果通常聚集在末尾，沿用 pi/OpenCode 的理由），并在配置了 spill 上限时把完整流追加到一个私有临时文件；省略 `spill` 则只保留尾部，即诊断尾部的形状。某条流大于 spill 上限时，会丢弃已不完整的 spill，仅返回带截断标记的尾部；spill 文件描述符在结算时封存，最终关闭失败时则不公布路径，以免声称存在不完整的文件。spill 文件权限为 `0600`、名称随机，位于按需延迟创建的 `0700` 每进程目录之下。 +- **凭据清除 + 显式合并**：以 `process.env` 为基础，移除形似凭据的变量（`*KEY*`／`*SECRET*`／`*TOKEN*`）和所有环境中已有的 `DSH_*` 名称；spec 的显式 `env` 在该清除之后合并且不做命名空间校验，因此有意提供的凭据或当前 `DSH_*` 事实会胜出，而陈旧的嵌套 harness 身份无法从环境中隐式漏入。提供的 stdin 会被写入后关闭；否则 fd 0 指向 `/dev/null`。参见 [stdin/env Agent Note（agent 决策记录）](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md)与[受管环境 Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md)。 +- **基于偏移量的读取**：收集模式的读取器以全流字节坐标返回增量；服务自身从不持有游标，因此消费方自有的游标（bash 的后台读取路径）与完整流重读可以共存，结算前后皆然。 +- **执行世界坐标**：`cwd` 是宿主进程 cwd，`runtimeRoot` 是所有者私有的临时目录，在资源释放时删除；`resolveExecutable` 检查绝对文件，或使用平台感知的可执行扩展名在清理后的有效 PATH 中查找。 +- **终端进程所有权**：`spawnTerminal` 分配 `node-pty`，桥接 UTF-8 终端字节，检查当前前台进程组并向其发送信号，并先于顶层 shell 清理后代。Linux 的 `/proc`／syscall 检查器与 macOS 的 `ps` 检查器会保留精确的 pid／启动身份，使 PID 复用无法把清理重定向到其他进程；上层 PTY 后端负责提示符就绪检测、缓冲和面向模型的操作。 +- **先终止再等待退出的 dispose**：服务保留存活句柄，只为让自身的 dispose 能对每个仍在运行的进程树执行升级并等待其退出；已结算与 spawn 失败的句柄在结算时即离开存活集合。 ## 模型体验 @@ -18,12 +20,14 @@ #### KV Cache 影响 -不会直接导致 KV Cache 失效；请求前缀变更由上述消费方负责。 +不会直接失效；请求前缀变更由具名消费方负责。 ## 已知限制与暂缓事项 -- **Windows 进程树支持仅为尽力而为**：终止经由 `taskkill /PID /T /F` 完成，所有结果都被就地吸收，不向外抛出（进程树已不存在、竞态、二进制缺失），存活探测则回退到直接子进程边界。 -- **凭据清除依赖名称启发式规则**：只匹配 `*KEY*`／`*PASSWORD*`／`*SECRET*`／`*TOKEN*`；名称不同的 secret（例如 `*PASSPHRASE*`）会继续传递，对误删变量引入白名单属于已记录的后续工作。 +- **Windows 进程树支持仅为尽力而为，且未经 CI 测试**：终止经由 `taskkill /PID /T /F` 完成，所有结果都被就地吸收，不向外抛出（进程树已不存在、竞态、二进制缺失），存活探测则回退到直接子进程边界；测试套件只通过注入的运行器覆盖这条路由，且 `packages/subprocess/*` 被排除在 Windows 测试矩阵之外。 +- **终端进程检查仅支持 Linux／macOS**：检查器没有受支持的平台实现时，终端原语会失败；Linux 精确探针覆盖 x64 与 arm64，macOS 使用 `ps` 快照。 +- **守护化的终端后代可能逃离已捕获进程树**：子进程若在拆卸前重新设定父进程，便无法再从 `node-pty` 根发现。本地提供方接受这个缺口，不向根 PID 的 POSIX 会话发送信号，因为其中可能包含无关的启动器进程。 +- **凭据清除依赖名称启发式规则**：只匹配 `*KEY*`／`*SECRET*`／`*TOKEN*`；名称不同的 secret（例如 `*PASSWORD*`）会继续传递，对误删变量引入白名单属于已记录的后续工作。 - **不会删除已完成的 spill 文件**：有界的完整输出恢复文件（以及每个进程的私有 spill 目录）会在 OS tmpdir 下累积，直到外部机制进行清理；超大的不完整 spill 会被丢弃并立即尝试删除，但清理失败可能留下一个有界文件。 原始进程处理位于 `src/spawn.ts`；`src/index.ts` 负责服务接线。 diff --git a/packages/subprocess/subprocess-local/package.json b/packages/subprocess/subprocess-local/package.json index e1429bffd0..7a68a9b91b 100644 --- a/packages/subprocess/subprocess-local/package.json +++ b/packages/subprocess/subprocess-local/package.json @@ -21,8 +21,12 @@ "files": [ "lib/index.js", "lib/invariant.js", + "scripts/ensure-spawn-helper.mjs", "lib/types/**/*.d.ts" ], + "scripts": { + "postinstall": "node scripts/ensure-spawn-helper.mjs" + }, "license": "BSD-3-Clause", "peerDependencies": { "@deepseek-ai/dsh-invariants": "^0.0.1", @@ -30,6 +34,9 @@ "@deepseek-ai/dsh-timeout": "^0.0.1", "cordis": "^4.0.0-rc.7" }, + "dependencies": { + "node-pty": "^1.1.0" + }, "devDependencies": { "@deepseek-ai/dsh-invariants": "workspace:^", "@deepseek-ai/dsh-subprocess": "workspace:^", diff --git a/packages/pty/pty-local/scripts/ensure-spawn-helper.mjs b/packages/subprocess/subprocess-local/scripts/ensure-spawn-helper.mjs similarity index 100% rename from packages/pty/pty-local/scripts/ensure-spawn-helper.mjs rename to packages/subprocess/subprocess-local/scripts/ensure-spawn-helper.mjs diff --git a/packages/subprocess/subprocess-local/src/index.ts b/packages/subprocess/subprocess-local/src/index.ts index eea1f1e739..40d2f52552 100644 --- a/packages/subprocess/subprocess-local/src/index.ts +++ b/packages/subprocess/subprocess-local/src/index.ts @@ -7,11 +7,26 @@ * @module @deepseek-ai/dsh-subprocess-local */ +import { constants } from 'node:fs' +import { mkdtempSync } from 'node:fs' +import { access, rm, stat } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { delimiter, extname, isAbsolute, join } from 'node:path' import { Context } from 'cordis' +import * as nodePty from 'node-pty' +import type { IPtyForkOptions } from 'node-pty' import { SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' -import { spawnSubprocess } from './spawn.ts' +import type { + SubprocessHandle, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' +import { childEnv, spawnSubprocess } from './spawn.ts' import type { SpawnInternals } from './spawn.ts' +import { createProcessInspector } from './process-inspector.ts' +import type { ProcessInspector } from './process-inspector.ts' +import { LocalTerminalHandle } from './terminal.ts' /** * Local subprocess service: detached process trees, Node-shaped stdio @@ -20,10 +35,16 @@ import type { SpawnInternals } from './spawn.ts' * SIGTERM→grace→SIGKILL escalation. */ export class LocalSubprocessService extends SubprocessService { + readonly cwd = process.cwd() + readonly runtimeRoot = mkdtempSync(join(tmpdir(), 'dsh-subprocess-runtime-')) /** Live handles retained only so disposal can terminate and join them. */ private live = new Set() + /** Live terminal sessions retained through whole-session quiescence. */ + private terminals = new Set() /** Test seam: spill and platform knobs forwarded to spawnSubprocess. */ internals: SpawnInternals = {} + /** Test seam for platform process inspection; production resolves lazily on terminal spawn. */ + terminalInspector: ProcessInspector | undefined constructor(ctx: Context) { super(ctx) @@ -37,11 +58,58 @@ export class LocalSubprocessService extends SubprocessService { // Spawn-failure rejections already settled and left the live set. pending.push(handle.done.catch(() => {}).then(() => handle.waitForExit())) } + for (const terminal of this.terminals) { + terminal.terminate() + // Cleanup may reject before the top-level process exits (for example, + // an identity-fenced descendant survives escalation). Await the cleanup + // transaction directly so disposal reports that failure rather than + // waiting forever on `done`. + pending.push(terminal.waitForExit()) + } this.live.clear() + this.terminals.clear() await Promise.all(pending) + await rm(this.runtimeRoot, { recursive: true, force: true }) }, 'local subprocess teardown') } + async resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise { + if (command.length === 0) throw new Error('subprocess-local: executable must be non-empty') + signal?.throwIfAborted() + const environment = childEnv(env) + const absolute = isAbsolute(command) + const candidates = absolute ? [command] : this.executableCandidates(command, environment) + for (const candidate of candidates) { + signal?.throwIfAborted() + try { + const info = await stat(candidate) + if (!info.isFile()) continue + await access(candidate, constants.X_OK) + signal?.throwIfAborted() + return candidate + } catch { + // Try the next PATH candidate; the final miss receives one stable error. + } + } + signal?.throwIfAborted() + throw new Error(absolute + ? `subprocess-local: command ${JSON.stringify(command)} is not an executable file` + : `subprocess-local: command ${JSON.stringify(command)} was not found on PATH`) + } + + private executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] { + const path = env.PATH ?? '' + const extensions = process.platform === 'win32' && extname(command) === '' + ? (env.PATHEXT ?? '.COM;.EXE;.BAT;.CMD').split(';') + : [''] + return path.split(delimiter).flatMap(directory => + directory === '' ? [] : extensions.map(extension => join(directory, command + extension))) + } + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const handle = spawnSubprocess(spec, this.internals) this.live.add(handle) @@ -54,6 +122,38 @@ export class LocalSubprocessService extends SubprocessService { handle.done.then(release, release) return handle } + + // Local PTY allocation is synchronous, but the provider seam permits remote asynchronous allocation. + // eslint-disable-next-line @typescript-eslint/require-await + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + const file = spec.argv[0] + if (file === undefined || file.length === 0) { + throw new Error('subprocess-local: terminal argv must contain a program') + } + for (const [name, value] of [['rows', spec.rows], ['cols', spec.cols], ['graceMs', spec.graceMs]] as const) { + if (!Number.isSafeInteger(value) || value <= 0) { + throw new Error(`subprocess-local: terminal ${name} must be a positive safe integer`) + } + } + spec.signal?.throwIfAborted() + const options: IPtyForkOptions = { + name: 'dumb', + rows: spec.rows, + cols: spec.cols, + cwd: spec.cwd, + env: childEnv(spec.env), + } + const inspector = this.terminalInspector ?? createProcessInspector() + const terminal = nodePty.spawn(file, [...spec.argv.slice(1)], options) + const handle = new LocalTerminalHandle(terminal, inspector, spec.graceMs, spec.signal) + this.terminals.add(handle) + const release = async (): Promise => { + await handle.waitForExit() + this.terminals.delete(handle) + } + void handle.done.then(release, release).catch(() => {}) + return handle + } } export default LocalSubprocessService diff --git a/packages/pty/pty-local/src/process-inspector.ts b/packages/subprocess/subprocess-local/src/process-inspector.ts similarity index 96% rename from packages/pty/pty-local/src/process-inspector.ts rename to packages/subprocess/subprocess-local/src/process-inspector.ts index 5be25a224a..f57184f289 100644 --- a/packages/pty/pty-local/src/process-inspector.ts +++ b/packages/subprocess/subprocess-local/src/process-inspector.ts @@ -1,8 +1,8 @@ -/** Platform process-table inspection used for readiness, signals, and teardown. */ +/** Platform process-table inspection for terminal readiness, signals, and teardown. */ import { closeSync, openSync, readFileSync, readdirSync, readSync } from 'node:fs' import { execFileSync } from 'node:child_process' -import type { PtySignal } from '@deepseek-ai/dsh-pty' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' /** PID plus start identity, preventing teardown escalation after PID reuse. */ export interface ProcessIdentity { @@ -18,7 +18,7 @@ export interface ProcessInspector { processTree(rootPid: number): ProcessIdentity[] /** Return whether the exact identity remains a non-quiescent process. */ isAlive(identity: ProcessIdentity): boolean - signalGroup(pgid: number, signal: PtySignal): void + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL'): void } @@ -203,7 +203,7 @@ abstract class PosixProcessInspector implements ProcessInspector { abstract processTree(rootPid: number): ProcessIdentity[] abstract isAlive(identity: ProcessIdentity): boolean - signalGroup(pgid: number, signal: PtySignal): void { + signalGroup(pgid: number, signal: SubprocessTerminalSignal): void { this.internals.kill(-pgid, signal) } @@ -327,5 +327,5 @@ export function createProcessInspector( ): ProcessInspector { if (platform === 'linux') return new LinuxProcessInspector(arch, internals) if (platform === 'darwin') return new MacProcessInspector(internals) - throw new Error(`pty-local: unsupported platform ${platform}`) + throw new Error(`subprocess-local: terminal inspection is unsupported on platform ${platform}`) } diff --git a/packages/subprocess/subprocess-local/src/terminal.ts b/packages/subprocess/subprocess-local/src/terminal.ts new file mode 100644 index 0000000000..d4b7bd26d1 --- /dev/null +++ b/packages/subprocess/subprocess-local/src/terminal.ts @@ -0,0 +1,226 @@ +/** Local node-pty terminal-process implementation for the subprocess seam. */ + +import { Buffer } from 'node:buffer' +import { constants } from 'node:os' +import { PassThrough } from 'node:stream' +import type { IDisposable, IPty } from 'node-pty' +import type { + SubprocessOutcome, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, +} from '@deepseek-ai/dsh-subprocess' +import type { ProcessIdentity, ProcessInspector } from './process-inspector.ts' + +function delay(ms: number): Promise { + return new Promise(resolve => setTimeout(resolve, ms)) +} + +function signalName(number: number | undefined): NodeJS.Signals | null { + if (number === undefined || number === 0) return null + for (const [name, value] of Object.entries(constants.signals)) { + if (value === number) return name as NodeJS.Signals + } + return null +} + +/** A local terminal whose process-session ownership stays below the PTY backend. */ +export class LocalTerminalHandle implements SubprocessTerminalHandle { + readonly pid: number + readonly output = new PassThrough() + readonly done: Promise + + private readonly outcome = Promise.withResolvers() + private readonly dataDisposable: IDisposable + private readonly exitDisposable: IDisposable + private exited = false + private termination: Promise | undefined + private removeAbort: (() => void) | undefined + + /** + * @param terminal - allocated node-pty process. + * @param inspector - platform process/session operations. + * @param graceMs - TERM-to-KILL and exit-wait grace. + * @param signal - optional lifetime cancellation. + */ + constructor( + private readonly terminal: IPty, + private readonly inspector: ProcessInspector, + private readonly graceMs: number, + signal?: AbortSignal, + ) { + this.pid = terminal.pid + this.done = this.outcome.promise + this.dataDisposable = terminal.onData((data) => { this.output.write(Buffer.from(data, 'utf8')) }) + this.exitDisposable = terminal.onExit(({ exitCode, signal: exitSignal }) => { + if (this.exited) return + this.exited = true + this.output.end() + this.outcome.resolve({ + exitCode: exitSignal === undefined || exitSignal === 0 ? exitCode : null, + signal: signalName(exitSignal), + }) + this.terminate() + }) + if (signal !== undefined) { + const onAbort = (): void => { this.terminate() } + signal.addEventListener('abort', onAbort, { once: true }) + this.removeAbort = () => { signal.removeEventListener('abort', onAbort) } + if (signal.aborted) this.terminate() + } + } + + // node-pty writes synchronously; the seam returns a promise for remote transports. + // eslint-disable-next-line @typescript-eslint/require-await + async write(data: Uint8Array): Promise { + if (this.exited) throw new Error('terminal process has exited') + let text: string + try { + text = new TextDecoder('utf-8', { fatal: true }).decode(data) + } catch (error: unknown) { + throw new Error('terminal input must be valid UTF-8', { cause: error }) + } + this.terminal.write(text) + } + + // Local inspection is synchronous; the seam returns a promise for remote transports. + // eslint-disable-next-line @typescript-eslint/require-await + async inspectForeground(): Promise { + const processGroupId = this.inspector.foregroundPgid(this.pid) + if (processGroupId === undefined) return undefined + return { + processGroupId, + inputWaiting: this.inspector.isStdinWaiting(processGroupId), + } + } + + async signalForeground(signal: SubprocessTerminalSignal): Promise { + const foreground = await this.inspectForeground() + if (foreground === undefined) { + throw new Error(`cannot resolve foreground process group for terminal ${this.pid}`) + } + if (signal === 'SIGKILL' && foreground.processGroupId === this.pid) { + throw new Error('refusing to SIGKILL the terminal shell; terminate the terminal session instead') + } + this.inspector.signalGroup(foreground.processGroupId, signal) + return foreground.processGroupId + } + + terminate(): void { + this.termination ??= this.closeOnce().catch((error: unknown) => { + this.termination = undefined + throw error + }) + void this.termination.catch(() => {}) + } + + async waitForExit(signal?: AbortSignal): Promise { + // A caller may begin waiting before the top-level process exits. The exit + // callback starts descendant cleanup in the same turn, so resolve that + // eventual transaction after `done` instead of snapshotting only `done`. + const quiescence = this.termination ?? this.done.then(() => this.termination) + if (signal === undefined) { + await quiescence + return true + } + if (signal.aborted) return false + return await new Promise((resolve, reject) => { + const onAbort = (): void => { cleanup(); resolve(false) } + const cleanup = (): void => { signal.removeEventListener('abort', onAbort) } + signal.addEventListener('abort', onAbort, { once: true }) + void quiescence.then( + () => { cleanup(); resolve(true) }, + (error: unknown) => { + cleanup() + // The owned cleanup transaction only throws Error diagnostics. + // eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors + reject(error) + }, + ) + }) + } + + private survivors(members: ProcessIdentity[]): ProcessIdentity[] { + return members.filter(member => this.inspector.isAlive(member)) + } + + private descendants(): ProcessIdentity[] { + return this.inspector.processTree(this.pid).filter(member => member.pid !== this.pid) + } + + private async waitForMembers(members: ProcessIdentity[]): Promise { + const until = Date.now() + this.graceMs + let survivors = this.survivors(members) + while (survivors.length > 0 && Date.now() < until) { + await delay(Math.min(25, Math.max(1, until - Date.now()))) + survivors = this.survivors(members) + } + return survivors + } + + private signalMembers(members: ProcessIdentity[], signal: 'SIGTERM' | 'SIGKILL'): void { + for (const member of members) { + try { + this.inspector.signalProcess(member, signal) + } catch (_alreadyExitedDuringSignal) { + // The exact process identity is rechecked; a same-tick exit is success. + } + } + } + + private unionMembers(...groups: ProcessIdentity[][]): ProcessIdentity[] { + const members: ProcessIdentity[] = [] + const seen = new Set() + for (const group of groups) { + for (const member of group) { + const key = `${member.pid}:${member.started}` + if (seen.has(key)) continue + seen.add(key) + members.push(member) + } + } + return members + } + + private async stopDescendants(): Promise { + const captured = this.descendants() + this.signalMembers(captured, 'SIGTERM') + const capturedSurvivors = await this.waitForMembers(captured) + const members = this.unionMembers(capturedSurvivors, this.descendants()) + this.signalMembers(members, 'SIGKILL') + const survivors = await this.waitForMembers(members) + return this.survivors(this.unionMembers(survivors, this.descendants())) + } + + private async stopShell(): Promise { + if (!this.exited) { + try { + this.terminal.kill('SIGTERM') + } catch (_topLevelAlreadyExitedDuringTerm) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!this.exited) { + try { + this.terminal.kill('SIGKILL') + } catch (_topLevelAlreadyExitedDuringKill) { + // The exit callback is authoritative. + } + await Promise.race([this.done.then(() => undefined), delay(this.graceMs)]) + } + if (!this.exited) throw new Error(`terminal cleanup failed; surviving pid: ${this.pid}`) + } + + private async closeOnce(): Promise { + const survivors = await this.stopDescendants() + if (survivors.length > 0) { + throw new Error(`terminal cleanup failed; surviving pids: ${survivors.map(member => member.pid).join(', ')}`) + } + await this.stopShell() + this.removeAbort?.() + this.removeAbort = undefined + this.dataDisposable.dispose() + this.exitDisposable.dispose() + } +} diff --git a/packages/subprocess/subprocess-local/tests/local.spec.ts b/packages/subprocess/subprocess-local/tests/local.spec.ts index ca723fdb87..bc77d24cfd 100644 --- a/packages/subprocess/subprocess-local/tests/local.spec.ts +++ b/packages/subprocess/subprocess-local/tests/local.spec.ts @@ -1,7 +1,10 @@ -import { describe, expect, it } from 'vitest' +import { PassThrough } from 'node:stream' +import { describe, expect, it, vi } from 'vitest' +import { stat } from 'node:fs/promises' +import { basename, delimiter, dirname } from 'node:path' import { Context } from 'cordis' import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local' -import type { SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { SubprocessSpawnSpec, SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from '@deepseek-ai/dsh-subprocess' function spec(command: string, overrides: Partial = {}): SubprocessSpawnSpec { return { @@ -18,6 +21,133 @@ function spec(command: string, overrides: Partial = {}): Su } describe('LocalSubprocessService', () => { + it('publishes execution-world paths and removes its private runtime directory', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const root = ctx.subprocess.runtimeRoot + expect(ctx.subprocess.cwd).toBe(process.cwd()) + expect((await stat(root)).isDirectory()).toBe(true) + await fiber.dispose() + await expect(stat(root)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('resolves absolute and PATH executables and honors lookup cancellation', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + expect(await ctx.subprocess.resolveExecutable(process.execPath)).toBe(process.execPath) + expect(await ctx.subprocess.resolveExecutable(basename(process.execPath), { + PATH: dirname(process.execPath), + })).toBe(process.execPath) + await expect(ctx.subprocess.resolveExecutable('')).rejects.toThrow('must be non-empty') + await expect(ctx.subprocess.resolveExecutable('dsh-command-that-does-not-exist', { PATH: '' })) + .rejects.toThrow('was not found on PATH') + await expect(ctx.subprocess.resolveExecutable('/dsh-absolute-command-that-does-not-exist')) + .rejects.toThrow('is not an executable file') + await expect(ctx.subprocess.resolveExecutable(process.cwd())) + .rejects.toThrow('is not an executable file') + await expect(ctx.subprocess.resolveExecutable(process.execPath, {}, AbortSignal.abort('stop'))) + .rejects.toBe('stop') + await fiber.dispose() + }) + + it('builds Windows executable candidates without empty PATH entries', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const service = ctx.subprocess as LocalSubprocessService + const candidates = (service as unknown as { + executableCandidates(command: string, env: NodeJS.ProcessEnv): string[] + }).executableCandidates.bind(service) + const platform = vi.spyOn(process, 'platform', 'get').mockReturnValue('win32') + try { + expect(candidates('tool', { PATH: `${delimiter}/bin`, PATHEXT: '.EXE;.CMD' })) + .toEqual(['/bin/tool.EXE', '/bin/tool.CMD']) + expect(candidates('tool.exe', {})).toEqual([]) + expect(candidates('tool', { PATH: '/bin' })).toHaveLength(4) + } finally { + platform.mockRestore() + await fiber.dispose() + } + }) + + it('validates terminal spawn specs before allocating a PTY', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const base: SubprocessTerminalSpawnSpec = { + argv: ['bash'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 10, + } + await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [] })).rejects.toThrow('must contain a program') + await expect(ctx.subprocess.spawnTerminal({ ...base, argv: [''] })).rejects.toThrow('must contain a program') + await expect(ctx.subprocess.spawnTerminal({ ...base, rows: 1.5 })).rejects.toThrow('rows') + await expect(ctx.subprocess.spawnTerminal({ ...base, cols: 0 })).rejects.toThrow('cols') + await expect(ctx.subprocess.spawnTerminal({ ...base, graceMs: 0 })).rejects.toThrow('graceMs') + await expect(ctx.subprocess.spawnTerminal({ ...base, signal: AbortSignal.abort('stop') })).rejects.toBe('stop') + await fiber.dispose() + }) + + it('terminates and joins an owned terminal during disposal', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalSubprocessService) + const terminate = vi.fn() + const waitForExit = vi.fn(async () => true) + const terminal: SubprocessTerminalHandle = { + pid: 1, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => undefined, + signalForeground: async () => 1, + terminate, + waitForExit, + } + ;(ctx.subprocess as unknown as { terminals: Set }).terminals.add(terminal) + await fiber.dispose() + expect(terminate).toHaveBeenCalledOnce() + expect(waitForExit).toHaveBeenCalledOnce() + }) + + it('contains a terminal release failure after top-level exit', async () => { + let exitListener: ((event: { exitCode: number; signal?: number }) => void) | undefined + const terminal = { + pid: 123, + onData: () => ({ dispose: () => {} }), + onExit: (listener: (event: { exitCode: number; signal?: number }) => void) => { + exitListener = listener + return { dispose: () => {} } + }, + write: () => {}, + kill: () => {}, + } + vi.resetModules() + vi.doMock('node-pty', () => ({ spawn: () => terminal })) + try { + const { default: IsolatedLocalSubprocessService } = await import('../src/index.ts') + const ctx = new Context() + const fiber = await ctx.plugin(IsolatedLocalSubprocessService) + const alive = new Set([124]) + ;(ctx.subprocess as InstanceType).terminalInspector = { + foregroundPgid: () => 123, + isStdinWaiting: () => false, + processTree: () => [{ pid: 124, started: 'child' }], + isAlive: identity => alive.has(identity.pid), + signalGroup: () => {}, + signalProcess: () => {}, + } + const handle = await ctx.subprocess.spawnTerminal({ + argv: ['shell'], cwd: process.cwd(), rows: 24, cols: 80, graceMs: 1, + }) + exitListener?.({ exitCode: 0 }) + await handle.done + await new Promise(resolve => setTimeout(resolve, 10)) + alive.clear() + handle.terminate() + await handle.waitForExit() + await fiber.dispose() + } finally { + vi.doUnmock('node-pty') + vi.resetModules() + } + }) + it('registers as ctx.subprocess and spawns managed handles', async () => { const ctx = new Context() const fiber = await ctx.plugin(LocalSubprocessService) diff --git a/packages/pty/pty-local/tests/process-inspector.spec.ts b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts similarity index 98% rename from packages/pty/pty-local/tests/process-inspector.spec.ts rename to packages/subprocess/subprocess-local/tests/process-inspector.spec.ts index 218a3d77fe..e8ad2fd11e 100644 --- a/packages/pty/pty-local/tests/process-inspector.spec.ts +++ b/packages/subprocess/subprocess-local/tests/process-inspector.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' -import { createProcessInspector, parseProcStat } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' -import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-pty-local/src/process-inspector.ts' +import { createProcessInspector, parseProcStat } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import type { ProcessInspectorInternals } from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' function stat(pid: number, pgrp: number, session: number, tpgid: number, started: string, parentPid = 1, state = 'S'): string { const rest = [state, String(parentPid), String(pgrp), String(session), '99', String(tpgid)] @@ -216,6 +216,6 @@ describe('macOS process inspector', () => { expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() fake.internals.exec = () => { throw new Error('gone') } expect(createProcessInspector('darwin', 'arm64', fake.internals).foregroundPgid(1)).toBeUndefined() - expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported platform win32') + expect(() => createProcessInspector('win32', 'x64', fake.internals)).toThrow('unsupported on platform win32') }) }) diff --git a/packages/subprocess/subprocess-local/tests/terminal.spec.ts b/packages/subprocess/subprocess-local/tests/terminal.spec.ts new file mode 100644 index 0000000000..417fd62517 --- /dev/null +++ b/packages/subprocess/subprocess-local/tests/terminal.spec.ts @@ -0,0 +1,275 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import type { IDisposable, IPty } from 'node-pty' +import { LocalTerminalHandle } from '@deepseek-ai/dsh-subprocess-local/src/terminal.ts' +import type { + ProcessIdentity, + ProcessInspector, +} from '@deepseek-ai/dsh-subprocess-local/src/process-inspector.ts' +import type { SubprocessTerminalSignal } from '@deepseek-ai/dsh-subprocess' + +class FakePty { + pid = 123 + readonly writes: string[] = [] + readonly kills: string[] = [] + autoExitOnKill = true + throwKill = false + private readonly dataListeners = new Set<(data: string) => void>() + private readonly exitListeners = new Set<(event: { exitCode: number; signal?: number }) => void>() + + readonly onData = (listener: (data: string) => void): IDisposable => { + this.dataListeners.add(listener) + return { dispose: () => { this.dataListeners.delete(listener) } } + } + + readonly onExit = (listener: (event: { exitCode: number; signal?: number }) => void): IDisposable => { + this.exitListeners.add(listener) + return { dispose: () => { this.exitListeners.delete(listener) } } + } + + emitData(data: string): void { + for (const listener of this.dataListeners) listener(data) + } + + emitExit(exitCode = 0, signal?: number): void { + for (const listener of this.exitListeners) listener({ exitCode, ...signal === undefined ? {} : { signal } }) + } + + write(data: string): void { this.writes.push(data) } + + kill(signal?: string): void { + if (this.throwKill) throw new Error('process raced') + this.kills.push(signal ?? 'SIGHUP') + if (this.autoExitOnKill) this.emitExit(0, signal === 'SIGKILL' ? 9 : 15) + } + + asPty(): IPty { + return this as unknown as IPty + } +} + +class FakeInspector implements ProcessInspector { + pgid: number | undefined = 456 + waiting = false + members: ProcessIdentity[] = [] + readonly alive = new Set() + readonly groups: Array<[number, SubprocessTerminalSignal]> = [] + readonly processes: Array<[number, 'SIGTERM' | 'SIGKILL']> = [] + throwGroup = false + throwProcess = false + removeOnSignal = true + + foregroundPgid() { return this.pgid } + isStdinWaiting() { return this.waiting } + processTree() { return this.members } + isAlive(identity: ProcessIdentity) { return this.alive.has(identity.pid) } + signalGroup(pgid: number, signal: SubprocessTerminalSignal) { + if (this.throwGroup) throw new Error('group failed') + this.groups.push([pgid, signal]) + } + signalProcess(identity: ProcessIdentity, signal: 'SIGTERM' | 'SIGKILL') { + if (this.throwProcess) throw new Error('process raced') + this.processes.push([identity.pid, signal]) + if (this.removeOnSignal) this.alive.delete(identity.pid) + } +} + +afterEach(() => { vi.useRealTimers() }) + +describe('LocalTerminalHandle', () => { + it('bridges terminal bytes, foreground control, and signalled exit facts', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.waiting = true + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + const chunks: Buffer[] = [] + handle.output.on('data', (chunk: Buffer) => { chunks.push(chunk) }) + + pty.emitData('hello €') + await handle.write(Buffer.from('input\r')) + expect(pty.writes).toEqual(['input\r']) + expect(await handle.inspectForeground()).toEqual({ processGroupId: 456, inputWaiting: true }) + expect(await handle.signalForeground('SIGINT')).toBe(456) + expect(inspector.groups).toEqual([[456, 'SIGINT']]) + + pty.emitExit(7, 9) + pty.emitExit(0) + expect(await handle.done).toEqual({ exitCode: null, signal: 'SIGKILL' }) + expect(await handle.waitForExit()).toBe(true) + expect(Buffer.concat(chunks).toString('utf8')).toBe('hello €') + }) + + it('rejects invalid input and unsafe foreground signals', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + await expect(handle.write(Uint8Array.from([0xff]))).rejects.toThrow('valid UTF-8') + + inspector.pgid = handle.pid + await expect(handle.signalForeground('SIGKILL')).rejects.toThrow('terminate the terminal session') + inspector.pgid = undefined + expect(await handle.inspectForeground()).toBeUndefined() + await expect(handle.signalForeground('SIGTERM')).rejects.toThrow('cannot resolve') + + pty.emitExit(3) + expect(await handle.done).toEqual({ exitCode: 3, signal: null }) + await handle.waitForExit() + await expect(handle.write(Buffer.from('late'))).rejects.toThrow('has exited') + }) + + it('keeps the shell alive until forced descendants leave', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + + handle.terminate() + const quiescent = handle.waitForExit() + await vi.advanceTimersByTimeAsync(20) + expect(inspector.processes).toContainEqual([124, 'SIGKILL']) + expect(pty.kills).toEqual([]) + + inspector.alive.delete(124) + await vi.advanceTimersByTimeAsync(20) + expect(await quiescent).toBe(true) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('keeps an early exit wait pending through descendant cleanup', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + const waiting = handle.waitForExit() + let settled = false + void waiting.then(() => { settled = true }) + + pty.emitExit() + await vi.advanceTimersByTimeAsync(10) + expect(settled).toBe(false) + + inspector.alive.delete(124) + await vi.advanceTimersByTimeAsync(20) + expect(await waiting).toBe(true) + }) + + it('rescans for descendants forked during TERM', async () => { + const pty = new FakePty() + const inspector = new FakeInspector() + let reads = 0 + inspector.processTree = () => { + reads += 1 + if (reads === 1) { + inspector.alive.add(124) + return [{ pid: 124, started: 'first' }] + } + if (reads === 2) { + inspector.alive.add(125) + return [{ pid: 125, started: 'late' }] + } + return [] + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + handle.terminate() + await handle.waitForExit() + expect(inspector.processes).toEqual([[124, 'SIGTERM'], [125, 'SIGKILL']]) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('retains captured descendants after reparenting', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + const captured = { pid: 124, started: 'captured' } + let reads = 0 + inspector.alive.add(captured.pid) + inspector.processTree = () => reads++ === 0 ? [captured] : [] + inspector.signalProcess = (identity, signal) => { + inspector.processes.push([identity.pid, signal]) + if (signal === 'SIGKILL') inspector.alive.delete(identity.pid) + } + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 20) + handle.terminate() + const quiescent = handle.waitForExit() + await vi.advanceTimersByTimeAsync(25) + expect(await quiescent).toBe(true) + expect(inspector.processes).toEqual([[124, 'SIGTERM'], [124, 'SIGKILL']]) + }) + + it('allows cleanup to retry after a surviving descendant leaves', async () => { + vi.useFakeTimers() + const pty = new FakePty() + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.removeOnSignal = false + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 10) + + handle.terminate() + const first = expect(handle.waitForExit(new AbortController().signal)).rejects.toThrow('surviving pids: 124') + await vi.advanceTimersByTimeAsync(25) + await first + + inspector.alive.delete(124) + handle.terminate() + expect(await handle.waitForExit()).toBe(true) + expect(pty.kills).toEqual(['SIGTERM']) + }) + + it('bounds waits and reports a top-level process that ignores escalation', async () => { + vi.useFakeTimers() + const pty = new FakePty() + pty.autoExitOnKill = false + const handle = new LocalTerminalHandle(pty.asPty(), new FakeInspector(), 10) + expect(await handle.waitForExit(AbortSignal.abort())).toBe(false) + const controller = new AbortController() + const bounded = handle.waitForExit(controller.signal) + controller.abort() + expect(await bounded).toBe(false) + + handle.terminate() + const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pid: 123') + await vi.advanceTimersByTimeAsync(25) + await failed + expect(pty.kills).toEqual(['SIGTERM', 'SIGKILL']) + + pty.emitExit(0, 999) + expect(await handle.done).toEqual({ exitCode: null, signal: null }) + handle.terminate() + expect(await handle.waitForExit()).toBe(true) + }) + + it('contains process races and reacts to lifetime cancellation', async () => { + const pty = new FakePty() + pty.throwKill = true + const inspector = new FakeInspector() + inspector.members = [{ pid: 124, started: 'child' }] + inspector.alive.add(124) + inspector.throwProcess = true + const controller = new AbortController() + const handle = new LocalTerminalHandle(pty.asPty(), inspector, 1, controller.signal) + controller.abort() + const failed = expect(handle.waitForExit()).rejects.toThrow('surviving pids: 124') + await failed + + inspector.alive.delete(124) + pty.throwKill = false + handle.terminate() + await handle.waitForExit() + + const preAbortedPty = new FakePty() + const preAborted = new LocalTerminalHandle( + preAbortedPty.asPty(), + new FakeInspector(), + 1, + AbortSignal.abort('stop'), + ) + await preAborted.waitForExit() + expect(preAbortedPty.kills).toEqual(['SIGTERM']) + }) +}) diff --git a/packages/subprocess/subprocess/README.i18n.yaml b/packages/subprocess/subprocess/README.i18n.yaml index b178daafb2..84d7d4a666 100644 --- a/packages/subprocess/subprocess/README.i18n.yaml +++ b/packages/subprocess/subprocess/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/subprocess/subprocess/README.md -README.md: e59dd96df036826f36bd0286c977438d2d87d1cf -README.zh.md: e8fb89dfd1f8c41a0caefc96469c13d9ae7d415d +README.md: 84b4b0c11c74c96929fa97b58fb33156d44e6ef1 +README.zh.md: dbd80a1c975719884481501f5cc43798e464a4fb diff --git a/packages/subprocess/subprocess/README.md b/packages/subprocess/subprocess/README.md index e59dd96df0..84b4b0c11c 100644 --- a/packages/subprocess/subprocess/README.md +++ b/packages/subprocess/subprocess/README.md @@ -2,15 +2,17 @@ English | [中文](README.zh.md) -The subprocess seam (`ctx.subprocess`). The abstract `SubprocessService` exposes one method — `spawn(spec): SubprocessHandle` — plus the vocabulary shared by every consumer: the fully-explicit `SubprocessSpawnSpec`, `SubprocessHandle` with its non-consuming offset-based output readers, `SubprocessOutcome`, `CollectedOutput`, and the managed `DSH_*` environment namespace (`DSH_ENV_PREFIX`, `DshEnvironment`). The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). +The subprocess seam (`ctx.subprocess`) is the process half of one execution world. The abstract `SubprocessService` exposes its canonical `cwd`, private `runtimeRoot`, executable lookup, ordinary managed `spawn`, and one terminal-process primitive; its vocabulary covers raw/collected stdio, process and terminal handles, exit facts, tree/session cleanup, and the managed `DSH_*` environment namespace. The local implementation lives in [`dsh-subprocess-local`](../subprocess-local/README.md). ## Contract - `spawn(spec)` returns immediately with a live handle; `done` resolves at process close with exit facts (`SubprocessOutcome` carries no output and no cause classification) and rejects only for spawn-level failures. -- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). Grace must be positive, finite, and no greater than [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md), so the implementation can represent it with one Node timer instead of accepting a value that Node collapses to one millisecond. `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. +- `cwd` and `runtimeRoot` are absolute paths in the provider's execution world. Consumers materialize private helpers below `runtimeRoot`, never in a host-only temp directory. `resolveExecutable(command, env?, signal?)` verifies absolute commands or resolves bare names against that world's scrubbed PATH plus explicit overrides. +- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself. - Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement. - Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `terminate()` — the only termination verb — escalates SIGTERM→grace→SIGKILL (idempotent, driven by the spec's abort signal too, a no-op once the tree is gone), and `waitForExit(signal?)` observes whole-tree liveness so a consumer-owned teardown ladder holds each tier on real quiescence — the manager reacts but never classifies why (callers own deadlines, teardown ladders, and cause classification). -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and the spec's explicit `env` merges after the scrub with no namespace validation — a string deliberately forwards or overrides a value, while an `undefined` tombstone removes an ordinary ambient entry. Spawners that cannot route through the service (node-pty backends, SDK-managed transports) import the scrub. +- `spawnTerminal(spec)` is the only non-pipe primitive. Its handle owns a real PTY, valid-UTF-8 byte I/O, foreground-process-group inspection/signalling, TERM-to-KILL whole-session cleanup, and a quiescence wait. The output stream ends after queued output when the top-level process exits; a live transport failure rejects `done`. These operations remain one substrate primitive because ordinary pipes cannot allocate a controlling terminal or prove and clean the complete terminal session; readiness, scrollback, and owner policy remain in the PTY consumer. +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, and explicit `env` merges after the scrub. The local ordinary and terminal spawns both apply it; SDK-managed transports that own their spawn may import it directly. - Disposal of the service terminates all still-running managed processes and awaits their exit. See the [subprocess data-structure catalog](../../../docs/core-data-structures/subprocess.md) and the [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md). @@ -25,5 +27,5 @@ No direct invalidation; the named consumers own any request-prefix changes. ## Known Limitations and Deferred Work -- **node-pty and SDK-managed spawns share only the scrub** — the PTY backend's terminal fork and the MCP SDK's own stdio transport cannot route their spawns through this seam (the library owns the fork/spawn call); they import `scrubbedParentEnv` so the environment policy stays single-sourced. +- **SDK-managed spawns remain outside** — an SDK transport that owns its internal spawn cannot route that call through this service; it can still import `scrubbedParentEnv` so environment policy stays single-sourced. - **Teardown ladders are consumer-owned** — the seam ships signalling verbs and the tree-liveness wait, not a canned quiesce sequence; each out-of-process consumer encodes its child's cooperation shape itself (the ACP backend's stdin-EOF-first ladder is the in-repo template). diff --git a/packages/subprocess/subprocess/README.zh.md b/packages/subprocess/subprocess/README.zh.md index e8fb89dfd1..dbd80a1c97 100644 --- a/packages/subprocess/subprocess/README.zh.md +++ b/packages/subprocess/subprocess/README.zh.md @@ -2,28 +2,30 @@ [English](README.md) | 中文 -子进程 seam（`ctx.subprocess`）。抽象的 `SubprocessService` 只暴露一个方法：`spawn(spec): SubprocessHandle`，外加所有消费方共享的词汇：完全显式的 `SubprocessSpawnSpec`、携带基于偏移量的非消费式输出读取器的 `SubprocessHandle`、`SubprocessOutcome`、`CollectedOutput`，以及受管的 `DSH_*` 环境命名空间（`DSH_ENV_PREFIX`、`DshEnvironment`）。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 +进程管理器 seam（`ctx.subprocess`）是同一执行世界中的进程侧。抽象的 `SubprocessService` 公开其规范化 `cwd`、私有 `runtimeRoot`、可执行文件查找、普通受管 `spawn` 和一项终端进程原语；其词汇涵盖原始／收集式 stdio、进程与终端句柄、退出事实、进程树／会话清理，以及受管的 `DSH_*` 环境命名空间。本地实现位于 [`dsh-subprocess-local`](../subprocess-local/README.md)。 ## 契约 -- `spawn(spec)` 立即返回一个活动句柄；`done` 在进程关闭时以退出事实 resolve（`SubprocessOutcome` 不携带输出，也不携带原因分类），仅在 spawn 层面失败时 reject。 -- spec 完全显式（argv、cwd、按流划分的 stdio 处置方式（disposition）、宽限期），因为随部署变化的默认值属于调用方 seam 的配置，而不属于某个隐藏的子进程默认值（`dsh-bash` 的 request/spec 拆分是这条规则的所属模板）。宽限期须为正有限值，且不得大于 [`MAX_TIMER_DELAY_MS`](../../util/timeout/README.md)，这样实现便可用一个 Node 定时器表示它，而不会接受会被 Node 折叠为 1 毫秒的值。`argv` 绝不经过 shell 解释；需要 shell 的消费方自行传入 `['bash', '-c', command]`。 -- stdio 按流采用 Node 风格：`'pipe'` 把原始流交给调用方做自己的协议分帧（LSP 的 JSON-RPC、ACP（Agent Client Protocol）的 ndjson），`'inherit'` 直通父进程描述符以承载诊断输出，收集模式（collect）`{ maxBytes, spill? }` 则缓冲一段有界尾部，外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费，因此独立的读取器不会抢走彼此的增量；偏移量滑出内存尾部窗口的读取标记为 `lossy`，并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 +- `spawn(spec)` 立即返回一个实时句柄；`done` 在进程关闭时以退出事实 resolve（`SubprocessOutcome` 不携带输出，也不携带原因分类），仅在 spawn 层面失败时 reject。 +- `cwd` 与 `runtimeRoot` 是提供方执行世界中的绝对路径。消费方在 `runtimeRoot` 下物化私有辅助程序，绝不使用仅宿主可见的临时目录。`resolveExecutable(command, env?, signal?)` 验证绝对命令，或根据该执行世界清理后的 PATH 加显式覆盖来解析裸名称。 +- spec 完全显式（argv、cwd、按流划分的 stdio 处置方式（disposition）、宽限期），因为随部署变化的默认值属于调用方 seam 的配置，而不属于某个隐藏的进程管理器默认值（`dsh-bash` 的 request/spec 拆分是这条规则的所属模板）。`argv` 绝不经过 shell 解释；需要 shell 的消费方自行传入 `['bash', '-c', command]`。 +- stdio 按流采用 Node 形状：`'pipe'` 把原始流交给调用方做自己的协议分帧（LSP 的 JSON-RPC、ACP（Agent Client Protocol）的 ndjson），`'inherit'` 直通父进程描述符以承载诊断输出，收集模式（collect）`{ maxBytes, spill? }` 则缓冲一段有界尾部，外加可选的完整流 spill 文件。收集模式的读取器接受全流字节偏移量且从不消费，因此独立的读取器不会抢走彼此的增量；偏移量滑出内存尾部窗口的读取标记为 `lossy`，并在 spill 文件存在时指向它。收集到的输出在结算后仍可读取。 - 终止在每个平台上都以进程树为范围（POSIX 用 detached 进程组并以直接子进程回退；Windows 用 `taskkill /T`）：`terminate()`（唯一的终止动词）执行 SIGTERM→宽限期→SIGKILL 升级（幂等，也由 spec 的 abort 信号驱动，进程树消亡后为空操作）；`waitForExit(signal?)` 观察整棵进程树的存活状态，使消费方自有的拆卸阶梯能在真正完全停稳后才进入下一层。管理器只响应中止，但绝不判定原因（deadline、拆卸阶梯与原因分类归调用方所有）。 -- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的环境清理定义：环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃，spec 的显式 `env` 在清理后合并且不做命名空间校验——字符串会有意转发或覆盖某个值，而 `undefined` tombstone 则会删除普通的环境条目。无法把 spawn 路由到该服务的进程启动方（node-pty 后端、由 SDK 管理的传输层）会导入该环境清理定义。 +- `spawnTerminal(spec)` 是唯一的非管道原语。其句柄负责真实 PTY、有效 UTF-8 字节 I/O、前台进程组检查／信号发送、TERM→KILL 全会话清理，以及等待完全停稳。顶层进程退出后，输出流会在排完队列中的输出后结束；存活期间的传输故障会拒绝 `done`。这些操作仍属于一项基底原语，因为普通管道无法分配控制终端，也无法证明并清理完整的终端会话；就绪检测、scrollback 与所有者策略仍归 PTY 消费方所有。 +- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` 是唯一一份共享的凭据清除定义：环境中形似凭据的名称与 `DSH_*` 名称都会被丢弃，显式 `env` 在清除之后合并。本地普通 spawn 与终端 spawn 都应用这一定义；自行拥有 spawn 的 SDK 管理传输层可以直接导入它。 - 服务自身的 dispose（资源释放）会终止所有仍在运行的受管进程并等待其退出。 -参见[子进程数据结构目录](../../../docs/core-data-structures/subprocess.md)与[seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 +参见[进程管理器数据结构目录](../../../docs/core-data-structures/subprocess.md)与 [seam Agent Note（agent 决策记录）](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md)。 ## 模型体验 -通过消费方 seam 间接影响（目前是 `dsh-tool-bash` 背后的 bash 执行器家族）；进程输出和生命周期的全部面向模型渲染均由消费方负责。 +通过消费方 seam 间接影响（目前是 `dsh-tool-bash` 背后的 bash 执行器家族）；进程输出与生命周期面向模型的全部渲染归消费方所有。 #### KV Cache 影响 -不会直接导致 KV Cache 失效；请求前缀变更由上述消费方负责。 +不会直接失效；请求前缀变更由具名消费方负责。 ## 已知限制与暂缓事项 -- **node-pty 与由 SDK 管理的 spawn 只共享环境清理**：PTY 后端的终端 fork 与 MCP SDK 自己的 stdio 传输层无法把 spawn 路由到这道 seam（fork/spawn 调用归库所有）；它们改为导入 `scrubbedParentEnv`，使环境策略保持单一来源。 -- **拆卸阶梯归消费方所有**：该 seam 只提供信号动词与进程树存活等待，不提供现成的停稳序列；每个进程外消费方自行编码其子进程的配合方式（ACP 后端以 stdin EOF 打头的阶梯是仓库内模板）。 +- **由 SDK 管理的 spawn 仍在服务之外**：自行拥有内部 spawn 的 SDK 传输层无法经该服务路由这次调用；它仍可导入 `scrubbedParentEnv`，使环境策略保持单一来源。 +- **拆卸阶梯归消费方所有**：该 seam 只提供信号动词与进程树存活等待，不提供现成的停稳序列；每个进程外消费方自行编码其子进程的配合形状（ACP 后端以 stdin EOF 打头的阶梯是仓库内模板）。 diff --git a/packages/subprocess/subprocess/src/index.ts b/packages/subprocess/subprocess/src/index.ts index d484faa24d..3c58a07a9e 100644 --- a/packages/subprocess/subprocess/src/index.ts +++ b/packages/subprocess/subprocess/src/index.ts @@ -1,10 +1,9 @@ /** - * The subprocess seam (`ctx.subprocess`): spawn fully-specified commands into - * managed process trees with Node-shaped stdio dispositions — raw pipes for - * protocol streams, inherit for diagnostics, bounded spill-backed collection - * for batch output — plus tree-scoped signalling. Command defaulting, shell - * semantics, deadlines, teardown ladders, framing, and presentation belong to - * consumers; the bash executor seam is the owning template. The local implementation lives in + * The subprocess seam (`ctx.subprocess`): execution-world process coordinates, + * executable lookup, fully specified managed process trees with raw or + * collected stdio, and one terminal-process primitive. Command defaulting, + * shell semantics, deadlines, protocol framing, terminal readiness, and + * presentation belong to consumers. The local implementation lives in * `@deepseek-ai/dsh-subprocess-local`. * @module @deepseek-ai/dsh-subprocess */ @@ -12,6 +11,7 @@ import { Context, Service } from 'cordis' import { DSH_ENV_PREFIX } from './types.ts' import type { SubprocessHandle, SubprocessSpawnSpec } from './types.ts' +import type { SubprocessTerminalHandle, SubprocessTerminalSpawnSpec } from './types.ts' export { DSH_ENV_PREFIX } from './types.ts' export type { @@ -28,6 +28,10 @@ export type { SubprocessSpawnSpec, SubprocessStdinMode, SubprocessStdio, + SubprocessTerminalForeground, + SubprocessTerminalHandle, + SubprocessTerminalSignal, + SubprocessTerminalSpawnSpec, } from './types.ts' /** @@ -74,6 +78,8 @@ declare module 'cordis' { * duplicate-service behavior). * * Implementations must honor these semantics: + * - {@link cwd}, {@link runtimeRoot}, and executable paths belong to one + * execution world shared with the mounted filesystem provider. * - {@link spawn} returns immediately with a live handle; `done` resolves at * process close with exit facts and rejects only for spawn-level failures. * - Collect-mode readers are offset-based and non-consuming, so independent @@ -87,12 +93,37 @@ declare module 'cordis' { * quiescence. * - Disposal of the service terminates all still-running managed processes * and awaits their exit. + * - {@link spawnTerminal} owns terminal allocation, byte transport, + * foreground groups, signalling, and whole-session quiescence; readiness + * and persistent-shell policy stay in the PTY consumer. Its output stream + * ends after queued terminal output when the top-level process exits. */ export abstract class SubprocessService extends Service { constructor(ctx: Context) { super(ctx, 'subprocess') } + /** Canonical default cwd in this provider's execution world. */ + abstract readonly cwd: string + + /** Private directory for runtime artifacts in this provider's execution world. */ + abstract readonly runtimeRoot: string + + /** + * Resolve one configured executable in this provider's execution world. + * Absolute paths are verified; bare names use the provider's scrubbed PATH + * plus explicit environment overrides. + * @param command - absolute executable path or bare PATH name. + * @param env - explicit environment entries used for lookup. + * @param signal - aborts remote or local lookup. + * @returns a canonical executable path. + */ + abstract resolveExecutable( + command: string, + env?: Readonly>, + signal?: AbortSignal, + ): Promise + /** * Start one managed child process from a fully-specified spec; this seam * applies no defaults. @@ -100,6 +131,15 @@ export abstract class SubprocessService extends Service { * @returns the live process handle (streams/readers, signalling, outcome promise). */ abstract spawn(spec: SubprocessSpawnSpec): SubprocessHandle + + /** + * Allocate a real terminal and start one owned process session. This is the + * only non-pipe process primitive: implementations own terminal byte I/O, + * foreground groups, signals, and complete session-tree cleanup. + * @param spec - fully specified argv, cwd, environment, dimensions, grace, and cancellation. + * @returns the live terminal handle after allocation succeeds. + */ + abstract spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise } export default SubprocessService diff --git a/packages/subprocess/subprocess/src/types.ts b/packages/subprocess/subprocess/src/types.ts index 6084cc8c8e..206104ad5b 100644 --- a/packages/subprocess/subprocess/src/types.ts +++ b/packages/subprocess/subprocess/src/types.ts @@ -192,3 +192,71 @@ export interface SubprocessHandle { */ waitForExit(signal?: AbortSignal): Promise } + +/** Signals supported by the terminal-process primitive. */ +export type SubprocessTerminalSignal = 'SIGINT' | 'SIGTERM' | 'SIGKILL' | 'SIGTSTP' | 'SIGHUP' + +/** A fully specified terminal-process spawn. */ +export interface SubprocessTerminalSpawnSpec { + /** Executable and arguments; `argv[0]` is the program. */ + argv: readonly string[] + /** Working directory in this subprocess provider's execution world. */ + cwd: string + /** Explicit environment layered after the provider's ambient scrub. */ + env?: Record | undefined + /** Initial terminal row count. */ + rows: number + /** Initial terminal column count. */ + cols: number + /** TERM-to-KILL cleanup grace for the complete terminal session. */ + graceMs: number + /** Cancellation of setup or the live terminal session. */ + signal?: AbortSignal | undefined +} + +/** Current foreground process-group facts for one terminal. */ +export interface SubprocessTerminalForeground { + /** Foreground process-group id published by the terminal driver. */ + processGroupId: number + /** Whether the provider can currently prove that group is waiting on terminal input. */ + inputWaiting: boolean +} + +/** + * One live terminal process and its owned OS session. Terminal allocation, + * foreground-group inspection/signalling, and session-tree cleanup are one + * deep subprocess primitive because none can be reconstructed from ordinary + * piped stdio without substrate-specific process control. + */ +export interface SubprocessTerminalHandle { + /** Top-level terminal process id. */ + readonly pid: number + /** UTF-8 terminal output bytes in delivery order; ends after queued output when the terminal exits. */ + readonly output: Readable + /** Resolves when the top-level process exits; rejects only for a live transport failure. */ + readonly done: Promise + /** + * Write bytes to the terminal input. + * @param data - valid UTF-8 bytes to deliver without implicit newline conversion. + */ + write(data: Uint8Array): Promise + /** + * Inspect the current foreground process group. + * @returns its id and input-wait fact, or undefined when no foreground group can be resolved. + */ + inspectForeground(): Promise + /** + * Deliver a signal to the current foreground process group. + * @param signal - permitted terminal signal. + * @returns the exact group id that received it. + */ + signalForeground(signal: SubprocessTerminalSignal): Promise + /** Begin idempotent TERM-to-KILL cleanup of the complete terminal session. */ + terminate(): void + /** + * Await whole-session quiescence, not only top-level process exit. + * @param signal - optional bound for this wait. + * @returns true after quiescence, false when `signal` aborts first. + */ + waitForExit(signal?: AbortSignal): Promise +} diff --git a/packages/subprocess/subprocess/tests/service.spec.ts b/packages/subprocess/subprocess/tests/service.spec.ts index d0ef5c9fd6..853b333400 100644 --- a/packages/subprocess/subprocess/tests/service.spec.ts +++ b/packages/subprocess/subprocess/tests/service.spec.ts @@ -1,7 +1,14 @@ import { describe, expect, it } from 'vitest' +import { PassThrough } from 'node:stream' import { Context } from 'cordis' import { scrubbedParentEnv, SubprocessService } from '@deepseek-ai/dsh-subprocess' -import type { SubprocessHandle, SubprocessOutputRead, SubprocessSpawnSpec } from '@deepseek-ai/dsh-subprocess' +import type { + SubprocessHandle, + SubprocessOutputRead, + SubprocessSpawnSpec, + SubprocessTerminalHandle, + SubprocessTerminalSpawnSpec, +} from '@deepseek-ai/dsh-subprocess' /** * Minimal concrete service: a hand-built handle. The seam is spawn-only — @@ -9,6 +16,13 @@ import type { SubprocessHandle, SubprocessOutputRead, SubprocessSpawnSpec } from * is all an implementation owes the abstract class. */ class StubSubprocessService extends SubprocessService { + readonly cwd = '/stub' + readonly runtimeRoot = '/stub/.runtime' + + async resolveExecutable(command: string): Promise { + return `/bin/${command}` + } + spawn(spec: SubprocessSpawnSpec): SubprocessHandle { const read: SubprocessOutputRead = { text: '', nextOffset: 0, lossy: false } const collected = spec.stdio.stdout !== 'pipe' && spec.stdio.stdout !== 'inherit' @@ -25,6 +39,19 @@ class StubSubprocessService extends SubprocessService { waitForExit: () => Promise.resolve(true), } } + + async spawnTerminal(spec: SubprocessTerminalSpawnSpec): Promise { + return { + pid: spec.argv.length, + output: new PassThrough(), + done: Promise.resolve({ exitCode: 0, signal: null }), + write: async () => {}, + inspectForeground: async () => ({ processGroupId: 1, inputWaiting: true }), + signalForeground: async () => 1, + terminate: () => {}, + waitForExit: async () => true, + } + } } describe('SubprocessService seam', () => { diff --git a/scripts/check-workspace-constraints.ts b/scripts/check-workspace-constraints.ts index 355b817e68..b20d5b5f3b 100644 --- a/scripts/check-workspace-constraints.ts +++ b/scripts/check-workspace-constraints.ts @@ -148,6 +148,7 @@ function expectedDshPackageFiles(manifest: PackageManifest): readonly string[] { 'lib/invariant.js', ...manifest.bin ? ['lib/bin.js'] : [], ...manifest.exports?.['./worker'] ? ['lib/worker.cjs'] : [], + ...exportDefault(manifest, './runtime-host') === './lib/runtime-host.js' ? ['lib/runtime-host.js'] : [], // UI plugin packages ship their browser bundle beside the node lib // (single-artifact ruling: dist/ retired, ./client resolves lib/client.js). // Keyed on the artifact path, not the subpath name: apiproxy's ./client is diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index 6d9665236a..256ae90caa 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -73,6 +73,8 @@ export const LINK_MAP: Readonly> = { SubprocessOutputRead: 'subprocess.md', SubprocessOutputReader: 'subprocess.md', SubprocessSpawnSpec: 'subprocess.md', + SubprocessTerminalHandle: 'subprocess.md', + SubprocessTerminalSpawnSpec: 'subprocess.md', CodeRunRequest: 'code-runtime.md', CodeRunResult: 'code-runtime.md', CompactionResult: 'compaction.md', @@ -242,6 +244,7 @@ export const FOUNDATION_TYPE_NAMES: ReadonlySet = new Set([ 'Partial', 'Pick', 'Promise', + 'Record', 'Readonly', ]) diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index 77432e7dbe..74a09a874e 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -623,11 +623,12 @@ function builtBinSmokeGate(needs: string[] = ['build']): Gate { 'packages/subagent/subagent-codex/tests/loader-composition.e2e.ts', 'packages/subagent/subagent-claude-code/tests/loader-composition.e2e.ts', 'packages/api/remotes/tests/built-lib.e2e.ts', - // The worker-entry packages' built bundles: the only automated proof - // that lib/index.js resolves its sibling lib/worker.cjs under plain node - // (the e2e lane runs unbuilt, so these files self-skip there). + // Built execution consumers: the only automated proof that package-name + // imports reach their lib/ entrypoints under plain Node. The e2e lane runs + // unbuilt, so these files self-skip there. 'packages/workflow/workflow-workerthread/tests/built-worker.e2e.ts', 'packages/code-runtime/code-runtime-worker/tests/built-lib.e2e.ts', + 'packages/lsp/lsp-local/tests/built-lib.e2e.ts', ], { label: 'built-bin smoke', needs, diff --git a/scripts/verify-package-readme-model-experience.ts b/scripts/verify-package-readme-model-experience.ts index 3ca0317cf4..9a249ca844 100644 --- a/scripts/verify-package-readme-model-experience.ts +++ b/scripts/verify-package-readme-model-experience.ts @@ -47,6 +47,7 @@ const SENTENCE_MODEL_EXPERIENCE: Readonly> = { 'packages/bash/bash-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-bash.' }, 'packages/bash/pwsh-local': { kind: 'indirect', reason: 'The executor backend delegates model rendering to dsh-tool-pwsh.' }, 'packages/code-runtime/code-runtime': { kind: 'indirect', reason: 'The service interface delegates model rendering to Code Mode in dsh-tools.' }, + 'packages/code-runtime/code-runtime-subprocess': { kind: 'indirect', reason: 'The subprocess backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/code-runtime/code-runtime-worker': { kind: 'indirect', reason: 'The worker backend delegates model rendering to Code Mode in dsh-tools.' }, 'packages/typert/registry': { kind: 'none', reason: 'Runtime type registry; consumers (cordis_inspect, wire faces, gates) own any model-visible projection of registry contents.' }, 'packages/typert/loader': { kind: 'none', reason: 'Loader integration only registers generated artifacts; consumers own any model-visible projection.' }, diff --git a/tsconfig.host.json b/tsconfig.host.json index 1a8ef6508a..6252733511 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -169,6 +169,7 @@ { "path": "./packages/pty/tool-pty" }, { "path": "./packages/code-runtime/code-runtime" }, { "path": "./packages/code-runtime/code-runtime-worker" }, + { "path": "./packages/code-runtime/code-runtime-subprocess" }, { "path": "./packages/llm/llm-deepseek" }, { "path": "./packages/llm/llm-pi-ai" }, { "path": "./packages/bash/bash-local" },